feat(serve): support fine-tuned models in deployment-config API#6041
Open
Lokiiiiii wants to merge 1 commit into
Open
feat(serve): support fine-tuned models in deployment-config API#6041Lokiiiiii wants to merge 1 commit into
Lokiiiiii wants to merge 1 commit into
Conversation
Lokiiiiii
force-pushed
the
select-recipe-hosting-config-by-instance-type
branch
from
July 17, 2026 15:55
0b762fc to
ab5d838
Compare
Lokiiiiii
force-pushed
the
select-recipe-hosting-config-by-instance-type
branch
from
July 17, 2026 16:30
ab5d838 to
877aea5
Compare
Lokiiiiii
force-pushed
the
select-recipe-hosting-config-by-instance-type
branch
from
July 17, 2026 17:26
877aea5 to
1c7b1fc
Compare
Lokiiiiii
force-pushed
the
select-recipe-hosting-config-by-instance-type
branch
from
July 17, 2026 17:42
1c7b1fc to
8556ed8
Compare
Lokiiiiii
force-pushed
the
select-recipe-hosting-config-by-instance-type
branch
from
July 17, 2026 18:05
8556ed8 to
142a2b6
Compare
Lokiiiiii
force-pushed
the
select-recipe-hosting-config-by-instance-type
branch
from
July 17, 2026 18:33
142a2b6 to
4a55e6f
Compare
Lokiiiiii
marked this pull request as ready for review
July 17, 2026 18:37
Make ModelBuilder.list_deployment_configs / get_deployment_config / set_deployment_config work for fine-tuned (model-customization) models in addition to base/JumpStart models, dispatching internally on _is_model_customization(). For fine-tuned models the recipe's published HostingConfigs are the source of truth; selection is by instance type (recipe configs are largely unnamed). The base "deployment config" vs recipe "hosting config" distinction is internal and does not surface: fine-tuned configs are normalized to a shape compatible with the base response (DeploymentConfigName plus a nested DeploymentArgs block, plus BenchmarkMetrics/AccelerationConfigs and an additive IsDefault flag). The normalized fine-tuned shape always populates the DeploymentArgs keys (None when unset); the base response may omit unset keys (its serializer drops empty slots), so the fine-tuned shape is a superset — callers consuming both pathways should use .get() for the optional keys (documented on the methods). set_deployment_config applies the whole matching config (image, environment, compute requirements) at build time. A caller-provided image_uri still takes precedence over the config's ImageUri (documented). Both branches fail fast on bad input: the fine-tuned branch requires instance_type and raises on an unpublished or ambiguous instance; the base branch requires config_name AND instance_type and now also rejects an unpublished config name or an instance the config does not support (validated against JumpStart metadata) instead of silently recording a no-op selection. Instance-type matching honors the config's full offered set, not only its default. A base config is a multi-instance bundle: list_deployment_ configs(instance_type=X) filters against each config's supported-instance metadata (supported_inference_instance_types) and materializes matched configs FOR X, so a config that supports X but defaults to another instance is neither discarded nor materialized at the wrong instance. Recipe hosting configs are per-instance bundles by contract, but SupportedInstanceTypes, when present, is honored for selection and filtering. list/get/set agree for the same selection: get_deployment_ config() materializes the pinned instance (matching what list returns and what build deploys), including when the pinned instance came from SupportedInstanceTypes (differs from the config's default). The pinned instance is preserved through build. Configs published with only DefaultInstanceType are matched consistently end to end. The build env merge tolerates a config that publishes an explicit null Environment. Config discovery (recipe-level with a top-level HostingConfigs fallback) is centralized in _extract_hosting_configs_from_hub() and used by BOTH the selection API and the build path, so a top-level config that is listable/selectable is also applied at build. Nova models are routed to their dedicated build path first (per-tier SMI validation + Nova env precedence), so the top-level fallback never diverts them. An explicit selection stores a deep copy of the raw config, is applied exactly at build (or raises if no longer published), and returned configs are copied so caller mutation cannot corrupt internal state. This is the SDK half of a paired change; the SageMaker agent model-deployment skill consumes this unified API. Internal CR: https://code.amazon.com/reviews/CR-289903643
Lokiiiiii
force-pushed
the
select-recipe-hosting-config-by-instance-type
branch
from
July 17, 2026 22:01
4a55e6f to
ed67e3d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Make
ModelBuilder.list_deployment_configs/get_deployment_config/set_deployment_configwork for fine-tuned (model-customization) models in addition to base/JumpStart models, dispatching internally on_is_model_customization(). For fine-tuned models the recipe's publishedHostingConfigsare the source of truth; selection is by instance type (recipe configs are largely unnamed).The base "deployment config" vs recipe "hosting config" distinction stays internal: fine-tuned configs are normalized to the same shape the base response returns (
DeploymentConfigName+ a nestedDeploymentArgsblock withImageUri/InstanceType/Environment/ComputeResourceRequirementsand the other baseDeploymentArgskeys, present asNonewhen a recipe doesn't provide them), so callers iterate either pathway identically.BenchmarkMetrics/AccelerationConfigsareNone(recipes publish none today); an additiveIsDefaultflag marks the recipe's Default config.set_deployment_config(instance_type=...)applies the whole matching config (image, env, compute requirements) at build time and raises on an unpublished/ambiguous instance type.list_deployment_configs(instance_type=...)filters to that instance. Configs published with onlyDefaultInstanceTypeare matched consistently end-to-end so they aren't silently dropped at build.Paired dependency
This is the SDK half of a paired change. The consumer is an Amazon-internal SageMaker agent model-deployment skill tracked in internal review CR-289903643 (
code.amazon.com/reviews/CR-289903643, not publicly accessible). The two are mutually dependent and are intended to land together.Testing
python3 -m pytest sagemaker-serve/tests/unit/test_recipe_hosting_config_selection.py(31 tests) plus the base-model regression suites (coverage boost, instance-type inference, nova hosting config, compute requirements) — 102 passed. Includes a shape-parity test tied to the baseDeploymentArgs/DeploymentConfigMetadatadataclass slots so it fails if base ever adds a field the fine-tuned shape doesn't mirror.