Applies toAzure OpenAI / Azure AI Foundry model deployments, Amazon Bedrock model IDs, OCI Generative AI; any provider with model deprecation schedules
Symptoms
- Errors such as
DeploymentNotFound,model_not_found, "The model ... has been deprecated",ValidationException: The provided model identifier is invalid(Bedrock), or 404s from the model endpoint. - Or no errors, but output style, length or JSON format changed overnight (auto-upgrade to a new version).
- Started at a date boundary, not after a deployment of yours.
Step 1: confirm it's lifecycle, not your change
Azure OpenAI:
az cognitiveservices account deployment list -g rg-ai -n aoai-prod \
--query "[].{name:name, model:properties.model.name, version:properties.model.version, upgrade:properties.versionUpgradeOption}" -o table
Check the model's retirement date on the Azure OpenAI model retirement page, and the Activity log for automatic deployment updates.
Bedrock:
aws bedrock get-foundation-model --model-identifier anthropic.claude-3-sonnet-20240229-v1:0 \
--query "modelDetails.modelLifecycle"
LEGACY or a missing model means it's past or near end of life.
Step 2: restore service
| Situation | Fastest fix |
|---|---|
| Deployment's version retired (Azure) | Update the deployment to a supported version, or create a new deployment with the replacement model and switch the app's deployment name |
| Auto-upgraded and output changed | Pin a version: create a deployment on the version that worked (if still available) with NoAutoUpgrade |
| Bedrock model ID retired | Change the model ID in config to the replacement (or an inference profile) and redeploy config |
| Replacement needs quota you don't have | Use another region/deployment type (Global/Data Zone) temporarily; request quota |
Azure: create the replacement deployment.
az cognitiveservices account deployment create -g rg-ai -n aoai-prod \
--deployment-name chat-main-v2 --model-name gpt-4o --model-version "2024-11-20" \
--model-format OpenAI --sku-name GlobalStandard --sku-capacity 100
Check the current model names and versions available in your region; don't copy versions from old posts.
Step 3: smoke-test before full switch
The replacement model is not identical:
- Run your evaluation golden set (or at least 20–50 representative prompts).
- Check structured output / JSON parsing, tool calling, max output tokens, refusals.
- Watch token counts and cost per request: newer models may tokenise or respond differently.
If the new model breaks a critical flow, switch that flow to safe mode while you adjust prompts.
Step 4: stabilise
- Update every environment (dev/test/prod), batch jobs and secondary regions, not just the main app.
- Update the gateway routing / fallback config that references the old deployment.
Prevent it next time
| Measure | How |
|---|---|
| Lifecycle register | Table of every model/deployment in use with retirement date and owner |
| Alerts | Subscribe to provider retirement notices; Azure Service Health and Advisor show upcoming retirements |
| Pinned versions | Set versionUpgradeOption to NoAutoUpgrade (or upgrade on expiry) deliberately, not by default |
| Config, not code | Deployment names / model IDs in config, so a switch is a config change |
| Evaluation pipeline | Re-run evaluations on the replacement model months before retirement |
Warning
Auto-upgrade avoids hard failures but can change behaviour silently. Pin versions for workflows that parse model output, and upgrade them on your own schedule with tests.