Sev1: Azure OpenAI deployment deleted or renamed. Every call returns 404 DeploymentNotFound

#Symptoms

• Applications log 404 DeploymentNotFound: The API deployment for this resource does not exist.
• Some features work (other deployments), the ones using one deployment name fail.
• Often right after a portal cleanup, a model upgrade done by "delete and recreate", or a Terraform/Bicep run.

#Step 1: confirm (2 minutes)

az cognitiveservices account deployment list -g rg-ai -n aoai-prod \
  --query "[].{name:name, model:properties.model.name, version:properties.model.version, sku:sku.name, capacity:sku.capacity, state:properties.provisioningState}" -o table

Compare with the name in the app configuration. Typical findings:

Who did it:

az monitor activity-log list -g rg-ai --offset 2d \
  --query "[?contains(operationName.value, 'deployments')].{t:eventTimestamp, op:operationName.value, who:caller, status:status.value}" -o table

#Step 2: recreate with the same name

The fastest fix is a deployment with the exact name the apps use. Recreate it with the previous model, version and SKU (get them from IaC, documentation or the activity log's request body):

az cognitiveservices account deployment create -g rg-ai -n aoai-prod \
  --deployment-name gpt-4o-prod \
  --model-format OpenAI --model-name gpt-4o --model-version "2024-11-20" \
  --sku-name GlobalStandard --sku-capacity 300

--sku-capacity is in units of 1,000 tokens per minute for standard deployments (or PTUs for provisioned). Check the CLI help for your version.

Warning

Quota is shared per subscription, region and model. If the old deployment's capacity was reassigned (or another team took it), creation fails or you get less capacity. Check before recreating: az cognitiveservices usage list -l swedencentral -o table

If the exact model version is no longer available for new deployments (retired or retiring), deploy the closest current version and test critical prompts quickly; output behaviour may change.

#Provisioned (PTU) deployments

Recreating a PTU deployment needs available PTU quota and regional capacity. If capacity is not available immediately:

• Create a Standard or Global Standard deployment with the same name as a stop-gap (higher latency variance, pay-per-token).
• Raise a support request for PTU capacity, referencing the deleted deployment.

#Step 3: if apps were switched to a new name

If someone recreated it under a new name and some apps were already updated, you now have both names in use. Short term, create both names (if quota allows) so every app works. Then standardise on one and update configuration centrally.

Better long term: route through a gateway (API Management) so applications call a stable path and the gateway maps it to a deployment.

<!-- APIM policy fragment: map a stable model alias to the current deployment -->
<set-backend-service base-url="https://aoai-prod.openai.azure.com/openai/deployments/gpt-4o-prod" />

#Step 4: restore settings that were not in the deployment

Recreating restores the model, not everything around it. Check:

• Content filter / RAI policy assigned to the deployment (custom policies must be reattached).
• Rate limit (TPM) and dynamic quota settings.
• Version upgrade policy (versionUpgradeOption): auto-upgrade or pinned.
• Diagnostic settings and alerts that referenced the deployment name.

az cognitiveservices account deployment show -g rg-ai -n aoai-prod --deployment-name gpt-4o-prod \
  --query "{model:properties.model, rai:properties.raiPolicyName, upgrade:properties.versionUpgradeOption, sku:sku}"

#Step 5: confirm

curl -s -o /dev/null -w "%{http_code}\n" \
  "https://aoai-prod.openai.azure.com/openai/deployments/gpt-4o-prod/chat/completions?api-version=2024-10-21" \
  -H "Authorization: Bearer $(az account get-access-token --resource https://cognitiveservices.azure.com --query accessToken -o tsv)" \
  -H "Content-Type: application/json" -d '{"messages":[{"role":"user","content":"ping"}],"max_tokens":5}'

• Returns 200; application error rate back to baseline.
• Capacity matches what you had (no new 429s at peak).
• Regression check on key prompts if the model version changed.

#Prevent it next time

• Deployments defined in Bicep or Terraform; changes only through the pipeline.

resource deployment 'Microsoft.CognitiveServices/accounts/deployments@2024-10-01' = {
  parent: aoai
  name: 'gpt-4o-prod'
  sku: { name: 'GlobalStandard', capacity: 300 }
  properties: {
    model: { format: 'OpenAI', name: 'gpt-4o', version: '2024-11-20' }
    versionUpgradeOption: 'NoAutoUpgrade'
    raiPolicyName: 'prod-content-filter'
  }
}

• CanNotDelete resource lock on production AI resources (it also blocks deleting child deployments).
• Model upgrades by deploying the new version under a new name, testing, then switching the gateway alias, never by deleting the live deployment.
• Alert on Activity Log delete operations for deployments.