Runbook: rotate Azure OpenAI API keys without downtime, then switch apps to Entra ID

#When to run this

• A key was committed to a repo, pasted in a ticket or shown in a screenshot.
• Your policy says rotate every N days.
• Someone who had keys left the team.
• A security finding says "key access should be disabled".

Each Azure OpenAI account has two keys, key1 and key2, and both are valid at the same time. Zero-downtime rotation means you move every client to the key you're not regenerating, then regenerate. The long-term fix is to stop using keys at all.

Warning

Regenerating a key invalidates it immediately for new requests. Any client still using it gets 401 Access denied due to invalid subscription key or wrong API endpoint. Do the inventory in step 1 first, unless the key is actively being abused, in which case regenerate now and accept the outage.

#1. Inventory who uses the keys

Keys don't carry an identity, so the service can't tell you which app sent a key-based request. Work from the other side: who reads the keys.

// Log Analytics workspace with the Activity log connected
AzureActivity
| where TimeGenerated > ago(90d)
| where OperationNameValue =~ "MICROSOFT.COGNITIVESERVICES/ACCOUNTS/LISTKEYS/ACTION"
| where _ResourceId has "<account-name>"
| summarize count(), last_seen = max(TimeGenerated) by Caller, CallerIpAddress
| order by last_seen desc

Then search where the key is stored:

# App Service / Functions settings that look like an OpenAI key or endpoint
az webapp config appsettings list -g rg-app -n app-orders \
  --query "[?contains(name, 'OPENAI')].{name:name, value:value}" -o table

# Key Vault secrets
az keyvault secret list --vault-name kv-prod --query "[].name" -o tsv

Also check AKS secrets, pipeline variables, APIM named values and developer laptops (.env files). Write the list down; you'll use it twice.

#2. Rotate with the key1/key2 swap

Assume clients use key1 today.

#Move clients to key2

RG=rg-ai; ACCT=aoai-prod-eus
KEY2=$(az cognitiveservices account keys list -n $ACCT -g $RG --query key2 -o tsv)

# Best case: one Key Vault secret that every app references
az keyvault secret set --vault-name kv-prod --name aoai-api-key --value "$KEY2" --output none

App Service and Functions resolve Key Vault references without a version on a schedule (up to about 24 hours), so restart to pick up the new value now:

az webapp restart -g rg-app -n app-orders

For AKS, update the secret and restart the deployment:

kubectl -n orders create secret generic aoai --from-literal=api-key="$KEY2" \
  --dry-run=client -o yaml | kubectl apply -f -
kubectl -n orders rollout restart deployment/orders-api
kubectl -n orders rollout status deployment/orders-api

#Verify, then regenerate key1

Confirm every client in your inventory is healthy on key2 (no 401s in app logs, normal request count on the account's Requests metric). Then:

az cognitiveservices account keys regenerate -n $ACCT -g $RG --key-name Key1 --output none

Next rotation, do the reverse: move clients to the new key1, regenerate key2.

#If the key was leaked

Regenerate the leaked key first, then fix clients. Also regenerate the other key if the person or system that leaked one could have read both. Review the account's Requests and Processed Prompt Tokens metrics for unexpected usage (metric display names vary slightly by portal version).

#3. Switch apps to Entra ID

Keys keep coming back unless apps stop needing them. With Entra ID, each app gets a token from its managed identity and you control access with RBAC.

Note

Entra ID authentication needs the account's custom subdomain endpoint (https://<name>.openai.azure.com/ or https://<name>.cognitiveservices.azure.com/). Regional endpoints only work with keys.

#Grant the role

PRINCIPAL=$(az webapp identity assign -g rg-app -n app-orders --query principalId -o tsv)
SCOPE=$(az cognitiveservices account show -n $ACCT -g $RG --query id -o tsv)

az role assignment create --assignee-object-id $PRINCIPAL \
  --assignee-principal-type ServicePrincipal \
  --role "Cognitive Services OpenAI User" --scope $SCOPE

Cognitive Services OpenAI User allows inference calls but not key listing or deployment changes. Role assignments can take several minutes to apply.

#Test from your shell

TOKEN=$(az account get-access-token --resource https://cognitiveservices.azure.com --query accessToken -o tsv)
curl -s "https://$ACCT.openai.azure.com/openai/deployments/<deployment>/chat/completions?api-version=<api-version>" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"ping"}],"max_tokens":5}'

Use an api-version your deployment supports; check the current Azure OpenAI API version list.

#Change the code

import os
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import AzureOpenAI

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
)

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    azure_ad_token_provider=token_provider,
    api_version=os.environ["AZURE_OPENAI_API_VERSION"],
)

The token provider caches and refreshes tokens, so don't fetch a token per request yourself. Remove the api-key setting from the app once it's deployed, so a fallback to keys can't hide a broken role assignment.

#Common errors after the switch

#4. Disable key authentication

When every client in your inventory runs on Entra ID, turn keys off.

Set-AzCognitiveServicesAccount -ResourceGroupName "rg-ai" -Name "aoai-prod-eus" -DisableLocalAuth $true
(Get-AzCognitiveServicesAccount -ResourceGroupName "rg-ai" -Name "aoai-prod-eus").DisableLocalAuth

az cognitiveservices account show -n $ACCT -g $RG --query properties.disableLocalAuth

Caution

Propagation isn't instant. Microsoft documents that the gateway can keep accepting previously valid keys for a few minutes and, in some cases, several hours. Don't treat this as an immediate cutoff for a leaked key; regenerate the key as well. Confirm the change by sending a request with the old key and checking for a 401.

To roll back, run the same cmdlet with -DisableLocalAuth $false. The same propagation delay applies.

#Prevent it next time

• Assign the built-in Azure Policy Azure AI Services resources should have key access disabled (disable local authentication) at subscription or management group scope, in Audit first, then Deny.
• Give each app its own managed identity and grant Cognitive Services OpenAI User at the account scope only.
• Restrict who holds listKeys (Contributor and above). Alert on the LISTKEYS activity log query above.
• If a gateway such as APIM fronts the account, let the gateway's managed identity call Azure OpenAI and give clients gateway subscriptions instead of account keys.
• Run GitHub secret scanning or an equivalent on every repo, so the next leaked key is found in minutes rather than months.