Home / Azure

AzureTroubleshootingAdvanced

App Service Key Vault references not resolving: the app sees the literal @Microsoft.KeyVault(...) string

An App Service or Function app setting that points at Key Vault returns the raw reference text instead of the secret. Check identity, Key Vault access model, network path and the reference syntax, then force a refresh.

Deepak Bhardwaj4 min read

Applies toAzure App Service, Azure Functions (Windows and Linux), Azure Key Vault (RBAC or access policies), system- and user-assigned managed identity

Symptoms

  • The app reads a setting and gets @Microsoft.KeyVault(SecretUri=https://...) as the value, not the secret.
  • Database connections fail right after you moved a connection string into Key Vault.
  • In the portal, under Environment variables (or Configuration), the setting shows a red X in the Source column instead of a green tick.

When App Service can't resolve a reference, it passes the reference string through unchanged. Nothing crashes at startup; the app just gets the wrong value.

Step 1: Read the resolution status

The portal shows the reason when you click the setting. From the CLI you can query the same information:

az rest --method get \
  --url "https://management.azure.com/subscriptions/<sub>/resourceGroups/rg-web/providers/Microsoft.Web/sites/app-prod/config/configreferences/appsettings?api-version=2022-03-01" \
  --query "value[].{name:name, status:properties.status, detail:properties.details}" -o table

Typical statuses: Resolved, AccessToKeyVaultDenied, SecretNotFound, VaultNotFound, IdentityNotFound, OtherReasons.

Step 2: Check the reference syntax

Two valid forms:

@Microsoft.KeyVault(SecretUri=https://kv-prod-app01.vault.azure.net/secrets/SqlConnection/)
@Microsoft.KeyVault(VaultName=kv-prod-app01;SecretName=SqlConnection)

Watch for:

  • A version GUID at the end of SecretUri pins that version forever. Leave it off to always get the latest.
  • Smart quotes or trailing spaces pasted from a document.
  • Wrong vault DNS suffix in sovereign clouds.

Step 3: Check the identity

az webapp identity show -g rg-web -n app-prod
  • No system-assigned identity and no user-assigned identity → turn one on: az webapp identity assign -g rg-web -n app-prod.
  • Using a user-assigned identity? App Service uses the system-assigned identity for Key Vault references by default. Tell it which one to use:
UAI_ID=$(az identity show -g rg-identity -n uai-app-prod --query id -o tsv)
az webapp update -g rg-web -n app-prod --set keyVaultReferenceIdentity=$UAI_ID

Deployment slots have their own system-assigned identity. A reference that works in production can fail in the staging slot.

Step 4: Check Key Vault permissions

Find which model the vault uses:

az keyvault show -n kv-prod-app01 --query "properties.enableRbacAuthorization"

RBAC (true): grant Key Vault Secrets User on the vault (or the single secret):

PRINCIPAL=$(az webapp identity show -g rg-web -n app-prod --query principalId -o tsv)
KV_ID=$(az keyvault show -n kv-prod-app01 --query id -o tsv)
az role assignment create --assignee-object-id $PRINCIPAL --assignee-principal-type ServicePrincipal \
  --role "Key Vault Secrets User" --scope $KV_ID

Access policies (false): grant get on secrets:

az keyvault set-policy -n kv-prod-app01 --object-id $PRINCIPAL --secret-permissions get

Tip

RBAC role assignments can take several minutes to apply. Don't rotate three other settings while you wait; change one thing, then refresh (Step 6).

Step 5: Check the network path

If the vault has public network access disabled or a firewall:

Vault setting What App Service needs
Private endpoint only VNet integration on the app, and the app must resolve kv-prod-app01.vault.azure.net to the private IP (private DNS zone privatelink.vaultcore.azure.net linked to the integration VNet)
Firewall with "Allow trusted Microsoft services" This does not cover App Service Key Vault references; use VNet integration + private endpoint or a service endpoint
Selected networks with service endpoint Integration subnet must have Microsoft.KeyVault service endpoint and be allowed on the vault

For apps with VNet integration, make reference resolution go through the VNet:

az webapp config set -g rg-web -n app-prod --vnet-route-all-enabled true

Check DNS from the app's own console (Kudu / SSH):

nslookup kv-prod-app01.vault.azure.net

A public IP means the private DNS zone isn't linked to the integration VNet, or your custom DNS servers don't forward privatelink.vaultcore.azure.net.

Step 6: Force a refresh

App Service caches resolved values and re-fetches them about every 24 hours, and on restart or config change. After fixing permissions or network:

az webapp restart -g rg-web -n app-prod

Or trigger a refresh without a full restart by changing any app setting (some teams keep a KV_REFRESH setting just for this).

Symptom → fix table

Status Fix
IdentityNotFound Enable identity, or set keyVaultReferenceIdentity for user-assigned
AccessToKeyVaultDenied Secrets User role (RBAC) or get policy; check network
SecretNotFound Typo in name, secret disabled or expired, wrong vault
VaultNotFound Wrong vault name, or DNS resolution fails over VNet
Works in prod, fails in slot Grant the slot's identity too

Prevent it next time

  • Grant Key Vault access in the same IaC module that creates the app and every slot.
  • Use a user-assigned identity shared by production and slots, so swaps don't change who reads secrets.
  • Add a startup health check that fails if a setting still starts with @Microsoft.KeyVault(.
  • Alert on Key Vault Forbidden responses in diagnostics (AzureDiagnostics | where ResultSignature == "Forbidden").

Related fixes