A gateway kill switch for all LLM traffic: Azure API Management named values and policies

#Why at the gateway

If every LLM call goes through an AI gateway, the gateway is the one place where you can stop traffic:

• for all apps at once,
• for one app (product or subscription),
• for one model deployment,

regardless of how each app is written. It's the second, independent switch behind app-level feature flags.

#1. Named values as switches

az apim nv create --resource-group rg-ai --service-name apim-ai-prod \
  --named-value-id ai-global-kill --display-name ai-global-kill --value off
az apim nv create --resource-group rg-ai --service-name apim-ai-prod \
  --named-value-id ai-kill-gpt4o --display-name ai-kill-gpt4o --value off

#2. Global policy (All APIs or the AI API)

<inbound>
  <base />
  <choose>
    <when condition="@("{{ai-global-kill}}" == "on")">
      <return-response>
        <set-status code="503" reason="Service Unavailable" />
        <set-header name="Retry-After" exists-action="override"><value>600</value></set-header>
        <set-header name="Content-Type" exists-action="override"><value>application/json</value></set-header>
        <set-body>{"error":{"code":"ai_disabled","message":"AI features are temporarily disabled by the operations team."}}</set-body>
      </return-response>
    </when>
  </choose>
</inbound>

{{name}} is replaced with the named value when the policy is evaluated, so changing the value takes effect without editing the policy.

#3. Per-model switch

Inside the Azure OpenAI API, the deployment name is in the URL (/deployments/{deployment-id}/...):

<choose>
  <when condition="@("{{ai-kill-gpt4o}}" == "on" && context.Request.MatchedParameters.GetValueOrDefault("deployment-id", "") == "gpt-4o")">
    <return-response>
      <set-status code="503" reason="Service Unavailable" />
      <set-body>{"error":{"code":"model_disabled","message":"This model is temporarily disabled."}}</set-body>
    </return-response>
  </when>
</choose>

Alternative: instead of returning 503, route to a different backend (a safer or smaller model). That's a model switch, not a kill. Decide which you want per incident type.

#4. Per-application switch

If each app has its own APIM product and subscription key:

• Turn off one app: suspend its subscription (state suspended). Requests get 401.
• Or add a product-level policy with its own named value, so you can return a friendly 503.

Suspending a subscription with the REST API:

SUB=/subscriptions/<sub-id>/resourceGroups/rg-ai/providers/Microsoft.ApiManagement/service/apim-ai-prod/subscriptions/app-helpdesk
az rest --method patch --url "https://management.azure.com${SUB}?api-version=2022-08-01" \
  --body '{"properties":{"state":"suspended"}}'

Check the current API version for your tenant.

#5. Flip the switch

az apim nv update --resource-group rg-ai --service-name apim-ai-prod --named-value-id ai-global-kill --value on

Test right away:

curl -s -o /dev/null -w "%{http_code}\n" -H "api-key: $KEY" \
  "https://apim-ai-prod.azure-api.net/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21" \
  -H "Content-Type: application/json" -d '{"messages":[{"role":"user","content":"ping"}]}'
# expect 503

Named value changes usually apply within a short time. Measure it in a drill and write the number in your runbook.

#6. Close the bypass

A gateway switch only works if apps can't call the model directly:

• Disable key authentication on the Azure OpenAI resource (disableLocalAuth) and give only APIM's managed identity the Cognitive Services OpenAI User role.
• Use private endpoints, with public network access disabled on the model resource.

az resource update --ids <openai-resource-id> --set properties.disableLocalAuth=true

Caution

Before disabling local auth, confirm no app still uses API keys directly, or you cause the outage you were trying to control. Check the resource's sign-in logs and metrics by authentication type.

#7. Audit and alert

• Activity log records named value and subscription changes with the caller.
• Alert on Microsoft.ApiManagement/service/namedValues/write, so the team sees each flip.
• Log gateway responses with code ai_disabled to measure user impact.

#Checklist

• Global, per-model and per-app switches defined as named values
• Policies return a clean JSON 503 with Retry-After
• Apps can't bypass the gateway
• Flip commands in the runbook; drill done and timed
• Changes alerted and audited