Home / Azure

AzureRunbookIntermediate

Sev1: production broken after an App Service slot swap — roll back in minutes

Roll back a bad release on Azure App Service by swapping slots back, understand why sticky settings and warm-up cause post-swap failures, and make swaps safe with preview and health checks.

Deepak Bhardwaj2 min read

Applies toAzure App Service (Standard tier and above) and Azure Functions on Premium/Dedicated plans

Symptoms

  • Errors or a 5xx spike right after a swap from staging to production.
  • The app connects to the wrong database or uses staging feature flags.
  • First requests after the swap are very slow or time out.

1. Roll back first, investigate second

After a swap, the previous production build is sitting in the staging slot. Swap again:

az webapp deployment slot swap -g rg-app -n app-prod --slot staging --target-slot production

This usually takes a minute or two and restores the last known good version. Confirm with the health endpoint and the error rate before you start investigating.

Warning

Do not deploy anything to the staging slot until you have rolled back. A deployment there overwrites your rollback copy.

2. Why it broke

Symptom Likely cause
Production uses staging DB or keys Setting not marked as deployment slot setting (sticky)
Works in staging, fails in production Production-only setting missing; sticky value differs
Slow or failing for the first minutes App not warmed up; swap completed before init finished
Only some instances fail Instance-local state or in-progress requests during swap

Compare settings across slots:

az webapp config appsettings list -g rg-app -n app-prod -o table
az webapp config appsettings list -g rg-app -n app-prod --slot staging -o table
az webapp config connection-string list -g rg-app -n app-prod -o table

Mark a setting sticky:

az webapp config appsettings set -g rg-app -n app-prod --slot-settings ConnectionStrings__Main="@Microsoft.KeyVault(SecretUri=...)"

3. Make the next swap safe

Swap with preview. Phase 1 applies production settings to the staging slot and restarts it, so you can test the real configuration before traffic moves.

az webapp deployment slot swap -g rg-app -n app-prod --slot staging --action preview
# test https://app-prod-staging.azurewebsites.net with production settings
az webapp deployment slot swap -g rg-app -n app-prod --slot staging --action swap
# or abandon
az webapp deployment slot swap -g rg-app -n app-prod --slot staging --action reset

Warm-up. Add an applicationInitialization section (Windows) or set WEBSITE_SWAP_WARMUP_PING_PATH and WEBSITE_SWAP_WARMUP_PING_STATUSES so the swap waits for a healthy response.

Health check. Enable the App Service health check path so unhealthy instances are taken out of rotation.

Prevent it next time

  • Treat "is this setting sticky?" as part of code review for every new setting.
  • Use preview swaps for production and automate a smoke test between phases.
  • Keep the rollback swap command in the release runbook, with who may run it.

Related fixes