Azure VM Scale Sets cheat sheet: scale, reimage, roll out changes and protect instances

Use this when you need to resize a scale set, replace a bad instance or roll out a change safely. The examples use resource group rg-web and scale set vmss-web.

Caution

restart, deallocate, stop, start and reimage can act on the whole scale set when you leave out --instance-ids. In production, always pass the instance you mean, and check the target before pressing Enter.

#Start here: what are you working with?

az vmss show -g rg-web -n vmss-web \
  --query "{mode:orchestrationMode, upgrade:upgradePolicy.mode, sku:sku.name, capacity:sku.capacity}" -o table

The --instance-ids behaviour above is documented for update-instances. The CLI reference doesn't describe Flexible behaviour for every other command, so try those on one non-production instance first.

#See the instances

# Uniform: ID, name, state and whether it runs the latest model
az vmss list-instances -g rg-web -n vmss-web \
  --query "[].{id:instanceId, name:name, state:provisioningState, latest:latestModelApplied}" -o table

# Uniform: full instance view for all instances, including health
az vmss get-instance-view -g rg-web -n vmss-web --instance-id "*"

# Flexible: instances with power state
az vm list -g rg-web -d \
  --query "[?virtualMachineScaleSet.id && contains(virtualMachineScaleSet.id, 'vmss-web')].{name:name, power:powerState, state:provisioningState}" -o table

# Connection details and sizes
az vmss list-instance-connection-info -g rg-web -n vmss-web
az vmss list-skus -g rg-web -n vmss-web -o table

# Extensions in the model
az vmss extension list -g rg-web --vmss-name vmss-web -o table

#Scale in or out

az vmss scale -g rg-web -n vmss-web --new-capacity 6

If autoscale is attached, it can undo your manual capacity at its next evaluation. Change its limits instead:

az monitor autoscale list -g rg-web -o table
az monitor autoscale show -g rg-web -n autoscale-web
az monitor autoscale update -g rg-web -n autoscale-web --min-count 4 --max-count 12 --count 6

To hold capacity during an incident:

az monitor autoscale update -g rg-web -n autoscale-web --enabled false

Warning

Add a reminder to re-enable it with --enabled true. A forgotten disabled setting won't scale out at the next traffic peak.

To control which instances are removed first (Default, NewestVM or OldestVM):

az vmss update -g rg-web -n vmss-web --scale-in-policy OldestVM

#Act on a single instance

Caution

Reimage rebuilds the OS disk, so anything written to it is lost. delete-instances with * removes every instance. Copy out logs and evidence first if you still need them.

az vmss stop powers instances off but they keep their compute allocation and charges. Use deallocate to stop paying for compute.

#Run a command without SSH or RDP

# Uniform
az vmss run-command invoke -g rg-web -n vmss-web --instance-id 3 \
  --command-id RunShellScript --scripts "df -h; systemctl --failed"

# Flexible: target the VM by name
az vm run-command invoke -g rg-web -n vmss-web_a1b2c3d4 \
  --command-id RunShellScript --scripts "df -h"

#Roll out a model change

A model change, such as a new image version, doesn't always reach the running instances. The upgrade policy decides what happens.

Warning

In Automatic mode, a model change can take every instance down at once. Check the mode with the first command in this post before you change the model of a production scale set.

#Manual: one canary, then the rest

# 1. Change the model
az vmss update -g rg-web -n vmss-web \
  --set virtualMachineProfile.storageProfile.imageReference.version=<image-version>

# 2. Update one instance and test it
az vmss update-instances -g rg-web -n vmss-web --instance-ids 0

# 3. Update the rest in small groups
az vmss update-instances -g rg-web -n vmss-web --instance-ids 1 2
az vmss update-instances -g rg-web -n vmss-web --instance-ids 3 4

To roll back, set the model to the previous version and update the changed instances the same way. Record the current version before you start.

#Rolling: set the policy

Set up the health signal first. A rolling upgrade relies on it to decide whether to continue.

az vmss update -g rg-web -n vmss-web \
  --upgrade-policy-mode Rolling \
  --max-batch-instance-percent 20 \
  --max-unhealthy-instance-percent 20 \
  --max-unhealthy-upgraded-instance-percent 20 \
  --pause-time-between-batches 60

#Watch or stop a rolling upgrade

az vmss rolling-upgrade get-latest -g rg-web -n vmss-web
az vmss rolling-upgrade cancel -g rg-web -n vmss-web
az vmss rolling-upgrade start -g rg-web -n vmss-web
az vmss get-os-upgrade-history -g rg-web -n vmss-web

• In get-latest, read runningStatus, progress and error.
• cancel stops the upgrade. To go back, change the model again and roll that out.
• start moves instances to the latest available platform image OS version. It doesn't apply other model changes, and instances already on the latest version aren't affected.

#Protect and repair instances

Keep one instance out of the way while you investigate it:

# Protect from scale-in only
az vmss update -g rg-web -n vmss-web --instance-id 3 --protect-from-scale-in true

# Also protect from model updates, reimage and other scale set actions
az vmss update -g rg-web -n vmss-web --instance-id 3 \
  --protect-from-scale-in true --protect-from-scale-set-actions true

Protection is set per instance, not in the scale set model. Flexible orchestration needs API version 2023-09-01 or later. Remove the protection when you finish.

Automatic repairs replace unhealthy instances. By default, the repair action is Replace.

az vmss update -g rg-web -n vmss-web \
  --enable-automatic-repairs true --automatic-repairs-grace-period 30

Confirm that the application's health endpoint reports correctly before you enable this, or healthy instances could be repaired. The scale set can use the Application Health extension or a load balancer health probe, but not both. Protected instances aren't repaired.

Pause repairs during maintenance, or when you need to keep a failed instance for investigation:

az vmss set-orchestration-service-state -g rg-web -n vmss-web \
  --service-name AutomaticRepairs --action Suspend

az vmss set-orchestration-service-state -g rg-web -n vmss-web \
  --service-name AutomaticRepairs --action Resume

#Quick symptom map

#Prevent it next time

• Set Rolling or Manual explicitly on production scale sets. Don't rely on someone remembering the mode.
• Pin image versions in the model, so a rollback has a known target.
• Test the health endpoint before you enable rolling upgrades or automatic repairs.
• Set autoscale minimums for your peak-hour floor, and review them after each incident.