Cluster level
- Standard tier (API server SLA; 99.95% with zones).
- Node pools across zones
1 2 3, minimum 3 nodes per pool. - Cluster autoscaler with enough maximum to absorb a lost zone.
- Separate system pool with
CriticalAddonsOnlytaint. - Planned maintenance windows and
maxSurgeon node pools.
Workload level
- At least 3 replicas for anything user-facing.
- Spread replicas across zones and nodes:
spec:
replicas: 3
template:
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector: { matchLabels: { app: orders-api } }
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector: { matchLabels: { app: orders-api } }
- PodDisruptionBudget so drains and upgrades keep enough pods running:
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata: { name: orders-api }
spec:
minAvailable: 2
selector: { matchLabels: { app: orders-api } }
Warning
A PDB with
minAvailableequal to the replica count (ormaxUnavailable: 0) blocks node drains forever and makes AKS upgrades fail. Leave room for at least one pod to move.
- Readiness probes (traffic only to ready pods) and liveness probes with generous thresholds; startup probes for slow starters.
- Requests and limits set, so the scheduler places pods realistically.
- Graceful shutdown: handle SIGTERM,
preStopsleep of a few seconds so the load balancer stops sending traffic first.
Storage
- Azure Disk PVs are zonal (LRS disks): a pod with a disk can only run in that zone. For stateful apps use ZRS disks (
skuName: Premium_ZRSStorageClass) or app-level replication. - Azure Files (ZRS) for shared storage that must follow pods across zones.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata: { name: managed-csi-premium-zrs }
provisioner: disk.csi.azure.com
parameters: { skuName: Premium_ZRS }
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
Ingress and egress
- Ingress controller (or Application Gateway for Containers) with 2+ replicas across zones.
- Load balancer Standard SKU, zone-redundant frontend.
- Egress via hub firewall (zone-redundant) or one NAT gateway per zone.
Verify
kubectl get pods -l app=orders-api -o wide
kubectl get nodes -L topology.kubernetes.io/zone
kubectl get pdb -A
Run a drain on all nodes of one zone in a test window (kubectl drain per node) and confirm the app keeps serving.