Optimization: AKS cost and efficiency - right-sized requests, autoscaler tuning and spot node pools

#The usual picture

Nodes show 70–90% of CPU requested but 15–25% used. The scheduler packs by requests, so you pay for reserved capacity that sits idle.

#1. Measure requests vs usage

kubectl top nodes
kubectl describe nodes | grep -A8 "Allocated resources"

Per namespace with Container Insights:

let usage = Perf
| where TimeGenerated > ago(7d) and ObjectName == "K8SContainer" and CounterName == "cpuUsageNanoCores"
| extend PodUid = tostring(split(InstanceName, "/")[-2])      // InstanceName = <cluster>/<podUid>/<container>
| summarize p95_mcores = percentile(CounterValue, 95) / 1000000 by PodUid, InstanceName;
KubePodInventory
| where TimeGenerated > ago(1d)
| distinct Namespace, PodUid
| join kind=inner usage on PodUid
| summarize p95_used_mcores = sum(p95_mcores) by Namespace
| sort by p95_used_mcores desc

The AKS cost analysis add-on (Standard/Premium tier clusters) shows cost by namespace in Cost Management:

az aks update -g rg-aks -n aks-prod --enable-cost-analysis

#2. Fix requests with VPA recommendations

Run VPA in recommendation-only mode first:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata: { name: orders-api, namespace: orders }
spec:
  targetRef: { apiVersion: apps/v1, kind: Deployment, name: orders-api }
  updatePolicy: { updateMode: "Off" }

kubectl get vpa orders-api -n orders -o jsonpath='{.status.recommendation.containerRecommendations}'

Set requests near the recommended target (p90-ish usage), keep memory limits to protect nodes, and be careful with CPU limits (throttling hurts latency). Enable the add-on with az aks update --enable-vpa if it isn't installed.

#3. Tune the cluster autoscaler

Defaults are conservative about scale-down. A profile for a busy production cluster:

az aks update -g rg-aks -n aks-prod --cluster-autoscaler-profile \
  scale-down-unneeded-time=5m scale-down-delay-after-add=5m \
  scale-down-utilization-threshold=0.6 skip-nodes-with-local-storage=false expander=least-waste

Things that block scale-down: pods without a controller, restrictive PodDisruptionBudgets (maxUnavailable: 0), safe-to-evict: "false" annotations, local storage. Find them:

kubectl get events -A --field-selector reason=ScaleDownFailed
kubectl get configmap cluster-autoscaler-status -n kube-system -o yaml

#4. Node sizes that pack well

Keep the system node pool small and dedicated (CriticalAddonsOnly taint) and put workloads in user pools that can scale to zero where possible.

#5. Spot node pools for tolerant workloads

az aks nodepool add -g rg-aks --cluster-name aks-prod -n spotpool \
  --priority Spot --eviction-policy Delete --spot-max-price -1 \
  --enable-cluster-autoscaler --min-count 0 --max-count 20 --node-vm-size Standard_D8s_v5

Spot nodes get the taint kubernetes.azure.com/scalesetpriority=spot:NoSchedule. Only workloads that tolerate it land there:

tolerations:
- { key: kubernetes.azure.com/scalesetpriority, operator: Equal, value: spot, effect: NoSchedule }

Good candidates: batch jobs, CI runners, dev namespaces, stateless workers with retries. Not: single-replica stateful services. Spot capacity can be evicted at any time.

#6. Scale workloads, not just nodes

• HPA or KEDA on real signals (requests/s, queue length) so replicas follow load.
• Scale non-prod deployments to zero out of hours (KEDA cron scaler) or stop dev clusters: az aks stop.

#7. Track the result

Warning

Lowering requests too far causes CPU contention and OOM kills under load. Change one namespace at a time and watch latency and restarts for a week.

#Checklist

• Requests vs usage measured per namespace
• VPA recommendations applied carefully
• Autoscaler profile tuned, scale-down blockers fixed
• Node sizes reviewed, system pool separated
• Spot pool for tolerant workloads
• Non-prod scaled down out of hours