ALB 502 errors during deployments: fix deregistration delay, keep-alive and graceful shutdown

#Symptom

Every release causes a short burst of 502 Bad Gateway. Sometimes there's also a low, steady rate of 502s all day, with no pattern. Targets look healthy, and the application logs show no errors, because the request never reached the app code.

#The two usual causes

#Step 1: confirm that the ALB generated the error

LB_DIM="app/alb-app/50dc6c495c0c9188"
aws cloudwatch get-metric-statistics --namespace AWS/ApplicationELB \
  --metric-name HTTPCode_ELB_502_Count \
  --dimensions Name=LoadBalancer,Value="$LB_DIM" \
  --start-time 2026-11-06T00:00:00Z --end-time 2026-11-06T06:00:00Z \
  --period 60 --statistics Sum --output table

Compare these metrics:

Use ALB access logs in Athena to separate the patterns:

SELECT date_trunc('minute', from_iso8601_timestamp(time)) AS minute,
       target_ip, elb_status_code, target_status_code,
       count(*) AS requests,
       avg(target_processing_time) AS avg_target_time
FROM alb_logs
WHERE day = '2026/11/06'
  AND elb_status_code IN (502, 503, 504)
GROUP BY 1, 2, 3, 4
ORDER BY 1 DESC;

Your table's column names and types depend on how you created it. Read the results like this:

• target_status_code is - and processing time is -1: the target closed the connection or never responded.
• Errors cluster on IPs that were being replaced: a shutdown race.
• Errors spread across all targets at random: a keep-alive mismatch.

#Fix 1: keep-alive timeout

The ALB reuses connections to targets. If the app closes an idle connection at the moment the ALB sends a request on it, the ALB returns a 502.

Rule: the target's keep-alive timeout must be longer than the ALB's idle timeout.

aws elbv2 describe-load-balancer-attributes --load-balancer-arn "$ALB_ARN" \
  --query "Attributes[?Key=='idle_timeout.timeout_seconds']"

If the ALB idle timeout is the default of 60 seconds, set the app's to about 65 or more:

# nginx
keepalive_timeout 75s;

// Node.js: the default keep-alive timeout is only a few seconds
const server = app.listen(8080);
server.keepAliveTimeout = 75000;
server.headersTimeout = 76000;

# gunicorn: the default is very low
gunicorn app:app --keep-alive 75

Node.js and gunicorn defaults are common culprits, and every proxy in the chain, such as a sidecar, needs the same treatment.

#Fix 2: shutdown order

During a rolling update, this sequence must hold:

1. The target is marked as draining, and the ALB stops sending new requests to it.
2. In-flight requests finish, up to the deregistration delay.
3. Only then does the process exit.

Set the delay to a little more than your longest normal request. The default of 300 seconds slows every deployment.

aws elbv2 modify-target-group-attributes --target-group-arn "$TG_ARN" \
  --attributes Key=deregistration_delay.timeout_seconds,Value=60

Your app must also handle SIGTERM properly: stop accepting new connections, finish current requests, then exit.

import signal, sys

def on_sigterm(signum, frame):
    server.shutdown()   # stop accepting; let current requests finish
    sys.exit(0)

signal.signal(signal.SIGTERM, on_sigterm)

#On ECS

ECS deregisters the task from the target group before it stops the container. Give the app enough time, and make sure PID 1 receives the signal:

{
  "name": "api",
  "stopTimeout": 60,
  "command": ["python", "app.py"]
}

Avoid running your app through sh -c "..." without exec, because the shell may not forward SIGTERM. Then ECS waits and sends SIGKILL, dropping requests.

#On EKS

Pod deletion and target deregistration happen in parallel, so the app can exit while the ALB is still sending traffic. Fix that with two settings.

Add a short sleep before SIGTERM reaches the app, and allow enough total time:

spec:
  terminationGracePeriodSeconds: 90
  containers:
    - name: api
      lifecycle:
        preStop:
          exec:
            command: ["sleep", "20"]

The image needs a sleep binary. The grace period must cover the preStop sleep plus the app's shutdown.

With the AWS Load Balancer Controller in IP mode, turn on readiness gates, so a rollout doesn't remove old pods before the new pods are registered and healthy in the ALB:

kubectl label namespace payments elbv2.k8s.aws/pod-readiness-gate-inject=enabled

This applies to pods created after the label is added, so restart the deployment.

#Fix 3: new targets that receive traffic too soon

If 5xx errors or slow responses come right after new targets register, check the health check path and timing.

aws elbv2 modify-target-group --target-group-arn "$TG_ARN" \
  --health-check-path /ready \
  --health-check-interval-seconds 10 \
  --healthy-threshold-count 2

aws elbv2 modify-target-group-attributes --target-group-arn "$TG_ARN" \
  --attributes Key=slow_start.duration_seconds,Value=60

/ready should check that the app can really serve traffic. Use slow start for apps with a cache warm-up or JIT compilation. It isn't compatible with some other target group settings, such as least outstanding requests.

Tip

A retry won't hide this reliably. The ALB doesn't automatically retry requests that may have reached the target, so fix the shutdown instead of relying on clients.

#Verify

Deploy under steady synthetic traffic, then check that HTTPCode_ELB_502_Count stays at zero for the entire rollout:

hey -z 5m -c 20 https://app.contoso.com/health

#Prevent it next time

• Add a deployment gate that fails a rollout if ALB 5xx errors rise.
• Put keep-alive timeouts in your service template, so new apps get them by default.
• Keep your longest request well below the deregistration delay.
• Related: ALB 502, 503 and 504 reference and all targets unhealthy.