First: is it the ALB or the target?
| Metric | Meaning |
|---|---|
HTTPCode_ELB_5XX_Count | The ALB generated the error (it couldn't get a good response) |
HTTPCode_Target_5XX_Count | Your application returned 5xx. The ALB just passed it on. |
UnHealthyHostCount / HealthyHostCount | Target group health |
TargetResponseTime | Backend latency |
If only HTTPCode_Target_5XX is rising, debug the application, not the load balancer.
502 Bad Gateway (ALB-generated)
The target closed the connection or sent an invalid response. The common causes:
- Keep-alive timeout on the target is shorter than the ALB idle timeout (60 s by default). The target closes an idle connection just as the ALB reuses it. Set the server keep-alive timeout above the ALB idle timeout, for example
keepAliveTimeout65 s in Node.js orKeepAliveTimeout 65in Apache. - The target crashed or reset mid-response (an OOM kill, a restart).
- Malformed response headers, or a header block that's too large.
- Lambda targets returning a response that isn't valid, or larger than 1 MB.
503 Service Unavailable
- No registered or healthy targets in the target group. Check the health check:
aws elbv2 describe-target-health --target-group-arn $TG \ --query "TargetHealthDescriptions[].{id:Target.Id,port:Target.Port,state:TargetHealth.State,reason:TargetHealth.Reason,desc:TargetHealth.Description}" --output table - Reasons such as
Target.FailedHealthChecks,Target.TimeoutorTarget.ResponseCodeMismatchpoint to the health check path, port or the security group.
504 Gateway Timeout
The target didn't respond within the idle timeout.
- Long requests (reports, exports) → raise the idle timeout, or better, make the work asynchronous.
aws elbv2 modify-load-balancer-attributes --load-balancer-arn $ALB --attributes Key=idle_timeout.timeout_seconds,Value=120 - The target security group doesn't allow the ALB → connections hang. Allow inbound from the ALB's security group.
Use access logs
Enable access logs to S3 and query them with Athena. The fields that matter:
elb_status_codevstarget_status_code: a-target code means no response was received from the target.target_processing_time = -1: the ALB couldn't get a response from the target (a connection error or timeout).error_reason(where present): for exampleTargetConnectionErrororTargetResponseError.
Health check tuning
| Setting | Recommendation |
|---|---|
| Path | A lightweight /health that doesn't depend on every downstream service |
| Healthy / unhealthy threshold | 2–3 / 2–3 |
| Interval / timeout | 10–15 s / 5 s |
| Success codes | 200 (or 200-399 if it redirects) |
| Deregistration delay | 30–60 s for most apps. The default 300 s slows deployments. |
Prevent it next time
Alarm on HTTPCode_ELB_5XX_Count and UnHealthyHostCount > 0, and on TargetResponseTime p95 against your SLO.