Home / AWS

AWSTroubleshootingIntermediate

ECS task keeps stopping or restarting: reading the stopped reason

Map ECS stopped reasons and exit codes to their causes, including image pulls, secrets, health checks, OOM kills and load balancer health, and fix each one.

Deepak Bhardwaj2 min read

Applies toAmazon ECS on Fargate and EC2

1. Get the stopped reason

CLUSTER=prod; SERVICE=orders-api
aws ecs list-tasks --cluster $CLUSTER --service-name $SERVICE --desired-status STOPPED --query "taskArns[0:5]" --output text \
| xargs aws ecs describe-tasks --cluster $CLUSTER --tasks \
  --query "tasks[].{reason:stoppedReason, code:stopCode, containers:containers[].{name:name,exit:exitCode,reason:reason}}"

Stopped tasks stay visible for about an hour. For longer history, capture ECS task state change events with EventBridge.

2. Map the reason to the fix

Stopped reason / exit Cause Fix
CannotPullContainerError Image or tag missing, no route to ECR, or execution role can't pull Check the image URI. The task execution role needs ECR permissions. Private subnets need NAT or ECR/S3 VPC endpoints.
ResourceInitializationError: unable to pull secrets Secrets Manager / SSM unreachable or not allowed Execution role needs secretsmanager:GetSecretValue / ssm:GetParameters (and KMS decrypt), plus endpoint access
Essential container in task exited, exit code 1 The app crashed on start Read the container logs: config, missing environment variables, DB unreachable
Exit code 137 / OutOfMemoryError: Container killed due to memory usage Memory limit hit Raise task/container memory, fix the leak, or tune JVM/Node heap settings
Exit code 139 Segfault Native library or base image problem
Task failed ELB health checks Target group health check failing Check the path, port and timeout. Set a healthCheckGracePeriodSeconds long enough for start-up.
Task failed container health checks The container HEALTHCHECK command fails Test the command inside the image. Check startPeriod.
Scaling activity initiated by deployment Normal replacement during a deployment None

3. Logs

aws logs tail /ecs/orders-api --since 30m --follow

No logs at all usually means the task never started (pull, secrets or network problem), or the awslogs configuration is wrong and the execution role can't create the log stream.

4. Deployment loops

When new tasks keep failing, the deployment circuit breaker can roll back automatically:

aws ecs update-service --cluster $CLUSTER --service $SERVICE \
  --deployment-configuration "deploymentCircuitBreaker={enable=true,rollback=true},maximumPercent=200,minimumHealthyPercent=100"

5. Fargate specifics

  • Tasks need an ENI IP in the subnet. Check for subnet IP exhaustion.
  • Platform version changes can matter. Pin LATEST deliberately and test.
  • Ephemeral storage defaults to 20 GiB. Exceeding it stops the task.

Tip

Turn on Container Insights (enhanced observability) to see per-task CPU and memory against limits, which makes OOM patterns obvious before tasks start dying.

Related fixes