Lambda in a VPC can't reach the internet or AWS services: subnets, NAT, endpoints and security groups

#The symptom

A function worked until someone attached it to a VPC so it could reach a database. Now every call to an outside API, S3, DynamoDB, Secrets Manager or STS hangs and the invocation ends with:

Task timed out after 30.00 seconds

There is no connection error, only a hang. That pattern almost always means packets are leaving the function's network interface and going nowhere. The function's own logs still reach CloudWatch, because Lambda ships logs through the service, not through your VPC. So working logs don't prove the network path is fine.

The single most common cause: a Lambda function in a VPC never gets a public IP address, even in a public subnet with a route to an internet gateway. It needs a NAT gateway or VPC endpoints.

#Map the symptom to the cause

#1. Check the function state

Before debugging routes, confirm Lambda actually created its network interfaces:

aws lambda get-function-configuration --function-name orders-sync \
  --query "{State:State,Reason:StateReasonCode,Update:LastUpdateStatus,UpdateReason:LastUpdateStatusReasonCode,Vpc:VpcConfig}"

A healthy function shows State: Active and LastUpdateStatus: Successful. If you see Failed or a reason code, fix that first:

aws iam attach-role-policy --role-name orders-sync-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole

Tip

A function that has been idle for a long time can go Inactive and have its network interfaces reclaimed. The next invoke fails while it goes back to Pending. That is expected; if it then fails, the reason code tells you why.

#2. Find the subnets and their routes

Get the subnets and security groups from the function:

aws lambda get-function-configuration --function-name orders-sync \
  --query "VpcConfig.[SubnetIds,SecurityGroupIds]" --output json

Then check the route table for each subnet. A subnet with no explicit association uses the VPC main route table, so check both:

aws ec2 describe-route-tables \
  --filters Name=association.subnet-id,Values=subnet-0aaa111,subnet-0bbb222 \
  --query "RouteTables[].{Id:RouteTableId,Subnets:Associations[].SubnetId,Routes:Routes[].[DestinationCidrBlock,DestinationPrefixListId,NatGatewayId,GatewayId,State]}" \
  --output json

What to look for:

• 0.0.0.0/0 → nat-...: correct for a function that needs the internet.
• 0.0.0.0/0 → igw-...: this is a public subnet. The function will not reach the internet from here. Move it to private subnets.
• A route with a pl-... prefix list → vpce-...: a gateway endpoint for S3 or DynamoDB is associated.
• Any route in blackhole state: the target (NAT gateway, peering, TGW attachment) was deleted.

Check the NAT gateway itself is available and sits in a public subnet:

aws ec2 describe-nat-gateways --nat-gateway-ids nat-0123456789abcdef0 \
  --query "NatGateways[].{State:State,Subnet:SubnetId,PublicIp:NatGatewayAddresses[0].PublicIp}"

#3. Check the security groups

Security groups are stateful, so the function's group needs outbound rules only; replies come back automatically. A group tightened to "outbound 5432 to the database" blocks everything else.

aws ec2 describe-security-groups --group-ids sg-0func1234 \
  --query "SecurityGroups[].IpPermissionsEgress"

For interface endpoints, the endpoint's security group needs inbound 443 from the function:

aws ec2 authorize-security-group-ingress --group-id sg-0endpoint99 \
  --protocol tcp --port 443 --source-group sg-0func1234

Network ACLs are stateless. If your subnets use custom NACLs, both directions need rules, including the ephemeral return ports (1024-65535).

#4. Check VPC endpoints and DNS

List endpoints in the VPC:

aws ec2 describe-vpc-endpoints --filters Name=vpc-id,Values=vpc-0abc123 \
  --query "VpcEndpoints[].{Service:ServiceName,Type:VpcEndpointType,State:State,PrivateDns:PrivateDnsEnabled,RouteTables:RouteTableIds,Subnets:SubnetIds}" \
  --output table

• Gateway endpoints (S3, DynamoDB) work only if associated with the route tables of the function's subnets.
• Interface endpoints (Secrets Manager, SSM, STS, SQS, Lambda itself and most others) need private DNS enabled so the normal service hostname resolves to the endpoint's private IPs. They also need an ENI in each AZ your function uses, otherwise traffic crosses AZs.

Private DNS needs both VPC DNS attributes turned on:

aws ec2 describe-vpc-attribute --vpc-id vpc-0abc123 --attribute enableDnsSupport
aws ec2 describe-vpc-attribute --vpc-id vpc-0abc123 --attribute enableDnsHostnames

To add a Secrets Manager endpoint:

aws ec2 create-vpc-endpoint --vpc-id vpc-0abc123 --vpc-endpoint-type Interface \
  --service-name com.amazonaws.eu-west-1.secretsmanager \
  --subnet-ids subnet-0aaa111 subnet-0bbb222 \
  --security-group-ids sg-0endpoint99 --private-dns-enabled

Warning

SDKs that call the global STS endpoint (sts.amazonaws.com) won't use a regional STS interface endpoint. Configure the SDK to use the regional endpoint, or the call still needs NAT. Defaults vary by SDK and version; check yours.

#5. Test from inside the function

Prove the path from where the code really runs. Deploy a short test handler (or a temporary test function with the same VPC config):

import socket, time

TARGETS = [("secretsmanager.eu-west-1.amazonaws.com", 443),
           ("s3.eu-west-1.amazonaws.com", 443),
           ("api.github.com", 443)]

def handler(event, context):
    out = {}
    for host, port in TARGETS:
        start = time.time()
        try:
            ip = socket.gethostbyname(host)
            socket.create_connection((ip, port), timeout=3).close()
            out[host] = f"OK {ip} {round(time.time()-start, 2)}s"
        except Exception as e:
            out[host] = f"FAIL {type(e).__name__}: {e}"
    return out

How to read it:

• DNS fails → VPC DNS or DHCP options.
• DNS returns a private IP (10.x, 172.16-31.x, 192.168.x) but connect times out → interface endpoint security group.
• DNS returns a public IP and connect times out → no NAT route, or the function is in a public subnet.

You can also run Reachability Analyzer from the function's ENI. Find it with:

aws ec2 describe-network-interfaces \
  --filters Name=interface-type,Values=lambda Name=vpc-id,Values=vpc-0abc123 \
  --query "NetworkInterfaces[].{Id:NetworkInterfaceId,Subnet:SubnetId,Desc:Description}" --output table

#Fix it

For most functions the fix is one of these:

1. Needs the internet: put the function only in private subnets whose route table sends 0.0.0.0/0 to a NAT gateway, ideally one NAT gateway per AZ.
2. Only needs AWS services: skip NAT. Add gateway endpoints for S3 and DynamoDB and interface endpoints for the rest. Cheaper per GB and keeps traffic private.
3. Doesn't need the VPC at all: if it reaches no private resources, remove the VPC config.

aws lambda update-function-configuration --function-name orders-sync \
  --vpc-config SubnetIds=subnet-0aaa111,subnet-0bbb222,SecurityGroupIds=sg-0func1234
aws lambda wait function-updated --function-name orders-sync

Caution

Changing VPC config replaces the function's network interfaces. Invocations during the update may fail or hit the old configuration. Do it in a quiet window, and remember that removed Lambda ENIs can take some time to delete, which blocks deleting the subnet or security group until they are gone.

#Prevent it next time

• Standardise on dedicated "lambda" private subnets in every AZ with a known route table, and reference them in IaC instead of picking subnets by hand.
• Create the endpoints your functions use (S3, DynamoDB, Secrets Manager, SSM, STS, SQS, KMS) as part of the VPC baseline, with one shared endpoint security group allowing 443 from the VPC CIDR.
• Set SDK connect timeouts well below the function timeout, so a broken path shows up as a fast connection error in the logs instead of a full timeout.
• Alarm on the Errors metric and on log lines containing Task timed out for VPC-attached functions.
• Use an AWS Config rule or IaC check that flags Lambda functions configured with public subnets.