Home / AWS

AWSTroubleshootingAdvanced

EKS Ingress has no ADDRESS: debug the AWS Load Balancer Controller's subnets, IAM and webhook

An EKS Ingress never gets an ALB. Read events and controller logs, then fix the ingress class, subnet discovery tags, IAM permissions, webhook errors and target-type settings that block it.

Deepak Bhardwaj6 min read

Applies toAmazon EKS, AWS Load Balancer Controller v2.x, Application Load Balancer, Network Load Balancer

Symptom

$ kubectl get ingress -n app
NAME   CLASS   HOSTS   ADDRESS   PORTS   AGE
web    alb     *                 80      22m

ADDRESS stays empty, and no load balancer appears in the console. Or kubectl apply itself fails with a webhook error.

The AWS Load Balancer Controller watches Ingress objects and creates ALBs. So this is either the controller not seeing your Ingress, or it seeing it and failing.

Step 1: read the events

kubectl describe ingress web -n app

Look at the Events section:

Event text Cause Go to
No events at all Controller isn't handling this Ingress Step 2
FailedBuildModel: couldn't auto-discover subnets Subnet tags Step 3
FailedDeployModel: AccessDenied IAM permissions Step 4
FailedBuildModel: ingress: ... Service ... not found The backend Service name or port is wrong Step 6
FailedBuildModel: NodePort errors Instance mode needs a NodePort Service Step 6
SuccessfullyReconciled with no ADDRESS Status not yet updated, or another Ingress issue Check again, then read the logs

Step 2: check the controller and class

Is the controller running?

kubectl get deployment -n kube-system aws-load-balancer-controller
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
kubectl logs -n kube-system deployment/aws-load-balancer-controller --tail=100

Is your Ingress using a class it owns?

kubectl get ingressclass
kubectl get ingress web -n app -o jsonpath='{.spec.ingressClassName}{"\n"}'

Your Ingress needs spec.ingressClassName: alb, or whatever class the controller owns. An older annotation, kubernetes.io/ingress.class, may also be in use. If another controller owns the class, this one ignores the Ingress, without any event.

A minimal working example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
  namespace: app
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 80

The default scheme is internal. If you expect a public ALB and didn't set scheme, you'll get a private one, or a subnet error.

Step 3: fix subnet discovery

This is the most common failure. The controller finds subnets by their tags:

Scheme Required tag on the subnet
internet-facing kubernetes.io/role/elb = 1
internal kubernetes.io/role/internal-elb = 1

Older controller versions also require kubernetes.io/cluster/<cluster-name> = shared or owned. Tagging it is harmless. See what you have:

CLUSTER=eks-prod
VPC_ID=$(aws eks describe-cluster --name "$CLUSTER" \
  --query cluster.resourcesVpcConfig.vpcId --output text)

aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" \
  --query 'Subnets[].{id:SubnetId, az:AvailabilityZone, free:AvailableIpAddressCount, public:Tags[?Key==`kubernetes.io/role/elb`]|[0].Value, internal:Tags[?Key==`kubernetes.io/role/internal-elb`]|[0].Value}' \
  --output table

An ALB needs subnets in at least two Availability Zones. Each one needs a few free IP addresses. Add the tags:

aws ec2 create-tags --resources subnet-aaa subnet-bbb \
  --tags Key=kubernetes.io/role/elb,Value=1

For an internet-facing ALB, these must be truly public subnets, with a route to an internet gateway.

You can also list subnets by hand, which skips discovery:

    alb.ingress.kubernetes.io/subnets: subnet-aaa,subnet-bbb

Step 4: fix IAM

The controller needs AWS permissions, usually through IRSA or Pod Identity. Find what it's using:

kubectl get serviceaccount aws-load-balancer-controller -n kube-system -o yaml | grep -A2 annotations
kubectl logs -n kube-system deployment/aws-load-balancer-controller --tail=500 | grep -iE "accessdenied|unauthorized|forbidden"

The error names the action that was denied. A very common one:

AccessDenied: User: arn:aws:sts::111122223333:assumed-role/... is not authorized
to perform: elasticloadbalancing:AddTags ...
Error Fix
Action denied on elasticloadbalancing:* or ec2:* Use the IAM policy that matches your controller version, from the controller's install documentation
AssumeRoleWithWebIdentity denied Fix the role's trust policy, or the service account annotation
The role is the node's role, not the controller's The annotation is missing, or the pods weren't restarted

The required policy changes between releases. After upgrading the controller, compare your policy with the one published for that version.

After a change to the service account, restart the controller:

kubectl rollout restart deployment aws-load-balancer-controller -n kube-system

Step 5: fix webhook errors

If kubectl apply fails like this, the API server can't reach the controller's admission webhook:

Error from server (InternalError): failed calling webhook "vingress.elbv2.k8s.aws":
no endpoints available for service "aws-load-balancer-webhook-service"

Check this in order:

kubectl get endpoints -n kube-system aws-load-balancer-webhook-service
kubectl get validatingwebhookconfiguration,mutatingwebhookconfiguration | grep -i elbv2
kubectl get secret -n kube-system aws-load-balancer-tls
Finding Cause Fix
No endpoints Controller pods are down Fix the pods first
context deadline exceeded The control plane can't reach the pods Allow TCP 9443 from the cluster security group to the nodes
x509 errors Webhook certificate problem Check the cert-manager or chart-managed certificate

Warning

If the controller is down, its webhooks can also block changes to Services and Ingresses across the cluster. Don't delete the webhook configurations as a first move. Get the controller healthy. If you do have to remove them in an emergency, record the change and reinstall the chart afterward.

Step 6: fix the backend and target type

Check that the Service exists, and that its port matches:

kubectl get svc web -n app -o wide
kubectl get endpointslices -n app -l kubernetes.io/service-name=web
Target type Requires
ip VPC CNI or similar routable pod IPs. Works with ClusterIP Services
instance A NodePort or LoadBalancer Service. The default if you don't set one

target-type: ip is required for Fargate pods.

For a target group that's created but stays unhealthy, set the health check to match the app:

    alb.ingress.kubernetes.io/healthcheck-path: /healthz
    alb.ingress.kubernetes.io/success-codes: "200"

Then see all targets unhealthy. The node or pod security groups also have to allow the ALB's traffic. The controller can manage these rules by default.

Don't delete with a stuck finalizer

Deleting an Ingress when the controller's down leaves it stuck, because its finalizer is waiting for the controller:

kubectl get ingress web -n app -o jsonpath='{.metadata.finalizers}{"\n"}'

Caution

Removing the finalizer by hand makes the Ingress disappear, but it leaves the ALB and target groups running in AWS. They keep costing money, and you'll have to delete them by hand. Fix the controller first, so that it can clean up.

Prevent it next time

  • Tag subnets when you create the VPC, in your infrastructure code.
  • Upgrade the controller's IAM policy along with its Helm chart.
  • Run at least two controller replicas, with a Pod Disruption Budget.
  • Alert on FailedDeployModel and FailedBuildModel events.
  • Related: choosing ALB, NLB or API Gateway and the EKS baseline.