Home / Data Platform / Airflow

Data PlatformChecklistIntermediate

Airflow DAG not running or tasks stuck in queued: a checklist

Go from DAG import errors and schedule logic to scheduler health, pools, concurrency limits and executor workers when Airflow tasks don't start.

Deepak Bhardwaj2 min read

Applies toApache Airflow 2.x, Astronomer, Amazon MWAA, Google Cloud Composer

The DAG doesn't appear, or never gets a run

  • Import errors: a syntax error or a failed import hides the DAG.
    airflow dags list-import-errors
  • The DAG is paused. New DAGs are often paused by default (dags_are_paused_at_creation).
    airflow dags unpause sales_daily
  • start_date and schedule: a run for an interval only starts after that interval ends. A @daily DAG with start_date=2026-10-22 first runs at the end of 22 Oct for the 22 Oct interval.
  • start_date isn't dynamic (datetime.now()), because that breaks scheduling. Use a fixed date.
  • catchup=False if you don't want a backfill of every missed interval.
  • Time zone: start_date without a time zone is UTC.

Tasks stay in scheduled or queued

  • Scheduler is healthy:

    airflow jobs check --job-type SchedulerJob --allow-multiple --limit 2

    Check the scheduler logs for heartbeat failures, or a metadata DB that's out of connections.

  • Concurrency limits:

    Limit Where
    parallelism Whole installation (airflow.cfg [core])
    max_active_tasks Per DAG
    max_active_runs Runs per DAG at the same time
    Pools default_pool has 128 slots by default 
    airflow pools list

  • Executor workers: with Celery, workers must be up and listening to the queue the task uses (queue= on the operator). With Kubernetes, pods may be Pending (see the K8s posts).

  • Sensors hogging slots: sensors in poke mode hold a worker slot for hours. Use mode="reschedule" or deferrable operators.

Tasks keep failing or getting killed

  • Read the task log in the UI. If there's no log, the task never started on a worker (executor or worker problem).
  • Task received SIGTERM or "zombie" task → the worker died (often OOM) or the task exceeded execution_timeout.
  • Test the task in isolation:
    airflow tasks test sales_daily load_orders 2026-10-21
  • Add retries and retry_delay for flaky external systems, and on_failure_callback for alerting.

Re-run safely

airflow tasks clear sales_daily -t load_orders -s 2026-10-20 -e 2026-10-21 --downstream

Tasks must be idempotent (overwrite a partition, or MERGE) so that clearing and rerunning doesn't duplicate data.

Tip

On managed Airflow (MWAA, Composer, Astronomer), check the environment's worker autoscaling limits and the scheduler/DB size first. Most "stuck queued" incidents there are capacity limits, not DAG bugs.

Related fixes