Home / Data Platform / Airflow

Data PlatformRunbookAdvanced

Sev1: after a deploy, Airflow DAGs disappear or fail to import and nothing runs

A release changed shared code, a requirement or a provider version, and now many DAGs show import errors, vanish from the UI or never get scheduled. Find the broken import, roll back or hotfix, catch up missed runs safely, and add the CI checks that stop it recurring.

Deepak Bhardwaj4 min read

Applies toApache Airflow 2.x and 3.x (self-managed, Kubernetes Helm chart, MWAA, Astronomer, Cloud Composer)

Symptoms

  • UI banner: Broken DAG: [/opt/airflow/dags/x.py] ModuleNotFoundError / ImportError / SyntaxError.
  • DAGs missing from the list, or present but paused/stale with no new runs.
  • Scheduled times pass with no runs created; SLAs breach across many teams at once.
  • Started right after a deploy of DAG code, a shared plugins/ or include/ module, or an image with new requirements.

Step 1: list the import errors (2 minutes)

airflow dags list-import-errors
airflow dags list-import-errors -o json | jq -r '.[].error' | sort | uniq -c | sort -rn | head

On managed services, the same list is in the UI and in the DAG processor / scheduler logs (CloudWatch for MWAA, Cloud Logging for Composer).

Group the errors. One common root cause usually explains them all:

Error pattern Typical cause
ModuleNotFoundError: No module named 'x' Requirement missing from the image, or a shared module moved
ImportError: cannot import name 'Y' from 'airflow.providers...' Provider upgraded and a class moved or was removed
SyntaxError / IndentationError Bad merge in a shared file
AirflowDagDuplicatedIdException Two files now define the same dag_id
DagBag import timeout Top-level code calls a slow API/DB during parsing
Airflow 3 import of airflow.models.DAG, removed operators DAG code not migrated to the new version

Step 2: roll back or hotfix (15 minutes)

Fastest: roll back the deploy. Redeploy the previous DAG bundle or image tag:

helm rollback airflow <previous-revision> -n airflow          # Helm chart
# or redeploy the previous git SHA of the dags repo / git-sync rev

For MWAA, point the environment back to the previous requirements.txt / plugins.zip object version; for Composer, revert the PyPI package change. Environment updates on managed services can take a while, so start this immediately while you investigate.

Or hotfix if the cause is one obvious line:

python -c "import ast,sys; ast.parse(open('dags/shared/utils.py').read())"   # syntax check
python dags/finance/daily_close.py                                              # import check in the same image

Tip

Reproduce in the same image the scheduler uses (docker run --rm -it <image> python /opt/airflow/dags/x.py). A module that imports fine on a laptop may be missing from the runtime image.

Step 3: confirm parsing recovered

airflow dags list-import-errors           # should be empty
airflow dags list | wc -l                 # back to the expected count
airflow dags report                        # parse time per file

Watch the DAG processor log for parse times; a file that takes longer than dagbag_import_timeout will keep failing.

Step 4: deal with missed runs

Runs that should have happened during the outage were not created if the DAGs failed to parse. Options per DAG:

  • catchup=True DAGs create the missing runs automatically once parsed. Check that this will not overload sources (limit with max_active_runs).
  • catchup=False DAGs only create the latest run. Trigger the missed ones explicitly:
airflow dags backfill finance_daily_close -s 2026-10-28 -e 2026-10-29 --reset-dagruns    # Airflow 2
airflow backfill create --dag-id finance_daily_close --from-date 2026-10-28 --to-date 2026-10-29   # Airflow 3

Prioritise by business impact: run the finance close before the marketing dashboard. Use pools to stop the catch-up from starving normal runs.

Warning

Make sure tasks are idempotent before mass backfilling. Non-idempotent loads (plain INSERT without dedup) will duplicate data.

Step 5: check for DAGs that were paused or removed

When a DAG file fails to parse long enough, Airflow can mark it inactive. After recovery confirm it is active and unpaused:

airflow dags list -o json | jq -r '.[] | select(.paused=="True") | .dag_id'
airflow dags unpause finance_daily_close

Step 6: confirm

  • Import errors: none.
  • New runs created on schedule for the next interval.
  • Backfilled runs complete; downstream data owners confirm freshness.

Prevent it next time

Add a CI job that loads every DAG in the production image:

# tests/test_dag_integrity.py
from airflow.models import DagBag

def test_no_import_errors():
    bag = DagBag(dag_folder="dags", include_examples=False)
    assert not bag.import_errors, bag.import_errors

def test_unique_dag_ids_and_tags():
    bag = DagBag(dag_folder="dags", include_examples=False)
    for dag_id, dag in bag.dags.items():
        assert dag.tags, f"{dag_id} has no tags"
  • Pin Airflow and providers with the official constraints file; upgrade providers deliberately, not as a side effect.
  • No network or database calls at module top level.
  • Deploy DAG changes to a staging environment with the same image first.
  • Alert when airflow dags list-import-errors is non-empty (or on the dag_processing.import_errors metric).

Related fixes