Home / Data Platform / Airflow

Data PlatformTroubleshootingAdvanced

Airflow 3.3.1 / 3.3.2 upgrade: DAG processor timeouts, DAGs deactivated and runs stuck queued - triage and safe rollback

Some teams upgrading from Airflow 3.3.0 to 3.3.1 or 3.3.2 on Kubernetes report every DAG file timing out in the DAG processor, DAGs being deactivated and runs stuck in queued. The root cause isn't confirmed yet. Here's how to tell whether you're hit, what to collect, and how to roll back cleanly.

Deepak Bhardwaj4 min read

Applies toApache Airflow 3.3.0-3.3.2 on Kubernetes (official Helm chart), GitDagBundle and other DAG bundles; KubernetesExecutor and CeleryExecutor

The reported problem

GitHub issue apache/airflow#71772 (open, labelled bug) describes this after upgrading from 3.3.0 to 3.3.1, and later the same on 3.3.2:

  • The DAG processor log shows every file timing out: ... has been running for 50.01 seconds, exceeding the timeout of 50.00 seconds. Killing it!
  • The DAG processing stats show errors for every file, and DAGs are eventually deactivated.
  • A simple health-check DAG's runs sit in queued, then show as scheduled.
  • On downgrade to 3.3.0, worker pods got stuck in Terminating and had to be force-deleted.

The reporter's environment: official Helm chart 1.22.0, Python 3.13 images, GitDagBundle, Keycloak auth manager, custom providers. The same DAG parsed quickly when loaded manually with DagBag inside the dag-processor pod. As of late October 2026 a maintainer couldn't reproduce it and no root cause was confirmed.

So treat this as "possible regression in your environment", not a known bug with a known fix.

1. Are you seeing the same thing?

# timeouts in the dag-processor
kubectl logs deploy/airflow-dag-processor -n airflow --since=1h | grep -c "exceeding the timeout"

# import errors and stats
kubectl exec deploy/airflow-dag-processor -n airflow -- airflow dags list-import-errors
kubectl exec deploy/airflow-scheduler -n airflow -- airflow dags list | head -20

If every file times out (including tiny ones), it's the processor environment, not your DAG code. If only a few large files time out, it's more likely ordinary slow parsing; see "Not this issue" below.

2. Collect evidence before changing anything

  1. Parse one DAG file by hand inside the dag-processor pod and time it:
kubectl exec -it deploy/airflow-dag-processor -n airflow -- bash -c '
time python -c "from airflow.models.dagbag import DagBag; b=DagBag(\"/opt/airflow/dags/repo/dags/health_check.py\", include_examples=False); print(b.import_errors)"'

The DagBag import path has moved between 3.x releases; adjust it if the import fails.

  1. Run the processor verbosely in a debug pod (same image and config) and capture the output: airflow dag-processor -vvv.
  2. Note versions: Airflow, Helm chart, Python, providers (pip freeze | grep -i airflow), bundle type.
  3. Add your findings to the upstream issue if they match. Reproducible details are what get regressions fixed.

3. Short-term mitigations to try

These don't fix a regression but may confirm where it is:

  • Raise [dag_processor] dag_file_processor_timeout temporarily (for example to 120 seconds). If parsing then succeeds slowly, something is slowing the subprocess rather than hanging it.
  • Reduce [dag_processor] parsing_processes to 1 to see whether concurrency matters.
  • Try the same release with a local DAG folder instead of a Git bundle in a test environment, to see whether the bundle layer is involved.

4. Roll back cleanly

If production is affected, roll back to 3.3.0. Before you do:

Warning

3.3.1 changed how pandas DataFrame XComs are stored, and the release notes say a downgrade is a one-way door for XComs written by the new version with pandas 3. If any task pushed DataFrames after the upgrade, those XComs may not be readable on 3.3.0. Check before downgrading. See Airflow 3.3.1 and 3.3.2 behaviour changes.

Database migrations: check the release notes for whether 3.3.1/3.3.2 added schema migrations. If they did, run airflow db downgrade --to-version 3.3.0 (after a backup) before deploying the old image.

helm upgrade airflow apache-airflow/airflow -n airflow -f values.yaml \
  --set airflowVersion=3.3.0 --set defaultAirflowTag=3.3.0

If worker pods hang in Terminating because their task never finishes:

kubectl delete pod <worker-pod> -n airflow --grace-period=0 --force

Then clear the affected task instances so they rerun on 3.3.0.

Not this issue

Ordinary DAG parse timeouts have well-known causes: top-level code calling databases or APIs, importing heavy libraries at module level, or generating thousands of tasks dynamically. Those affect specific files, and moving work into tasks fixes them.

Checklist

  • Confirmed every DAG times out, not just a few
  • Manual parse timing and verbose processor logs collected
  • Versions and bundle type recorded; upstream issue checked or updated
  • XCom and DB migration impact checked before any downgrade
  • Rollback tested in staging first
  • Next upgrade attempt staged in a non-production environment

Sources

Related fixes