Airflow connections and variables cheat sheet: CLI, environment variables, secrets backends and export/import

#Where Airflow looks for a connection or variable

When a task asks for conn_id or a variable key, Airflow checks these in order and uses the first match:

1. The configured secrets backend (Key Vault, Secrets Manager, Vault ...), if any
2. Environment variables (AIRFLOW_CONN_*, AIRFLOW_VAR_*)
3. The metadata database (what the UI and airflow connections add write to)

This explains the classic "I changed it in the UI and nothing happened": an environment variable or secret with the same ID wins.

#Connections

# List, show one (output includes the password - mind your terminal history)
airflow connections list
airflow connections list -o yaml
airflow connections get my_postgres -o json

# Add from a URI
airflow connections add my_postgres \
  --conn-uri 'postgresql://etl_user:S3cret%21@pg01.internal:5432/sales'

# Add field by field (safer for special characters)
airflow connections add my_postgres \
  --conn-type postgres --conn-host pg01.internal --conn-port 5432 \
  --conn-login etl_user --conn-password 'S3cret!' --conn-schema sales

# Add with JSON, including extras
airflow connections add my_api --conn-json '{
  "conn_type": "http", "host": "https://api.example.com",
  "extra": {"timeout": 30, "verify": true}
}'

# Delete
airflow connections delete my_postgres

Tip

In a URI, special characters in passwords must be URL-encoded (! becomes %21, @ becomes %40). A wrong encoding gives "password authentication failed" even though the password "looks right". Use field-by-field or JSON form to avoid it.

#Test a connection

airflow connections test my_postgres

Connection testing is disabled by default in recent versions. Enable it in airflow.cfg ([core] test_connection = Enabled) or with AIRFLOW__CORE__TEST_CONNECTION=Enabled, ideally only in non-production. The test runs from wherever the CLI runs, which may not be where workers run.

#Export and import

airflow connections export connections.json
airflow connections export connections.yaml --file-format yaml
airflow connections import connections.json

Caution

Export files contain passwords in plain text. Write them to an encrypted location, move them, import, then delete them.

#Variables

airflow variables list
airflow variables get batch_date
airflow variables get config_json --json            # deserialise JSON
airflow variables get maybe_missing --default none

airflow variables set batch_date 2026-10-27
airflow variables set config_json '{"retries": 3, "region": "eu"}' --json
airflow variables delete batch_date

airflow variables export variables.json
airflow variables import variables.json

Values whose key contains words such as password, secret, api_key or token are masked in the UI and logs. Name sensitive variables accordingly, or better, keep them in a secrets backend.

#Environment variables

Good for containers and Helm charts. The ID is upper-case after the prefix.

# Connection as URI
export AIRFLOW_CONN_MY_POSTGRES='postgresql://etl_user:S3cret%21@pg01.internal:5432/sales'

# Connection as JSON (Airflow 2.3+)
export AIRFLOW_CONN_MY_API='{"conn_type": "http", "host": "https://api.example.com", "extra": {"timeout": 30}}'

# Variable
export AIRFLOW_VAR_BATCH_DATE='2026-10-27'

Connections and variables defined this way do not appear in the UI or in airflow connections list. Use airflow connections get my_postgres to confirm which value Airflow resolves.

#Secrets backends

#Azure Key Vault

Needs the apache-airflow-providers-microsoft-azure package.

[secrets]
backend = airflow.providers.microsoft.azure.secrets.key_vault.AzureKeyVaultBackend
backend_kwargs = {"connections_prefix": "airflow-connections", "variables_prefix": "airflow-variables", "vault_url": "https://kv-airflow-prod.vault.azure.net/"}

Secret names use the prefix and the ID: connection my_postgres is the secret airflow-connections-my-postgres (Key Vault names cannot contain underscores, and the backend handles the separator; check your provider version's docs for its exact mapping). Store the connection as a URI or JSON.

az keyvault secret set --vault-name kv-airflow-prod \
  --name airflow-connections-my-postgres \
  --value 'postgresql://etl_user:S3cret%21@pg01.internal:5432/sales'

#AWS Secrets Manager

Needs apache-airflow-providers-amazon.

[secrets]
backend = airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend
backend_kwargs = {"connections_prefix": "airflow/connections", "variables_prefix": "airflow/variables"}

aws secretsmanager create-secret --name airflow/connections/my_postgres \
  --secret-string 'postgresql://etl_user:S3cret%21@pg01.internal:5432/sales'

Set connections_prefix or variables_prefix to null in backend_kwargs to stop the backend looking up that type (saves API calls and cost).

The workers, scheduler and (for UI tests) the API server all need permission to read the secrets: a managed identity with Key Vault Secrets User, or an IAM role with secretsmanager:GetSecretValue.

#Troubleshooting

#Prevent it next time

• Keep production credentials in a secrets backend; keep the metadata DB for non-sensitive values only.
• Store non-secret connection definitions (host, port, extras) in Git and apply them in deployment.
• Never call Variable.get() at DAG module level.
• Rotate secrets in the backend; tasks pick up the new value on their next run.