Applies toAzure Databricks, Databricks on AWS and GCP (classic compute)
Get the termination reason
Compute → the cluster → Event log, or:
databricks clusters get <cluster-id> | jq '.state, .termination_reason'
databricks clusters events <cluster-id> | jq '.events[0:10][] | {type, details}'
Reason codes and fixes
| Termination reason | Meaning | Fix |
|---|---|---|
CLOUD_PROVIDER_LAUNCH_FAILURE + quota text | The cloud subscription/account vCPU quota for that VM family is used up | Request a quota increase for the family and region, or use another node type |
CLOUD_PROVIDER_LAUNCH_FAILURE + "SKU not available" / capacity | No capacity for the VM size in the region/zone | Another node type or zone. Use pools or fleet instance types. |
SUBNET_EXHAUSTED_FAILURE / IP exhaustion | VNet-injected workspace subnets are out of IPs | Bigger subnets (each node uses 2 IPs on Azure: host and container) |
INIT_SCRIPT_FAILURE | A cluster-scoped or global init script exited non-zero | Read the init script logs (cluster log delivery), and fix or remove the script |
DRIVER_UNREACHABLE / NPIP_TUNNEL_SETUP_FAILURE | Nodes can't reach the control plane (secure cluster connectivity) | Egress firewall or UDR blocks the required Databricks endpoints. Check NSG rules and routes. |
INSTANCE_POOL_CLUSTER_FAILURE | The pool couldn't provide instances | Check the pool's own errors, max capacity and idle instances |
SPOT_INSTANCE_TERMINATION | Spot VMs were reclaimed | Use on-demand for the driver, or spot with fallback to on-demand |
INACTIVITY | Auto-termination (normal) | None |
Networking checks (VNet injection / customer-managed VPC)
- NSG/security group rules required by Databricks are present. Don't remove the rules Databricks adds.
- If egress goes through a firewall, the Databricks control plane, artifact storage, metastore and event hub endpoints for your region are allowed.
- DNS resolves the workspace and storage endpoints (private link setups need the right private DNS zones).
Init scripts
# Script logs land under the cluster log destination:
# <log-destination>/<cluster-id>/init_scripts/<node>/
Common causes: apt-get failing because of egress rules, scripts not written for the current OS image, and scripts stored in workspace files without the right permissions. Prefer libraries and cluster policies over complex init scripts.
Jobs that depend on the cluster
- Use job clusters (created per run) rather than all-purpose clusters for production jobs. A broken all-purpose cluster then can't take every job down.
- Turn on job retries with a delay for transient cloud capacity errors.
- Serverless jobs remove most of these start-up failures, if your workspace and workload support them.
Tip
Alert on job failures whose message contains
CLOUD_PROVIDER_LAUNCH_FAILURE. Repeated capacity or quota failures are an early warning that you need a quota request or a node-type change.