Databricks jobs failing after a user leaves: transfer job owners, run-as identities and tokens

#Symptoms

The morning after a leaver's Entra ID account is disabled (or removed from the Databricks enterprise app):

• Scheduled jobs fail at start with errors saying the owner or run-as user is inactive, not found or not authorized.
• Lakeflow pipelines that ran as that person fail to start an update.
• Power BI refreshes, JDBC/ODBC tools and scripts get 401/403, because they used that person's personal access token (PAT).
• Shared queries, alerts or dashboards that ran with the owner's credentials stop returning data.

This is what Databricks documents. A deactivated user keeps their permissions and objects, but they can't authenticate, their tokens stop working, and scheduled jobs they created fail until a new owner is assigned. Deleting a user from the account is worse. Jobs they own fail, clusters they own stop, and anything shared with "run as owner" credentials has to be reassigned.

Warning

Don't delete the leaver from the account just to clean up. Deactivate instead. Deletion is destructive and harder to reverse. If SCIM provisioning is still assigned, the account may also come back on the next sync.

#1. Confirm the identity state

Look the user up at the workspace level, using a workspace admin profile:

databricks users list --filter 'userName eq "leaver@contoso.com"' --output json \
  | jq '.[] | {id, userName, active}'

active: false means the account was deactivated, either by SCIM or automatic identity management syncing a disabled Entra account, or by an admin. If the user doesn't appear at all, they were removed from the workspace or the account.

#2. Find everything tied to that identity

#Jobs and pipelines (system tables)

system.lakeflow.jobs and system.lakeflow.pipelines are SCD2 tables. Take the latest row per object, then drop deleted ones:

WITH latest_jobs AS (
  SELECT *, ROW_NUMBER() OVER (PARTITION BY workspace_id, job_id ORDER BY change_time DESC) AS rn
  FROM system.lakeflow.jobs
  QUALIFY rn = 1
)
SELECT workspace_id, job_id, name, creator_user_name, run_as_user_name
FROM latest_jobs
WHERE delete_time IS NULL
  AND (run_as_user_name = 'leaver@contoso.com' OR creator_user_name = 'leaver@contoso.com');

WITH latest_pipelines AS (
  SELECT *, ROW_NUMBER() OVER (PARTITION BY workspace_id, pipeline_id ORDER BY change_time DESC) AS rn
  FROM system.lakeflow.pipelines
  QUALIFY rn = 1
)
SELECT workspace_id, pipeline_id, name, created_by, run_as
FROM latest_pipelines
WHERE delete_time IS NULL
  AND (run_as = 'leaver@contoso.com' OR created_by = 'leaver@contoso.com');

Tip

run_as_user_name and creator_user_name aren't populated on job rows emitted before early December 2025. If a job hasn't changed since then, match on the run_as and creator_id columns using the user's ID from step 1.

#Personal access tokens

databricks token-management list --created-by-username leaver@contoso.com --output json \
  | jq -r '.[] | [.token_id, .comment, (.expiry_time/1000|todate)] | @tsv'

Each token comment usually names the tool that used it, such as Power BI, a script or a third-party connector. Those are the consumers you need to move.

#Unity Catalog objects they own

Ownership doesn't break reads for other people, but nobody can grant on or alter the object until ownership moves.

SELECT 'catalog' AS kind, catalog_name AS name FROM system.information_schema.catalogs WHERE catalog_owner = 'leaver@contoso.com'
UNION ALL
SELECT 'schema', catalog_name || '.' || schema_name FROM system.information_schema.schemata WHERE schema_owner = 'leaver@contoso.com'
UNION ALL
SELECT 'table', table_catalog || '.' || table_schema || '.' || table_name FROM system.information_schema.tables WHERE table_owner = 'leaver@contoso.com';

#Code that lives in their home folder

A job can have a valid run-as identity and still point at /Workspace/Users/leaver@contoso.com/.... Those notebooks stay in place after deactivation, but nobody maintains them and they're easy to delete by accident later.

for id in $(databricks jobs list --output json | jq -r '.[].job_id'); do
  databricks jobs get "$id" --output json \
    | jq -r --arg id "$id" '.settings.tasks[]? | select((.notebook_task.notebook_path // "") | contains("leaver@contoso.com")) | "\($id)\t\(.task_key)\t\(.notebook_task.notebook_path)"'
done

#3. Fix: symptom → cause → action

#Move a job to a service principal

You need workspace admin rights and the Service Principal User role on the service principal.

JOB_ID=123456789
SP_APP_ID=8a7c1e2f-0000-1111-2222-333344445555

# Run as the service principal
databricks jobs update $JOB_ID --json "{\"new_settings\": {\"run_as\": {\"service_principal_name\": \"$SP_APP_ID\"}}}"

# Make a team group the owner (or the SP, if that's your standard)
databricks jobs update-permissions $JOB_ID --json '{
  "access_control_list": [
    {"group_name": "data-eng-prod", "permission_level": "IS_OWNER"}
  ]
}'

databricks jobs get-permissions $JOB_ID

Names vary by organization. Some require the service principal as owner, and some allow a group. Check what your platform standard says. Then confirm the service principal can read every table the job reads and write every table it writes. Jobs that worked under a person often relied on that person's broad grants:

SHOW GRANTS `8a7c1e2f-0000-1111-2222-333344445555` ON SCHEMA prod.sales;

Give the service principal its own grants, and only the ones the job needs:

GRANT USE CATALOG ON CATALOG prod TO `8a7c1e2f-0000-1111-2222-333344445555`;
GRANT USE SCHEMA, SELECT, MODIFY ON SCHEMA prod.sales TO `8a7c1e2f-0000-1111-2222-333344445555`;

If the job is deployed with Databricks Asset Bundles, change run_as in the bundle and redeploy. Otherwise the next deploy puts the old setting back.

#Pipelines

Open the pipeline, go to Settings, and set Run as to the service principal (or change the owner). Then start a validation update. Check the event log if it fails.

#Unity Catalog ownership

ALTER SCHEMA prod.sales OWNER TO `data-eng-prod`;
ALTER TABLE prod.sales.orders OWNER TO `data-eng-prod`;

#Tokens

Revoke the leaver's tokens after the consumers have moved:

databricks token-management delete <token-id>

Caution

Before moving a job, check the leaver's home folder for notebooks the job reads. Copy them to a shared or Git-backed location and point the job there before anyone cleans up that folder. If you delete them first, the job breaks again with "notebook not found".

#Rerun and verify

databricks jobs run-now $JOB_ID
databricks jobs list-runs --job-id $JOB_ID --limit 3 --output json | jq '.[] | {run_id, state}'

#Prevent it next time

• No person runs production. Every production job and pipeline uses run_as with a service principal, and a group owns it. Enforce this in bundle templates and code review.
• Weekly report. Schedule the system-table queries above with run_as_user_name NOT IN (<service principal IDs>) in the filter to catch production jobs running as people, and alert on any results.
• No PATs for shared tools. Power BI and other BI tools connect with a service principal. People use OAuth sign-in.
• Offboarding checklist. HR leaver tickets include a Databricks step: run step 2, transfer, then deactivate. Don't delete.
• Code in Git folders or shared folders. Never in /Users/<person>.