Identity migration for Databricks between tenants: users, groups, service principals and ownership

#1. Why identities are the critical path

Grants, job owners, run_as, table owners, Git credentials, tokens and secret scope ACLs all reference identities. In a new tenant, users have new object IDs (and often new UPNs), and service principals have new application IDs.

#2. Build the identity map

Sources: account console export, SCIM GET /api/2.0/account/scim/v2/Users|Groups|ServicePrincipals, and the inventory of owners and run-as identities.

#3. Provision in the target

• Automatic identity management (Azure Databricks): users and groups from Entra ID are available without a SCIM app — enable it in the target account.
• Or configure the SCIM provisioning app in the target Entra tenant, assign the groups that should exist in Databricks.
• Add groups to workspaces with the right entitlements (Workspace access, Databricks SQL access, cluster creation only via policies).

Prefer Entra groups everywhere — grants to groups survive future people changes.

#4. Service principals

For each automation identity:

1. Create an Entra app registration / managed identity in the target tenant.
2. Add it to the Databricks account as a service principal; assign workspaces and roles.
3. Authentication: OAuth M2M with a Databricks-managed secret, or workload identity federation from GitHub/Azure DevOps (no secrets).
4. Update CI/CD and orchestrators (ADF linked services, Airflow connections) to the new identity.

#5. Ownership and run-as

• Jobs: set run_as to a service principal, never a person (Asset Bundles: run_as: service_principal_name: ...).
• UC objects: owners = groups.
• SQL warehouses, dashboards, alerts: owner or run-as identities updated; dashboards can embed credentials of the publisher — re-publish with the right identity.
• Git folders: each user creates their own Git credential in the target; jobs use a service principal's Git credential.

#6. Tokens and secrets

• Personal access tokens are not migrated. Disable PAT creation in the target if policy allows; use OAuth.
• Secret scopes: recreate as Azure Key Vault-backed scopes (or Databricks scopes) and load values from the source of truth (the vault), not by reading secrets out of the old workspace.

#7. Validate access

• Test with a sample user from each group: can browse the expected catalogs, run the expected jobs, nothing more.
• Query system.access.audit in the target for PERMISSION_DENIED during parallel run and fix gaps.

Warning

Watch out for email-based matching. If a user's UPN changes during the migration, any code or grant that uses the email string (row filters with current_user(), notification lists, /Users/<email>/ paths) silently stops matching. Search for email addresses in code and functions.