Migrating Unity Catalog to a new tenant: metastore, catalogs, storage credentials, external locations and grants

#1. Metastore

One metastore per region per account. In the target account console: Catalog → Create metastore in the target region (a metastore-level managed storage location is optional; prefer storage per catalog). Assign it to the target workspaces.

Metastore admins: a group (uc-metastore-admins), not a person.

#2. Storage credentials and external locations

Old credentials point at managed identities / IAM roles in the old tenant — they can't be reused.

Azure: create an Access Connector for Azure Databricks (managed identity) in the target subscription, give it Storage Blob Data Contributor on the target storage accounts, and register it:

resource "databricks_storage_credential" "lake" {
  name = "sc-lake-prod"
  azure_managed_identity { access_connector_id = azurerm_databricks_access_connector.uc.id }
}
resource "databricks_external_location" "bronze" {
  name            = "el-bronze-prod"
  url             = "abfss://bronze@stlakeprod.dfs.core.windows.net/"
  credential_name = databricks_storage_credential.lake.id
}

AWS: a new IAM role with a trust policy for the target Databricks account's UC role and the external ID.

#3. Catalogs and schemas as code

resource "databricks_catalog" "sales" {
  name         = "prod_sales"
  storage_root = "abfss://sales@stlakeprod.dfs.core.windows.net/managed"
  owner        = "grp-sales-data-owners"
}
resource "databricks_schema" "orders" {
  catalog_name = databricks_catalog.sales.name
  name         = "orders"
  owner        = "grp-sales-data-owners"
}

Use the migration to fix naming (<env>_<domain>), and give each catalog its own managed storage location (separation and simpler data copy).

#4. Translate grants

Export grants from the source (information_schema.*_privileges), then map principals through your identity mapping table:

import csv
mapping = {r["source"]: r["target"] for r in csv.DictReader(open("identity_map.csv"))}
cat_map = {"prod": "prod_sales"}   # source catalog -> target catalog
rows = spark.sql("""SELECT grantee, table_catalog, table_schema, table_name, privilege_type
                    FROM system.information_schema.table_privileges""").collect()
for r in rows:
    tgt = mapping.get(r.grantee)
    if tgt:
        print(f"GRANT {r.privilege_type} ON TABLE {cat_map[r.table_catalog]}.{r.table_schema}.{r.table_name} TO `{tgt}`;")

Better: grant at schema or catalog level to groups in the target (databricks_grants in Terraform), instead of reproducing thousands of table-level grants.

#5. Volumes, functions, views, row filters and masks

• Volumes: create in the target; copy files (see the data migration post).
• Views and SQL functions: export DDL (SHOW CREATE TABLE for views; DESCRIBE FUNCTION EXTENDED) and replace catalog names.
• Row filters and column masks: recreate the functions, then reapply ALTER TABLE ... SET ROW FILTER / ALTER COLUMN ... SET MASK.
• Tags and comments: export from information_schema.table_tags / column_tags and reapply.

#6. Hive metastore leftovers

Don't carry hive_metastore into the target. Options: upgrade to UC in the source first with UCX, or migrate HMS tables directly into target UC catalogs as part of the data copy.

#7. Lineage and history

Lineage and audit history in system tables are not transferred. Export what you need for compliance (to a table or storage) before decommissioning the source.

Tip

Build the target UC entirely in Terraform from day one. Re-running terraform apply per wave is safer than clicking through hundreds of grants, and it gives you the governance-as-code you probably didn't have before.