Row-level security and access control in Azure Databricks: from sign-in to the last masked column

Access in Azure Databricks works in four layers. A user needs all four to see a row of data:

1. Account and workspace: can they sign in at all?
2. Workspace objects and compute: can they open this notebook and attach to this cluster or warehouse?
3. Unity Catalog privileges: can they SELECT from this table?
4. Fine-grained rules: which rows and which column values do they see?

#Layer 1: who gets into the workspace

• Provision users and groups from Microsoft Entra ID with automatic identity management or SCIM. Manage groups, never individuals.
• Assign groups to the workspace with the User or Admin role in the account console.
• Entitlements per group: Workspace access, Databricks SQL access, Allow unrestricted cluster creation. Give the last one to nobody. Use cluster policies instead.
• Conditional Access in Entra (MFA, compliant device) applies at sign-in.
• Network: Private Link or IP access lists.

#Layer 2: objects and compute

Set them on folders so they inherit, and give production jobs a service principal as run_as. Then nobody's personal account is needed to run production.

#Layer 3: Unity Catalog privileges

Privileges inherit downwards: catalog, then schema, then table. Users also need USE CATALOG and USE SCHEMA on the parents.

GRANT USE CATALOG ON CATALOG sales TO `sales-analysts`;
GRANT USE SCHEMA, SELECT ON SCHEMA sales.curated TO `sales-analysts`;      -- every current and future table
GRANT MODIFY ON TABLE sales.curated.targets TO `sales-planners`;
GRANT ALL PRIVILEGES ON SCHEMA sales.sandbox TO `sales-engineers`;

SHOW GRANTS ON SCHEMA sales.curated;
SHOW GRANTS `sales-analysts` ON CATALOG sales;
REVOKE SELECT ON TABLE sales.curated.salaries FROM `sales-analysts`;

Good habits:

• Owners are groups, not people: use ALTER TABLE ... SET OWNER TO with a group such as sales-data-owners.
• Grant at schema level for read access. Per-table grants become unmanageable.
• Use workspace-catalog binding so the prod catalog isn't visible from dev workspaces.
• Use BROWSE on a catalog to let people discover tables without reading data.

#Layer 4a: row filters (row-level security)

A row filter is a SQL UDF that returns a boolean. Unity Catalog applies it to every query on the table.

-- mapping table: which group sees which region
CREATE TABLE sec.region_access (group_name STRING, region STRING);
INSERT INTO sec.region_access VALUES ('sales-emea', 'EMEA'), ('sales-apac', 'APAC');

CREATE OR REPLACE FUNCTION sec.region_filter(row_region STRING)
RETURN is_account_group_member('sales-global')                          -- sees everything
    OR EXISTS (SELECT 1 FROM sec.region_access a
               WHERE a.region = row_region AND is_account_group_member(a.group_name));

ALTER TABLE sales.curated.orders SET ROW FILTER sec.region_filter ON (region);
-- remove: ALTER TABLE sales.curated.orders DROP ROW FILTER;

Use is_account_group_member() (account-level groups), not is_member(), which checks workspace-local groups.

#Layer 4b: column masks

CREATE OR REPLACE FUNCTION sec.mask_email(email STRING)
RETURN CASE WHEN is_account_group_member('pii-readers') THEN email
            ELSE concat('***@', split_part(email, '@', 2)) END;

CREATE OR REPLACE FUNCTION sec.mask_salary(salary DECIMAL(12,2))
RETURN CASE WHEN is_account_group_member('hr') THEN salary ELSE NULL END;

ALTER TABLE hr.curated.employees ALTER COLUMN email  SET MASK sec.mask_email;
ALTER TABLE hr.curated.employees ALTER COLUMN salary SET MASK sec.mask_salary;
-- remove: ALTER TABLE hr.curated.employees ALTER COLUMN email DROP MASK;

A mask must return the same type as the column. Masks can take extra columns, for example USING COLUMNS (country), when the rule depends on another field.

#Alternative: dynamic views

Before row filters existed, teams published secured views. These still work, and are useful when you want a different shape for different consumers:

CREATE OR REPLACE VIEW sales.published.orders_v AS
SELECT order_id, region, amount,
       CASE WHEN is_account_group_member('pii-readers') THEN customer_email ELSE NULL END AS customer_email
FROM sales.curated.orders
WHERE is_account_group_member('sales-global') OR region = 'EMEA';

GRANT SELECT ON VIEW sales.published.orders_v TO `sales-analysts`;   -- and no grant on the base table

#Attribute-based access control (tags + policies)

Unity Catalog is adding ABAC: you tag columns or tables (for example pii=email) and define policies once at catalog or schema level. Those policies apply masks or filters to everything with that tag. If it is available in your region, it scales much better than per-table ALTER statements. Check its current preview/GA status and limits in the documentation before you standardise on it.

#Compute requirements

Row filters, column masks and dynamic views are enforced on:

• SQL warehouses and serverless compute,
• clusters in standard (shared) access mode, and
• dedicated (single-user) clusters on recent runtimes, where filtering is offloaded to serverless.

On older runtimes, dedicated clusters may refuse to query a filtered table, or require that offload. "No isolation" clusters can't access Unity Catalog at all. Check the runtime matrix in the docs for your version.

#Testing and auditing

-- test as yourself; for other identities, run the query from a test user or service principal in that group
SELECT current_user(), is_account_group_member('sales-emea');
SELECT region, count(*) FROM sales.curated.orders GROUP BY region;

-- which tables have filters or masks?
SELECT table_catalog, table_schema, table_name, filter_name FROM system.information_schema.row_filters;
SELECT table_catalog, table_schema, table_name, column_name, mask_name FROM system.information_schema.column_masks;

-- who changed permissions recently?
SELECT event_time, user_identity.email, action_name, request_params
FROM system.access.audit
WHERE action_name IN ('updatePermissions', 'createFunction') AND event_date >= current_date() - 7;

Tip

Keep the filter and mask functions in a dedicated sec schema that only the security team can modify. Anyone who can replace the function can change who sees the data.