Applies toAzure Databricks diagnostic settings, system.access.audit table, Microsoft Sentinel / Log Analytics, Unity Catalog audit events
Two ways to get audit data
System table system.access.audit | Diagnostic settings → Log Analytics / Sentinel | |
|---|---|---|
| Where | Inside Databricks (Unity Catalog) | Azure Monitor workspace |
| Query | SQL | KQL |
| Fits | Platform team analysis, dashboards | SOC detections, correlation with Entra ID and Azure logs |
| Retention | Per system table retention (check docs) | Your Log Analytics retention/archive settings |
| Cost | Included storage within limits | Ingestion and retention charges |
Many organisations use both: SOC detections in Sentinel, platform analytics in system tables.
Event categories to collect
| Category | Why |
|---|---|
| accounts / login | Sign-ins, token creation, admin changes |
| unityCatalog | Grants, data access, external location changes |
| clusters, jobs | Who created/changed compute and jobs |
| secrets | Secret reads and ACL changes |
| workspace | Settings changes (IP lists, token settings) |
| notebook / databrickssql | Command and query activity (volume can be high) |
Enable verbose audit logs only if you need command-level detail and can afford the volume.
Detections worth building
| Detection | Signal |
|---|---|
| New PAT creation by a user in a workspace where PATs should be rare | tokenLogin / token generate events |
| Admin role granted | Account/workspace admin assignment events |
Grants to account users or broad groups on sensitive catalogs | Unity Catalog updatePermissions |
| Mass data download | Large result downloads, export events |
| Secret read by unusual principal | getSecret by identity not seen before |
| IP access list or network setting changed | Workspace configuration events |
-- Example: permission changes on Unity Catalog in the last 7 days
SELECT event_time, user_identity.email, action_name, request_params
FROM system.access.audit
WHERE service_name = 'unityCatalog'
AND action_name = 'updatePermissions'
AND event_date >= current_date() - 7
ORDER BY event_time DESC;
// Sentinel/Log Analytics: Databricks admin and token events (table names depend on diagnostic categories)
DatabricksAccounts
| where TimeGenerated > ago(1d)
| where ActionName in ("generateDbToken", "addPrincipalToGroup", "setAdmin")
| project TimeGenerated, Identity, ActionName, RequestParams, SourceIPAddress
Check the exact table and action names in your workspace; they differ by category.
Retention and access
- Keep audit data as long as your policy requires (often a year or more).
- Restrict who can read audit tables; they contain sensitive activity data.
- Protect logs from deletion by the people being audited.
Tip
Start with five detections tied to clear runbooks (who investigates, what to check). A small set that gets acted on beats a hundred noisy rules.
Checklist
- Audit path(s) chosen
- Required categories enabled
- Detections built with runbooks
- Retention meets policy
- Access to audit data restricted