Databricks Audit Logs: Auditing, Tracking, and Compliance in Practice

"Who, when, what data, what operation?" — if you cannot answer that question instantly, you cannot investigate security incidents or respond to compliance audits. Databricks Audit Logs automatically record every operation in the workspace and make them queryable via SQL through the system.access.audit table (System Tables).

This article walks through the structure of audit logs, the major event catalog, practical SQL query examples, Log Delivery setup (AWS S3 / Azure Event Hubs / GCS), key points for SOC 2 / ISO 27001 compliance, and the patterns asked on the Databricks certification exams.

Audit Logs Overview (system.access.audit)

Databricks audit logs are delivered as part of System Tables. System Tables are schemas under the system catalog — read-only Delta Tables that Databricks populates and refreshes automatically. Audit logs live in system.access.audit.

-- Inspect the audit log table schema
DESCRIBE TABLE system.access.audit;

-- Key columns
-- event_time        TIMESTAMP   When the event occurred
-- event_date        DATE        Event date (partition key)
-- workspace_id      BIGINT      Workspace ID
-- action_name       STRING      Operation type (e.g. createCluster, getTable)
-- user_identity     STRUCT      User / service principal performing the action
-- service_name      STRING      Service name (e.g. clusters, unityCatalog, jobs)
-- request_params    MAP         Request parameters (details of the target object)
-- response          STRUCT      Response info (status code, error message)
-- source_ip_address STRING      Source IP address of the request
-- audit_level       STRING      Log level (ACCOUNT_LEVEL / WORKSPACE_LEVEL)

Catalog of Key Events

Practical SQL Query Examples

Detecting unauthorized access (monitoring failed logins)

SELECT
  event_time,
  user_identity.email AS user_email,
  source_ip_address,
  response.status_code,
  response.error_message
FROM system.access.audit
WHERE action_name = 'login'
  AND response.status_code != 200
  AND event_date >= current_date() - INTERVAL 7 DAYS
ORDER BY event_time DESC;

Auditing access to Unity Catalog tables

SELECT
  event_date,
  user_identity.email AS user_email,
  action_name,
  request_params['full_name_arg'] AS table_name,
  COUNT(*) AS access_count
FROM system.access.audit
WHERE service_name = 'unityCatalog'
  AND action_name IN ('getTable', 'createTable', 'deleteTable')
  AND event_date >= current_date() - INTERVAL 30 DAYS
GROUP BY event_date, user_identity.email, action_name,
         request_params['full_name_arg']
ORDER BY event_date DESC, access_count DESC;

Job run history and failure analysis

SELECT
  event_time,
  user_identity.email AS triggered_by,
  request_params['jobId'] AS job_id,
  request_params['runId'] AS run_id,
  action_name,
  CASE
    WHEN action_name = 'runSucceeded' THEN 'SUCCESS'
    WHEN action_name = 'runFailed' THEN 'FAILED'
    ELSE action_name
  END AS status
FROM system.access.audit
WHERE service_name = 'jobs'
  AND action_name IN ('runSucceeded', 'runFailed', 'runNow')
  AND event_date >= current_date() - INTERVAL 14 DAYS
ORDER BY event_time DESC;

Tracking permission changes (GRANT/REVOKE)

SELECT
  event_time,
  user_identity.email AS granted_by,
  action_name,
  request_params['securable_type'] AS object_type,
  request_params['full_name_arg'] AS object_name,
  request_params['principal'] AS granted_to,
  request_params['privileges'] AS privileges
FROM system.access.audit
WHERE service_name = 'unityCatalog'
  AND action_name IN ('grantPermission', 'revokePermission', 'updatePermissions')
  AND event_date >= current_date() - INTERVAL 90 DAYS
ORDER BY event_time DESC;

Cluster cost-related auditing

SELECT
  event_date,
  user_identity.email AS creator,
  request_params['cluster_name'] AS cluster_name,
  request_params['node_type_id'] AS instance_type,
  request_params['num_workers'] AS num_workers,
  request_params['autotermination_minutes'] AS auto_terminate_min
FROM system.access.audit
WHERE service_name = 'clusters'
  AND action_name = 'create'
  AND event_date >= current_date() - INTERVAL 30 DAYS
ORDER BY event_date DESC;

Log Delivery Setup

On top of SQL analysis via System Tables, you can also deliver audit logs to external storage or SIEM tools. Log Delivery is configured from the Account Console (or Account API) and enabled at the account level.

# AWS: configure Log Delivery via the Account API
curl -X POST "https://accounts.cloud.databricks.com/api/2.0/accounts/<account_id>/log-delivery" \
  -H "Authorization: Bearer <admin_token>" \
  -H "Content-Type: application/json" \
  -d '{
    "log_type": "AUDIT_LOGS",
    "output_format": "JSON",
    "credentials_id": "<credential_config_id>",
    "storage_configuration_id": "<storage_config_id>",
    "delivery_path_prefix": "databricks/audit-logs"
  }'

System Tables vs Log Delivery: Comparison

Compliance

Databricks audit logs can support the major compliance frameworks.

Recommended best practices for compliance:

• Two-tier setup: System Tables for day-to-day monitoring plus Log Delivery for long-term archival
• Configure alerts on permission-change events (grantPermission / revokePermission)
• Detect login-failure thresholds being exceeded (early detection of brute-force attacks)
• Quarterly audit log reviews and permission inventories
• Apply encryption, access control, and versioning to Log Delivery destinations (S3/ADLS)

What the Exam Asks

Audit Logs are in scope for Data Engineer Associate / Professional and for Administration-related exams.

• "Which System Tables table stores audit logs?" → system.access.audit
• "How do I see who accessed which table?" → Filter system.access.audit with service_name='unityCatalog'
• "How do I forward audit logs to an external SIEM tool?" → Configure Log Delivery
• "What format is System Tables data?" → Delta Table (directly queryable with SQL)
• "How do I extend audit log retention?" → Archive to external storage via Log Delivery

Check Your Understanding

Frequently Asked Questions