Databricks Single User Compute: Permission Boundaries and ML vs SQL Execution

Databricks compute has several access modes, but Single User mode is the most flexible — it runs "dedicated to one user." It is the only mode that supports ML development, distributed training, and low-level Spark APIs (RDD / SparkContext) all at once, and under Unity Catalog data access is controlled using "the credentials of the attached user."

This article breaks down how Single User mode's execution identity works, the permission boundaries under UC, and the concrete differences between ML workload and SQL execution paths — plus the exam talking points and the design criteria you actually use in production.

What Is Single User Access Mode?

Single User is an access mode that dedicates a cluster to a single user. When you create the cluster, select "Single User" and pick a target user — no one else will be able to attach to the cluster. Concurrent notebook use is not possible, so it is not suited to team-wide sharing, but it has the following advantages no other mode can match:

• ML Runtime (DBR ML) is available: full support for GPUs, distributed training, and ML libraries
• You can use RDDs, SparkContext, and other low-level APIs
• No restrictions on custom UDFs (Python / Scala)
• No restrictions on init scripts or library installs
• Credential pass-through-style behavior: data is accessed using the attached user's permissions

How Execution Identity Works

The most important characteristic of Single User mode is its "execution identity." In this mode, every piece of code that runs on the cluster (SQL / Python / Scala / R) runs with the credentials of the user attached to the cluster.

Concretely, here is how it behaves:

Understanding exactly "whose permissions are used for access" is critical both on the exam and in real work. In particular, for job runs the job owner's permissions are used — not the user attached to the cluster — which often leads to the classic problem of "the table I could read in the notebook can't be read once it runs as a job."

Permission Boundaries Under Unity Catalog

In a Unity Catalog (UC) environment, Single User mode functions as the minimum unit of data governance. In workspaces where UC is enabled, data access always flows through UC's permission model (GRANT / DENY).

Here are the key points for understanding the permission boundaries:

• 3-tier permissions: catalog → schema → table: Without USE CATALOG and USE SCHEMA at the higher levels, even a SELECT grant on the table itself effectively gives you no access.
• Cluster access mode and UC permissions are independent: Single User mode doesn't mean you can see all the data — you are still limited to the GRANTs explicitly given to that user.
• External Location permissions: To read or write cloud storage directly, you need READ_FILES / WRITE_FILES on the UC External Location.
• Relationship with storage credentials: In Single User mode the storage credential is used transparently, but the user must hold ACCESS permission on the credential.

ML Workload vs SQL Execution Paths

Running an ML workload on a Single User cluster takes a very different internal path from running SQL. The comparison table below lays out the differences:

SQL Execution Paths: Notebook SQL vs SQL Warehouse

When you "run SQL" on Databricks, there are two options: running SQL on a Single User cluster, or using a Databricks SQL Warehouse (Serverless / Pro / Classic). Here is how they compare.

On the exam this often shows up as: "Which compute should you use if you want to mix Python and SQL in your ETL pipeline?" The answer is a Single User cluster (or a Job Cluster) — SQL Warehouse cannot run Python.

DLT Pipelines and Job Clusters

Delta Live Tables (DLT) pipelines automatically spin up a Single-User-equivalent cluster internally. You don't have to configure the cluster explicitly — the DLT runtime picks suitable settings. In that case the execution identity is the pipeline owner.

Job Clusters are usually created in Single User mode as well. When you define the job you specify "Single User," and data is accessed using the job owner's permissions. Unlike an All-Purpose Cluster, the cluster is automatically torn down once the job finishes, making it more cost-efficient.

Design Guidance for Production Use

Here are the best practices for using Single User mode in production:

• Always use Single User + DBR ML for ML development: Shared mode does not let you pick the ML Runtime, so all ML development should be done on Single User.
• Enforce auto-termination via Cluster Policy: Single User clusters are dedicated to one person, so idle time tends to be long. Cap autotermination_minutes at 10-30 minutes through a policy.
• Separate development and production-job clusters: Use an All-Purpose Cluster (Single User) for development and a Job Cluster (Single User) for production. This makes cost attribution and permission management cleaner.
• Combine with Instance Pools: Single User clusters start and stop frequently, so reserving instances in advance with an Instance Pool brings start-up time down to roughly 30 seconds to a minute.

Key Exam Points

Single User mode shows up frequently on Data Engineer Associate / ML Associate / ML Professional. The main question patterns are:

• "Which access mode do you need for distributed ML model training?" → Single User
• "What's the difference between Shared and Single User mode?" → RDD support, ML support, no concurrent use, etc.
• "What is the execution identity for a job run?" → The job owner (not the user attached to the cluster)
• "What compute mode do DLT pipelines use?" → A Single-User-equivalent cluster is configured automatically under the hood
• "Why can't I access a UC table?" → Missing USE CATALOG / USE SCHEMA permissions

Check Yourself with a Sample Question

Frequently Asked Questions