Shared Compute Design Guide: Multi-User Access and Security

Databricks Shared compute is an access mode that lets multiple users use a single cluster simultaneously. Built on Unity Catalog and process-level isolation, it prevents data leakage between users while drastically cutting cost by reducing cluster count.

That said, Shared mode comes with important constraints that Single User mode does not: no RDDs, no ML runtime, and restricted custom UDFs. This article walks through the isolation mechanism, the full set of constraints, the cost optimization payoff, and a decision flow for choosing between Single User and Shared.

Shared Access Mode Overview

Shared mode is an access mode where multiple users can use the same cluster resources at the same time. Each user attaches to the cluster and runs their own notebook, but under the hood process-level isolation prevents any user from touching another user's data or code.

Shared mode guarantees the following behaviors:

• Each user's code runs in an isolated process
• Unity Catalog permission checks are evaluated against each user's own credentials
• Users cannot access another user's temporary tables or views
• Cluster-level resources (CPU / memory) are scheduled fairly across users

Isolation Mechanism Built on Unity Catalog

Shared mode security is implemented as a two-layer stack: the Unity Catalog data governance layer plus process-level isolation.

Full List of Shared Mode Constraints

In exchange for stronger security, Shared mode restricts the following capabilities. This is a frequently tested topic on the exam and directly drives mode selection in production.

Cost Optimization Benefits

Cost efficiency is the headline benefit of Shared mode. With Single User mode, each user occupies their own cluster, so you end up paying for as many clusters as you have team members. With Shared mode, the whole team shares 1-2 clusters and you get the following cost savings.

• Fewer clusters: a team of 10 collapses from 10 clusters down to 1-2. Driver node cost alone drops significantly
• Less idle time: with Single User mode, a cluster keeps running even when its owner steps away. In Shared mode other users keep the resources busy, so waste shrinks
• More efficient auto-scaling: by aggregating multiple users' load, worker utilization rises and scale-down decisions become more efficient
• Shared instance pools: one pool serves every user, so pre-reserved instances stay busier

Single User vs Shared Decision Flow

Which mode to pick in production is driven by your workload requirements. Use the decision flow below.

[Workload needs ML libraries / GPU?]
  ├─ Yes → Single User (DBR ML)
  └─ No
      ├─ [Need RDD / SparkContext / low-level APIs?]
      │   ├─ Yes → Single User
      │   └─ No
      │       ├─ [Need custom init scripts / arbitrary libraries?]
      │       │   ├─ Yes → Single User
      │       │   └─ No
      │       │       ├─ [Multiple concurrent users?]
      │       │       │   ├─ Yes → Shared (best cost efficiency)
      │       │       │   └─ No → Single User or Shared (either works)
      │       │       └─ END
      │       └─ END
      └─ END

Shared vs Single User Comparison

What the Exam Tests

Shared mode comes up often on the Data Engineer Associate and Platform Admin exams. Expect these patterns:

• "Multiple users want to share a cluster to control cost" → Shared
• "Tried to use RDDs in Shared mode and got an error. Why?" → RDDs are blocked to preserve process isolation
• "Can I run ML workloads on a Shared cluster?" → No (DBR ML is not supported)
• "How do I use UDFs in Shared mode?" → Only UC-registered UDFs are allowed
• "What guarantees Shared mode security?" → UC permissions plus process isolation

Check with a Sample Question

Frequently Asked Questions