Delta Sharing Complete Guide: The Open Data Sharing Protocol

Delta Sharing is the world's first open protocol for securely sharing Delta Lake tables across organizations and platforms. Developed under the Linux Foundation, it uses a simple REST API-based design to enable real-time sharing without ever creating a copy of the data. All a provider has to do is create a Share object in Unity Catalog, and recipients can access the data from any tool — Pandas, Spark, Power BI, Tableau, dbt, and more — even without Databricks.

This article covers Delta Sharing's architecture, Unity Catalog integration, the difference between open sharing and Databricks-to-Databricks sharing, the types of objects you can share, Change Data Feed support, the security model, and a comparison with Snowflake Data Sharing. It also organizes the points that come up in the Data Governance domain of the Data Engineer Associate (DEA) exam.

Delta Sharing Overview

Delta Sharing was designed to fundamentally solve the problems of traditional data sharing methods — file copies, SFTP transfers, custom-built APIs, and ETL pipelines — namely stale data, heavy operational overhead, and complex security management. The heart of the protocol is zero-copy sharing: recipients read directly from the provider's cloud storage.

• Zero copy: no data duplication, so storage costs do not grow and provider-side updates are immediately reflected on the recipient side
• Open protocol: Apache 2.0 open source with no vendor lock-in. The recipient does not need Databricks
• REST API based: simple HTTPS + bearer token authentication that works with existing firewall and proxy configurations
• Fine-grained sharing controls: share entire tables, specific partitions, views, notebook output, and Volumes individually

Architecture

Delta Sharing consists of three core concepts — Provider, Share, and Recipient — plus the Delta Sharing server (which handles authentication, authorization, and discovery). In the Databricks managed Delta Sharing, Unity Catalog plays the role of the sharing server.

┌─────────────────────────────────────────────────────────┐
│                    Provider（プロバイダー）                │
│                                                         │
│  Unity Catalog                                          │
│  ├── Share: customer_data_share                         │
│  │    ├── Schema: sales                                 │
│  │    │    ├── Table: orders (全パーティション)            │
│  │    │    └── Table: customers (region='APAC'のみ)      │
│  │    └── Schema: analytics                             │
│  │         └── View: daily_summary                      │
│  │                                                      │
│  ├── Recipient: partner_a  ──→ Token / D2D認証           │
│  └── Recipient: partner_b  ──→ Token / D2D認証           │
│                                                         │
│  Cloud Storage (S3/ADLS/GCS)                            │
│  └── Delta Lake files (Parquet + _delta_log)            │
└───────────────┬─────────────────────────────────────────┘
                │  REST API (HTTPS + Bearer Token)
                │  短期間の署名付きURL発行
                ▼
┌──────────────────────────────────────────────────────────┐
│              Recipient（レシピエント）                      │
│                                                          │
│  ┌──────────┐ ┌──────────┐ ┌────────┐ ┌──────────────┐  │
│  │ Pandas   │ │ Spark    │ │Power BI│ │ Databricks   │  │
│  │ (Python) │ │ (OSS)    │ │Tableau │ │ ワークスペース │  │
│  └──────────┘ └──────────┘ └────────┘ └──────────────┘  │
│  ※ Databricksライセンス不要（オープン共有の場合）           │
└──────────────────────────────────────────────────────────┘

The data flow goes as follows: (1) the recipient sends a request to the Delta Sharing server with an auth token → (2) the server validates authentication and authorization, then returns short-lived pre-signed URLs for the target files → (3) the recipient uses the pre-signed URLs to read the Parquet files directly from cloud storage. Because the recipient accesses provider storage directly, the sharing server is not a bottleneck, and the system scales smoothly even with very large datasets.

Unity Catalog Integration

Databricks' managed Delta Sharing is natively integrated with Unity Catalog. Shares, Recipients, and Providers are all managed as Unity Catalog securable objects, so GRANT/REVOKE permission management, audit logs via System Tables, and data lineage all apply as-is.

-- ========================================
-- 1. Shareオブジェクトの作成
-- ========================================
CREATE SHARE IF NOT EXISTS customer_data_share
COMMENT 'パートナー企業向け顧客データ共有';

-- ========================================
-- 2. Shareにテーブルを追加
-- ========================================
-- テーブル全体を共有
ALTER SHARE customer_data_share
ADD TABLE prod_catalog.sales.customers;

-- パーティションフィルタで部分共有（APAC地域のみ）
ALTER SHARE customer_data_share
ADD TABLE prod_catalog.sales.orders
PARTITION (region = 'APAC');

-- Change Data Feed（CDC履歴）付きで共有
ALTER SHARE customer_data_share
ADD TABLE prod_catalog.sales.transactions
WITH HISTORY;

-- ========================================
-- 3. レシピエントの作成
-- ========================================
-- オープン共有（トークンベース）
CREATE RECIPIENT IF NOT EXISTS partner_company_a
COMMENT 'Partner Company A - APAC region';

-- Databricks-to-Databricks共有
CREATE RECIPIENT partner_company_b
USING ID 'aws:us-west-2:workspace-id-12345';

-- ========================================
-- 4. レシピエントにShareへのアクセスを付与
-- ========================================
GRANT SELECT ON SHARE customer_data_share
TO RECIPIENT partner_company_a;

-- ========================================
-- 5. 共有状況の確認
-- ========================================
SHOW ALL IN SHARE customer_data_share;
SHOW GRANTS ON SHARE customer_data_share;
DESCRIBE RECIPIENT partner_company_a;

Recipient-Side Access

Open sharing recipients access the data using a profile file (a .share file) provided by the provider. The profile file contains the sharing server endpoint URL, the bearer token, and an expiration time.

# ─── プロファイルファイル（config.share）の構造 ───
# {
#   "shareCredentialsVersion": 1,
#   "endpoint": "https://<workspace>.cloud.databricks.com/api/2.0/delta-sharing/",
#   "bearerToken": "xxxxxxxxxxxxx",
#   "expirationTime": "2026-12-31T23:59:59Z"
# }

import delta_sharing

profile_file = "/path/to/config.share"

# 共有テーブルの一覧を取得
client = delta_sharing.SharingClient(profile_file)
shares = client.list_shares()
for share in shares:
    schemas = client.list_schemas(share)
    for schema in schemas:
        tables = client.list_tables(schema)
        print(f"{share.name}.{schema.name}: {[t.name for t in tables]}")

# Pandas DataFrameとして読み込み
df_pandas = delta_sharing.load_as_pandas(
    f"{profile_file}#customer_data_share.sales.customers"
)

# Spark DataFrameとして読み込み
df_spark = delta_sharing.load_as_spark(
    f"{profile_file}#customer_data_share.sales.orders"
)

# Change Data Feed（CDC履歴）の読み込み
cdf = delta_sharing.load_table_changes_as_pandas(
    f"{profile_file}#customer_data_share.sales.transactions",
    starting_version=0
)

Open Sharing vs Databricks-to-Databricks Sharing

Delta Sharing has two sharing modes. Choose between them based on the recipient's environment and your governance requirements.

Shareable Objects

Tables are not the only thing you can share with Delta Sharing. In a Databricks environment integrated with Unity Catalog, you can share the following objects.

-- ビューの共有（D2Dのみ）
ALTER SHARE customer_data_share
ADD VIEW prod_catalog.analytics.daily_revenue_summary;

-- Volumeの共有（D2Dのみ）
ALTER SHARE customer_data_share
ADD VOLUME prod_catalog.raw.model_artifacts;

-- ノートブック出力の共有
ALTER SHARE customer_data_share
ADD NOTEBOOK prod_catalog.reports.quarterly_analysis;

-- テーブルからオブジェクトを削除
ALTER SHARE customer_data_share
REMOVE TABLE prod_catalog.sales.deprecated_table;

Sharing Change Data Feed (CDC Support)

When you share Delta Lake's Change Data Feed (CDF) through Delta Sharing, recipients can pull only the deltas (INSERT/UPDATE/DELETE) since their last read instead of re-reading the full dataset. This is useful for incrementally syncing large tables and building near-real-time data pipelines.

-- プロバイダー側: CDFを有効化してShareに追加
ALTER TABLE prod_catalog.sales.transactions
SET TBLPROPERTIES (delta.enableChangeDataFeed = true);

ALTER SHARE customer_data_share
ADD TABLE prod_catalog.sales.transactions
WITH HISTORY;

-- レシピエント側（Python）: 差分データの取得
import delta_sharing

cdf = delta_sharing.load_table_changes_as_pandas(
    "/path/to/config.share#customer_data_share.sales.transactions",
    starting_version=5,       # バージョン5以降の変更
    ending_version=10         # バージョン10まで
)

# CDFの各行には _change_type カラムが含まれる
# _change_type: "insert" / "update_preimage" / "update_postimage" / "delete"
print(cdf[["_change_type", "_commit_version", "_commit_timestamp"]].head())

Security and Permissions

Delta Sharing is protected by multiple layers of security. Because Unity Catalog's permission system applies directly, it stays consistent with your existing governance policies.

• Authentication: bearer token (open sharing) or Unity Catalog cross-workspace authentication (D2D)
• Authorization: access control at the Share, table, and partition levels
• Token expiration: set a per-recipient token lifetime; access is automatically cut off after expiry
• IP access lists: set an IP allowlist per recipient to restrict where connections can originate
• Audit logs: every access is recorded in system.access.audit. You can query who, when, which Share, which table, and how many times via SQL.

-- ========================================
-- Share関連の権限管理
-- ========================================

-- USE SHARE: Shareオブジェクトの詳細を表示する権限
GRANT USE SHARE ON METASTORE TO data_governors;

-- SET SHARE PERMISSION: Shareの権限を他ユーザーに委譲できる権限
GRANT SET SHARE PERMISSION ON METASTORE TO share_admins;

-- CREATE SHARE: 新しいShareを作成する権限
GRANT CREATE SHARE ON METASTORE TO data_engineers;

-- CREATE RECIPIENT: レシピエントを作成する権限
GRANT CREATE RECIPIENT ON METASTORE TO data_engineers;

-- レシピエントのトークンをローテーション（再発行）
ALTER RECIPIENT partner_company_a ROTATE TOKEN;

-- レシピエントにIPアクセスリストを設定
ALTER RECIPIENT partner_company_a
SET IP_ACCESS_LIST = ('203.0.113.0/24', '198.51.100.0/24');

-- レシピエントの詳細・アクセス状況を確認
DESCRIBE RECIPIENT partner_company_a;
SHOW GRANTS TO RECIPIENT partner_company_a;

Comparison with Snowflake Data Sharing

Snowflake also supports cross-account data sharing via its Secure Data Sharing feature. Here we lay out the design and constraint differences between Delta Sharing and Snowflake Data Sharing.

Exam Focus Points (DEA: Data Governance)

On the Data Engineer Associate (DEA) exam, Delta Sharing appears in the Data Governance domain (about 17% of the exam). It is tightly tied to the Unity Catalog section, so make sure to learn Share, Recipient, and permission management as a set.

Sample Question

Frequently Asked Questions