Microsoft Fabric Lakehouse Intro: OneLake, Shortcut, Direct Lake & Medallion

Microsoft Fabric Lakehouse is the unified data platform offered by the Data Engineering workload of Microsoft Fabric. It is Microsoft's implementation of the Lakehouse architecture, combining the characteristics of a Data Lake and a Data Warehouse. Data is stored internally in Delta Parquet format in OneLake (Fabric's unified storage) and processed with Apache Spark. The SQL Endpoint supports read-only T-SQL queries, and Power BI Direct Lake mode delivers ultra-low-latency analytics. Typical use cases: 1) unifying data lake and data warehouse, 2) implementing Bronze / Silver / Gold Medallion Architecture, 3) PySpark-based data processing, 4) Notebook-based interactive development. It is a core topic of the DP-700 (Fabric Data Engineer Associate) exam and the latest data platform rapidly replacing Azure Synapse / Databricks-style stacks.

Lakehouse: Apache Spark-based, processed with PySpark / Spark SQL / Scala, Delta Lake format, supports structured + semi-structured + unstructured data, Notebook-based development, streaming ingestion via Auto Loader, flexible Schema-on-Read. Warehouse: T-SQL based, runs on the Microsoft SQL engine, ACID transactions + Distributed Query, primarily structured data, stored procedures and functions, strict Schema-on-Write, optimized for BI / reporting. Decision guide: 1) Big Data / ML / Streaming -> Lakehouse, 2) SQL analysts / BI reporting -> Warehouse, 3) Mixed -> run both on the same OneLake (joinable via Cross-warehouse Query). The standard pattern for new Fabric projects is the hybrid 'Lakehouse (Bronze/Silver) + Warehouse (Gold)' setup, letting data engineers and BI developers collaborate on the same data platform.

OneLake is Fabric's unified storage layer, shared by every Fabric workload (Lakehouse, Warehouse, KQL Database, Power BI Semantic Model). It is built on ADLS Gen2 and uses the Delta Parquet format. Shortcut: a feature that logically references external data sources from OneLake (no actual data copy). Supported sources: 1) ADLS Gen2 Storage Account, 2) Amazon S3, 3) Google Cloud Storage, 4) Dataverse, 5) OneLake in another Workspace. Typical use cases: 1) directly use existing ADLS Gen2 data from Fabric (no migration), 2) multi-cloud data integration over AWS S3, 3) data sharing across Workspaces (access control stays with the source Workspace). Shortcuts let Fabric achieve 'data integration without data movement,' making them a cornerstone of cross-cloud data lake strategies.

Direct Lake is one of Power BI Semantic Model's connection modes in Microsoft Fabric. It is the fast mode that lets Power BI analyze Delta Parquet data in Lakehouse / Warehouse directly. Compared with the traditional Power BI connection modes: 1) Import Mode (copies data into Power BI; fastest but not fresh, requires periodic refresh), 2) DirectQuery (hits the source per query; fresh but slow), 3) Direct Lake (reads Delta Parquet directly; Import-level speed plus fresh data). How it works: Power BI loads Delta Parquet files directly into memory -> user queries execute -> analytics over the latest Lakehouse / Warehouse data. No refresh required (always up-to-date), handles larger datasets than Import (billions of rows+). Requires Power BI Premium / Fabric Capacity. It is one of Fabric's biggest differentiators and dramatically streamlines collaboration between data engineers and BI developers.

Creating a Lakehouse in a Fabric Workspace: 1) Workspace -> New Item -> Lakehouse -> enter a name (created in seconds), 2) Lakehouse Explorer shows two sections, Tables (Delta Tables) and Files (Raw Files), 3) upload files into Files (CSV / JSON / Parquet / various formats), 4) run PySpark code in a Notebook to process data, 5) create a Delta Table with df.write.format('delta').saveAsTable('table_name'), 6) run T-SQL queries via the SQL Endpoint, 7) BI analytics through the auto-created Power BI Semantic Model (Direct Lake connection). Sample Spark code: spark.read.format('csv').load('Files/raw/').filter(F.col('amount') > 0).write.format('delta').saveAsTable('cleaned_data'). Fabric's development experience tightly integrates Notebooks and Lakehouse, giving you a Databricks-like feel while staying entirely inside the Microsoft ecosystem.

Medallion implementation patterns on Fabric Lakehouse: 1) create three Lakehouses (Bronze / Silver / Gold) in the Lakehouse Workspace (environment isolation), or split by Schema/Folder inside one Lakehouse, 2) Bronze: drop source data as-is into Files (CSV / JSON / Parquet), or store as minimally-schemaed Delta Tables in Tables, 3) Silver: read from Bronze in PySpark -> cleansing / JOIN / type casting -> write to Delta Tables, 4) Gold: aggregate from Silver -> store as Star Schema or Wide Tables in Delta format, 5) orchestrate the Bronze -> Silver -> Gold dependencies with Pipelines, 6) analyze Gold via Power BI Direct Lake. Implement each stage in Fabric Notebooks; persistent Spark sessions improve dev productivity. Databricks veterans pick it up quickly, and the structure is approachable for new data engineers too.

Fabric's billing unit is the Capacity Unit (CU). SKUs are F2 (2 CU), F4, F8, F16, F32, F64, F128, F256, F512, F1024, F2048 (2048 CU). Lakehouse Spark compute is dynamically allocated from CUs, and every action — Pipeline Activities, Power BI refresh, Notebook execution — consumes CUs. Sample pricing: F64 (production standard, ~640k JPY/month), F128 (~1.3M JPY/month). Cost optimization: 1) use F2 / F4 for dev (pausable) and F64+ only for prod, 2) Reserved Instance (1-year) saves ~41%, 3) Auto Scale aligns with demand, 4) Pause off-hours for Dev / Stage, 5) analyze usage patterns and right-size with the Capacity Metrics App, 6) split capacities by workload (heavy Lakehouse processing on its own capacity). In production, monthly right-sizing via Capacity Metrics is the standard practice.

DP-700 (Fabric Data Engineer Associate) is the headline certification covering Fabric Lakehouse in depth (Lakehouse construction, Notebooks, PySpark, Pipelines, Power BI Direct Lake integration). DP-600 (Fabric Analytics Engineer Associate) covers Lakehouse usage from a BI / modeling angle, AZ-305 (Solutions Architect Expert) covers Fabric adoption from an architect's perspective, and AI-103 (GA 2026-06) covers integration between Lakehouse and Azure AI Foundry for RAG. Pairing this with Databricks certifications (Data Engineer Associate / Professional) is also valued in multi-cloud data platform projects. Microsoft Fabric is positioned as the centerpiece of Azure's data platform for the next 5-10 years and is an essential skill for data engineers.