Understanding dbt compile: How Jinja, Macros, and SQL Expansion Really Work

dbt compile is the no-execution side of dbt. It evaluates Jinja and macros, resolves dependencies, and expands templates into executable SQL — but it does not create tables or views. Understanding that boundary dramatically improves review efficiency and reduces failures.

This article unpacks what happens inside compile based on the official documentation, the exam points the Analytics Engineer certification likes to target, and the verification workflows that pay off in real projects.

What dbt compile Is: Purpose and Place in the Pipeline

dbt compile loads the project's nodes — models, snapshots, tests — evaluates Jinja and macros, resolves dependencies such as ref, source, and config, and writes the resulting executable SQL under target/compiled. Because it issues no DML or DDL against the database, you can safely inspect the final SQL.

It is ideal for pre-run checks, diff reviews, and detecting syntax breakage in CI. Being able to visually confirm how ephemeral models get inlined and how adapter-specific macros expand is a major win.

• Evaluates Jinja and macros to expand SQL templates
• Resolves ref() and source() into real table/view names (FQN, schema, alias)
• Inlines ephemeral models as CTEs into the referring model
• Does not execute against the database (no DDL/DML)
• Outputs go mainly to target/compiled and manifest.json

The dbt pipeline stages and where compile fits

Minimal model before/after (conceptual)

-- models/orders.sql (template)
with src as (
  select * from {{ source('raw', 'orders') }}
)
select * from src where order_date >= '{{ var('from_date', '2024-01-01') }}'

-- target/compiled/<project>/models/orders.sql (example, after expansion)
with src as (
  select * from RAW.SOURCE_ORDERS  -- resolved to an FQN by the adapter/naming rules
)
select * from src where order_date >= '2024-01-01'

What Happens During Compile: Jinja, Macros, and Resolution Order

compile identifies the selected nodes and renders each resource's SQL file inside a Jinja environment. Macro calls and adapter-specific implementations chosen via adapter.dispatch are expanded, and ref/source are replaced with the relation names that exist (or are about to be created). Because ephemeral models are not materialized upstream, they are embedded as CTEs inside the referring query.

Tests are translated into executable SQL during compile — for example, a SELECT that returns failing rows. The crucial point is that compile itself does not execute that SQL.

• Selection: determine the node set using selectors, tags, paths, and the + operator
• Rendering: Jinja evaluation, config() application, macro expansion
• Resolution: replace ref()/source()/exposure/metrics references with FQNs
• Inlining: materialized='ephemeral' becomes a CTE
• Emission: write rendered SQL to target/compiled and update manifest.json

How ephemeral inlining looks

-- models/_dim_ephemeral.sql
-- {{ config(materialized='ephemeral') }}
select id, lower(email) as email_norm from {{ ref('stg_users') }}

-- models/dim_users.sql
select u.*, e.email_norm
from {{ ref('stg_users') }} u
left join {{ ref('_dim_ephemeral') }} e on u.id = e.id

-- target/compiled/.../dim_users.sql (conceptual)
select u.*, e.email_norm
from PROD.ANALYTICS.STG_USERS u
left join (
  select id, lower(email) as email_norm from PROD.ANALYTICS.STG_USERS
) e on u.id = e.id

Output Artifacts: Where to Look for What

After compile runs, rendered SQL is written under target/compiled/<project>/.... That is the static snapshot of "the SQL that would be executed." manifest.json carries the DAG and metadata (relation names, dependencies, configs) and is useful for tooling integrations and diff analysis.

Unlike run or test, compile normally does not produce target/run. Do not expect execution logs or run_results.json from it.

• target/compiled: where rendered SQL is stored (use it for review and static inspection)
• target/manifest.json: node metadata and the DAG (use it to drive selection and CI logic)
• logs/dbt.log: clues about rendering warnings and unresolved references

A typical output tree

$ dbt compile --select +models/orders.sql

target/
  compiled/
    my_project/
      models/
        staging/
          stg_users.sql
        marts/
          orders.sql        # final SQL after expansion
  manifest.json            # DAG and metadata
logs/
  dbt.log

Branching and Evaluation Gotchas: is_incremental, vars, env vars

Because compile actually evaluates Jinja and macros, var(), env_var(), config()-driven branches, and adapter.dispatch dialect switches are all reflected in the output. SQL itself is not executed, however, so the final effect of conditions that depend on real data cannot be fully verified.

is_incremental() drives branches inside incremental models, but it depends on the compile-time context, connectivity, and whether the existing relation can be detected. compile does not issue DDL or DML, but it may still consult adapter information during macro evaluation. Final behavior must be confirmed with run or build.

• var('name', default) is expanded with values from --vars or dbt_project.yml
• env_var('NAME', default) pulls values from the shell environment
• adapter.dispatch reflects dialect differences across Snowflake, BigQuery, Redshift, and so on in the SQL
• Branches that use is_incremental() depend on compile-time assumptions; finalize verification with run or build

An incremental branch example (expanded at compile time)

-- models/fct_events.sql
{{ config(materialized='incremental', unique_key='id') }}
select * from {{ ref('stg_events') }}
{% if is_incremental() %}
  where _ingest_ts > (select max(_ingest_ts) from {{ this }})
{% endif %}

-- Note: compile expands the above, but assumptions like whether the table exists are not finalized until execution.

Selection and Scope: How Selectors and Dependencies Behave in compile

Selection in compile mirrors run: --select, --exclude, the + (parent/child) operator, tags, and resource types (model:, test:, and so on) are all available. That lets you narrow the review surface and inspect only the diffed expansion.

Since ephemeral upstreams get inlined into their downstream referrers, it is safer to compile the surrounding nodes too using the + operator or path-based selection to avoid missing changes.

• --select marts.orders+ to inspect the expansion including downstream dependencies
• --select tag:critical to inspect only the critical models
• Pick only changed nodes with --state (CI optimization)
• Use resource_type filters (model:, test:, snapshot:) to make the target explicit

Selection examples

$ dbt compile --select path:models/marts/
$ dbt compile --select model:orders+
$ dbt compile --select state:modified --state ./target

Troubleshooting and Exam Tips

Undefined vars, unresolved ref/source, and macro dispatch mismatches tend to surface during compile. A solid workflow is to knock out syntax errors early with dbt parse, then use compile to inspect the rendered output.

The Analytics Engineer exam frequently asks you to recognize the characteristics of compile: it does not execute, it inlines ephemeral models, it resolves ref/source to FQNs, and it produces target/compiled and manifest.json. In real projects, attaching the compiled SQL to PRs is the standard way to make reviews faster.

• Example error: 'Compilation Error: Required var not provided' → check --vars or dbt_project.yml
• Example error: 'ref not found' → check the name, package, and selection scope
• Verifying dialect differences: compare how adapter.dispatch expands in compiled output
• CI: dbt parse → dbt compile → static checks on branches → dbt build (when needed)

Debugging basics

$ dbt clean && dbt deps
$ dbt parse                # syntax and DAG
$ dbt compile -s +model:orders --vars '{from_date: 2024-01-01}'
$ tail -n 100 logs/dbt.log # look for macro and unresolved-reference traces

Check Your Understanding

Frequently Asked Questions