skip to content

Jinja and Macros

dbt compiles Jinja before it runs any SQL, which gives you loops, conditionals and reusable macros. Interviewers ask where the line is, because heavy templating produces SQL nobody can read or debug.

on this pageshow

questions

6

In dbt, when is the Jinja in a model rendered, and where can you read the resulting SQL?

level: juniorimportance: must knowfreq 70%

answer

  1. the warehouse never sees a curly brace
  2. two steps happen before the query runs
  3. there is a folder holding the rendered output
  4. target/compiled holds the model's SQL

basics

~10 s

dbt renders a model's Jinja into plain SQL before anything reaches the warehouse. The rendered SELECT is written to target/compiled/, and the same SQL wrapped in create or merge statements is written to target/run/.

solid answer

~40 s

A dbt model file is a Jinja template that produces SQL, not SQL itself. dbt first **parses** the project to build the DAG, then **compiles** each selected model by rendering every `{{ }}` expression and `{% %}` block into a finished SQL string, and only then **runs** it. The warehouse never sees a curly brace. The compiled SELECT lands in `target/compiled/<project>/models/...`, and the same SQL wrapped in whatever DDL or DML the materialization needs lands in `target/run/...`. `dbt compile` does the render without building anything, which is the fastest way to see what your templating actually produced. Because Jinja runs before execution, it can decide which SQL text exists, but it can never react to a row value — that still needs `case when`.

code

sql · 10 lines
sql
-- models/stg_orders.sql
select
    order_id,
    customer_id,
    ordered_at,
    amount_cents / 100.0 as amount
from {{ source('shop', 'orders') }}
{% if target.name != 'prod' %}
where ordered_at >= current_date - 3
{% endif %}

go deeper

for a junior

Be ready to say plainly that dbt turns the template into SQL first and sends only SQL, and to name target/compiled as the place you look when output surprises you.

for a middle

Explain the parse, compile and run phases and what differs between them, including that target/run holds the materialization wrapper while target/compiled holds the bare SELECT.

for a senior

Show the debugging habit: reproduce a failure by pulling the compiled statement and running it directly in a warehouse console, and know when compilation itself issues queries.

for a principal

Own the consequence for review and tooling — the artifact reviewers read is not the artifact that runs, so CI or review conventions have to surface compiled SQL for anything heavily templated.

## Two languages in one file A dbt model is a `.sql` file, but it is not pure SQL. It is a Jinja template whose *output* is SQL. Jinja is a Python templating language with three delimiters: - `{{ ... }}` — an expression; its value is rendered into the output text. - `{% ... %}` — a statement (`if`, `for`, `set`, `macro`); it controls what gets rendered but emits no text of its own. - `{# ... #}` — a comment; it disappears entirely. Everything outside those delimiters is copied through untouched. A model, read literally, is a small program that prints a query. ## The order of operations **Parse.** dbt reads every file in the project and renders enough of each model to discover its `ref()`, `source()` and `config()` calls. From those it builds the manifest and the dependency graph. During parsing the special variable `execute` is `False`, and introspective calls such as `run_query()` return nothing — dbt is still working out *what* to build. **Compile.** For each selected node dbt renders the template all the way to a SQL string and writes it to `target/compiled/<project_name>/models/<path>.sql`. During a real run `execute` is `True` here. **Run.** dbt wraps that compiled SELECT in the statements the chosen materialization requires — `create or replace view ... as`, `create table as`, a `merge`, a set of temp-table steps — and sends the wrapped statement to the warehouse. That final text is written to `target/run/...`. The practical consequence: when the warehouse returns a syntax error naming something you cannot find in your model file, you are reading the wrong artifact. Open the compiled file. ## What you can do with it Because a template runs before the query, you get ordinary programming constructs around your SQL: ```sql select * from {{ source('shop', 'orders') }} {% if target.name != 'prod' %} where ordered_at >= current_date - 3 {% endif %} ``` On a dev target that renders with the `where` clause and scans three days; on prod it renders without it. `target` is a dict-like context variable describing the connection dbt is using (`target.name`, `target.schema`, `target.type`). Other everyday context members include `{{ var('start_date', '2020-01-01') }}` for values declared in `dbt_project.yml` or passed with `--vars`, `{{ env_var('SOME_VAR') }}` for environment variables, and `{{ this }}` for the relation the current model writes to. ## Where to look, and with which command `dbt compile` renders every selected model into `target/compiled/` without building models. It still opens a warehouse connection, because introspective macros genuinely execute during compilation. `dbt run` compiles and then builds. `dbt build` compiles, builds, and runs the tests and snapshots interleaved with the models. In dbt Cloud the same compiled text appears in the IDE's compiled-code pane; on the command line it is a file you can `cat`, diff and paste into a warehouse console. Reading compiled SQL is the single most useful debugging habit in dbt. A macro that emits a stray comma, a loop that produces `and` with nothing after it, a variable that rendered empty — all of these are obvious in the compiled file and invisible in the source. ## The boundary Jinja cannot cross Jinja is a text step. It has no access to the rows in your tables at render time. `{% if amount > 0 %}` is not a filter on a column; `amount` there is a Jinja variable, and if it does not exist the expression is undefined. Row-level logic belongs in SQL: `case when amount > 0 then ... end`. The one way templating can look at data is by *running a query during compilation*, which is a deliberate, separate mechanism with its own guard. Quoting is the other common trap. `{{ var('country') }}` renders raw text, so a string literal needs its own quotes in the template: `where country = '{{ var("country") }}'`. Note the swapped inner quote style — nesting the same quote character breaks the expression. ## Whitespace Jinja statement lines leave blank lines behind in the output. The `{%- ... -%}` form trims whitespace on the marked side. This is cosmetic, but compiled SQL that a human can still read is worth the trimming, and it matters when a loop must place commas precisely.

  • How would you make a dbt model read a value supplied at run time rather than hard-coding it?
    Use `{{ var('start_date', '2020-01-01') }}` and declare the default in `dbt_project.yml`, overriding it on the command line with `--vars '{start_date: 2024-01-01}'`. For secrets or environment-specific endpoints use `{{ env_var('MY_VAR') }}` instead, which reads the process environment at render time and fails loudly if the variable is missing and no default is given.
  • How do you print a value from inside a macro while a dbt run is in progress?
    Call `{{ log("payment types: " ~ payment_types, info=True) }}`. Without `info=True` the message goes only to `logs/dbt.log`; with it, the message also appears on stdout. It is the Jinja equivalent of a print statement and is the quickest way to see what a variable held during compilation.
  • Why can a dbt model reference another model with ref() before either has been built?
    `ref()` is Jinja, evaluated during parsing. dbt records the dependency, builds the graph, decides the run order, and only then renders `ref()` into the fully-qualified relation name of the already-built upstream model. The graph is known before a single query executes, which is exactly why the render step exists.

It is like a mail merge: the letter template with placeholders is what you edit, but the printed letters are what actually get posted.

saying these in an interview costs you the question

  • Thinks the warehouse itself evaluates the Jinja
  • Cannot say where compiled SQL is written on disk
  • Believes Jinja conditionals can test column values per row
  • Debugs templating by guessing instead of reading compiled output
  • Assumes dbt run sends the model file verbatim to the database

context

open as a page

How do you define a reusable macro in a dbt project and call it from a model?

level: juniorimportance: must knowfreq 66%

basics

~20 s

Put a {% macro name(args) %} ... {% endmacro %} block in a .sql file under the macros/ directory, then call it from any model with {{ name(args) }}. The macro's rendered text is spliced into the model's SQL at compile time.

open as a page

In dbt, how do you install the dbt_utils package and call its macros from a model?

level: middleimportance: should knowfreq 55%

basics

~10 s

List the package with a version range in packages.yml, run dbt deps to install it under dbt_packages/, then call its macros namespaced by package name, for example {{ dbt_utils.star(from=ref('orders')) }}.

open as a page

In dbt, how do you generate model columns from values a query returns at run time?

level: middleimportance: should knowfreq 50%

basics

~20 s

Query the values with run_query(), guard the result with {% if execute %} because run_query returns nothing during parsing, pull the column with results.columns[0].values(), then loop over that list to emit one SQL expression per value.

open as a page

How do you decide how much Jinja templating a dbt project should allow in its models?

level: principalimportance: should knowfreq 38%

basics

~20 s

Allow Jinja that removes duplication without hiding what a model does: shared expressions, environment switches, config. Ban templating that decides a model's grain, joins or output columns, because reviewers then read a template while the warehouse runs something else.

open as a page

In dbt, how does adapter.dispatch let one macro work across Snowflake and BigQuery?

level: seniorimportance: nice to knowfreq 28%

basics

~10 s

adapter.dispatch resolves a macro name to an adapter-prefixed implementation at compile time: on Snowflake it looks for snowflake__my_macro and falls back to default__my_macro. You write one entry point plus one implementation per warehouse.

open as a page