skip to content

In dbt, how does adapter.dispatch let one macro work across Snowflake and BigQuery?

level: seniorimportance: nice to knowfreq 28%

answer

  1. one call site, several dialects
  2. a naming convention decides which body runs
  3. the prefix is the warehouse's own name
  4. adapter name plus double underscore, with a default fallback

basics

~10 s

adapter.dispatch resolves a macro name to an adapter-prefixed implementation at compile time: on Snowflake it looks for snowflake__my_macro and falls back to default__my_macro. You write one entry point plus one implementation per warehouse.

solid answer

~40 s

`adapter.dispatch` is dbt's mechanism for cross-database macros. The entry point is a thin macro that calls `{{ return(adapter.dispatch('datediff', macro_namespace='dbt_utils')(a, b, part)) }}`; at compile time dbt looks for an implementation named `<adapter>__datediff` — `snowflake__datediff`, `bigquery__datediff`, `postgres__datediff` — and falls back to `default__datediff` if there is none. That is how a package supports many warehouses without an `{% if target.type %}` chain. The second half is the `dispatch` config in `dbt_project.yml`: giving a `macro_namespace` a `search_order` that lists your project first lets you supply your own `snowflake__datediff` and have it win over the package's, which is the supported way to patch a package macro without forking it.

code

sql · 12 lines
sql
-- macros/datediff.sql
{% macro datediff(first_date, second_date, datepart) %}
    {{ return(adapter.dispatch('datediff', macro_namespace='my_package')(first_date, second_date, datepart)) }}
{% endmacro %}

{% macro default__datediff(first_date, second_date, datepart) %}
    datediff({{ datepart }}, {{ first_date }}, {{ second_date }})
{% endmacro %}

{% macro bigquery__datediff(first_date, second_date, datepart) %}
    date_diff(cast({{ second_date }} as date), cast({{ first_date }} as date), {{ datepart }})
{% endmacro %}

go deeper

for a junior

You will rarely write this. Recognising that dbt resolves warehouse-specific macro bodies by a naming convention is enough at this level.

for a middle

Explain the resolution: adapter prefix, double underscore, default fallback, all decided at compile time rather than by the database.

for a senior

Show that you have used it — writing a multi-adapter macro, or overriding a package implementation through search_order rather than forking, and knowing the blast radius that override carries.

for a principal

Own the strategy: whether an internal macro library is dispatched at all, how a multi-warehouse or migration period is supported, and how overrides of third-party packages are reviewed and documented.

## The problem it solves SQL dialects diverge on exactly the things utility macros wrap: date arithmetic, string aggregation, safe casting, hashing, type names. A package that wants to run on Snowflake, BigQuery, Redshift, Postgres and Databricks needs one call site and several implementations. The naive answer is a conditional on `target.type`, which produces an ever-growing `{% elif %}` ladder that no third party can extend. Dispatch replaces it with a naming convention plus a resolution order. ## The naming convention An implementation is a macro whose name is `<adapter_prefix>__<macro_name>`, with a **double** underscore separating them: ```sql {% macro default__datediff(first_date, second_date, datepart) %} datediff({{ datepart }}, {{ first_date }}, {{ second_date }}) {% endmacro %} {% macro bigquery__datediff(first_date, second_date, datepart) %} date_diff(cast({{ second_date }} as date), cast({{ first_date }} as date), {{ datepart }}) {% endmacro %} ``` The prefix is the adapter name in lowercase — `snowflake`, `bigquery`, `redshift`, `postgres`, `databricks`, `spark`. `default__` is the fallback used when no adapter-specific implementation exists. ## The entry point The macro that models call is a one-liner that hands off to dispatch: ```sql {% macro datediff(first_date, second_date, datepart) %} {{ return(adapter.dispatch('datediff', macro_namespace='my_package')(first_date, second_date, datepart)) }} {% endmacro %} ``` Read the shape carefully: `adapter.dispatch(...)` **returns a macro**, and the trailing `(...)` calls it with the real arguments. Forgetting the second parentheses is the classic mistake, and it renders the macro object rather than its output. The `macro_namespace` argument tells dbt which package the macro conceptually belongs to. That is what makes the search order configurable by the *consuming* project rather than fixed by the package author. ## Overriding a package's implementation Suppose dbt_utils' Snowflake implementation of some macro does not do what your warehouse setup needs. You do not fork the package. You write your own `snowflake__that_macro` in your project's `macros/` directory and then tell dbt to look in your project first: ```yaml # dbt_project.yml dispatch: - macro_namespace: dbt_utils search_order: ['my_project', 'dbt_utils'] ``` Now every dispatched call in that namespace searches `my_project` for `snowflake__<name>`, then `dbt_utils`. Models keep calling `{{ dbt_utils.that_macro(...) }}`; the implementation underneath changed. This is genuinely powerful and genuinely a footgun: the override applies to every call in the project including calls made from *inside* the package's other macros, so a subtly different implementation can change behaviour far from where you edited. ## Resolution, step by step On a Snowflake target, `adapter.dispatch('datediff', macro_namespace='dbt_utils')` will: 1. Walk the configured `search_order` for the `dbt_utils` namespace, defaulting to the package itself if nothing is configured. 2. In each candidate project, look for `snowflake__datediff`. 3. If none is found anywhere, look for `default__datediff` in the same order. 4. Raise a compilation error if neither exists. All of this happens during compilation, so a dispatch mistake surfaces as a compile error naming the macro, not as a warehouse error. ## When you actually need it Most analytics teams run on one warehouse and never need dispatch for their own macros — a plain macro is simpler and clearer. It becomes worth the ceremony when: - you are **publishing a package** meant to run on more than one adapter; - you maintain an internal macro library shared by projects on different warehouses, for example a Snowflake production project and a DuckDB or Postgres project used for local development and testing; - you need to **patch a package's behaviour** for your adapter without forking it, which is the search-order case; - you are **migrating** between warehouses and want models to compile against both during the transition. One related fact worth knowing: since dbt Core 1.2 a set of cross-database macros — `datediff`, `dateadd`, `safe_cast`, `hash`, `split_part`, `listagg` and others — ships in dbt Core itself and is called *without* a namespace. They were previously in dbt_utils, and old projects still carry `dbt_utils.datediff` calls that now shadow or duplicate the built-in. When you meet dispatch in an interview, mentioning that migration shows you have actually maintained a project across versions rather than read the docs page.

  • What is the most common mistake when writing a dispatched entry-point macro?
    Forgetting the second set of parentheses. `adapter.dispatch('name', macro_namespace='pkg')` returns the macro itself; you must call it with the real arguments — `...(arg1, arg2)` — and usually wrap the whole thing in `return()`. Omitting the call renders a macro object into your SQL instead of the dialect-specific text.
  • How do you make your own implementation win over a package's without forking the package?
    Add a `dispatch` block to `dbt_project.yml` giving that package's `macro_namespace` a `search_order` that lists your project first, then define `<adapter>__<macro_name>` in your own macros directory. Every dispatched call in that namespace now finds yours first, including calls made from inside the package's other macros.
  • When is dispatch overkill for an internal macro?
    When the project only ever runs on one warehouse. A plain macro is shorter, easier to read and easier to grep. Dispatch earns its ceremony when you publish a package, share a macro library across projects on different adapters, run a different engine locally than in production, or are mid-migration between warehouses.

saying these in an interview costs you the question

  • Uses a single underscore in the adapter prefix
  • Returns the dispatch result without calling it with arguments
  • Thinks dispatch inspects the SQL to guess the dialect
  • Believes overriding requires forking the package
  • Adds an if-target.type ladder instead and calls it equivalent

context