In a dbt incremental model, when does the is_incremental() macro return true?

## Why incremental models exist A `table` materialization rebuilds the whole model on every `dbt run`. For a fact table with billions of rows, or a source that only ever gains yesterday's events, that is enormous wasted compute. The `incremental` materialization builds the table once and then, on each subsequent run, processes only a slice of rows and writes them into the existing table. That creates a problem the other materializations do not have: the model SQL must behave **differently on the first run than on later runs**. On the first run there is no table, so the model must produce all of history. On later runs it should produce only the new slice. `is_incremental()` is the switch that makes one file do both. ## The three conditions `is_incremental()` returns true when **all** of these hold: 1. The model is configured with `materialized='incremental'`. 2. The target relation already exists in the warehouse. 3. dbt is not running in full-refresh mode (`dbt run --full-refresh`, or `dbt build --full-refresh`). If any is false, the macro returns false and the templated block never reaches the warehouse — the compiled SQL simply has no `where` clause from it. That makes the behaviour predictable: - **First ever run** → false → full build, table created. - **Normal scheduled run** → true → filtered build, results merged/inserted. - **`--full-refresh` run** → false → dbt drops and recreates the table from all history. - **Someone dropped the table manually** → false → dbt silently rebuilds it in full. This is why a mysteriously long run can simply mean the table went missing. ## What the model looks like ```sql {{ config( materialized='incremental', unique_key='event_id' ) }} select * from {{ ref('stg_events') }} {% if is_incremental() %} where ingested_at > (select max(ingested_at) from {{ this }}) {% endif %} ``` `{{ this }}` is the Jinja reference to the model's own relation — the table dbt is about to write into. You can only reference it inside the guarded block, because on a full build it does not exist yet. ## What dbt actually executes On an incremental run dbt does not append your `SELECT` straight into the table. It first materializes the filtered result into a temporary or intermediate relation, then issues a second statement that combines that relation with the target — an `insert` for the append strategy, a `MERGE` when a `unique_key` is set on a warehouse whose default strategy is merge, or a delete followed by an insert for `delete+insert`. Understanding this two-step shape explains a lot of behaviour: your model SQL controls *which rows are candidates*, and the strategy plus `unique_key` control *what happens to candidates that already exist in the target*. ## Why the filter is your responsibility A crucial and commonly missed point: dbt does not track state for you. There is no bookmark, no watermark table, no memory of the last successful run. `is_incremental()` only tells you *whether* to filter — the filter predicate itself is ordinary SQL you write, usually by asking the target table what it already has (`select max(...) from {{ this }}`) or by comparing against a run-time variable. If your predicate is wrong, dbt will happily do the wrong thing on schedule, and nothing will error. Two common predicates and their trade-offs: - `where loaded_at > (select max(loaded_at) from {{ this }})` — based on when the row arrived. Robust against late-arriving business events, because a row loaded today is picked up today regardless of its event date. - `where event_at > (select max(event_at) from {{ this }})` — based on business time. Simple, but it permanently skips any row whose event time is older than the current maximum, which is exactly how late data gets silently lost. ## Operational habits Because the incremental branch is only exercised after a table exists, it is easy to ship a model whose incremental path is broken and never notice in development. Habits that catch it: run the model twice in dev (once to create, once to exercise the filter); inspect `target/compiled/` to read the SQL dbt actually generated for both paths; and schedule a periodic `--full-refresh` so drift between the incremental result and a from-scratch build gets corrected and detected. Also be aware of `full_refresh: false` in a model's config — it prevents `--full-refresh` from affecting that model, which is a deliberate guard on tables so large that a rebuild is unaffordable, and a nasty surprise if you forget it is there. ## When incremental is not worth it Incremental buys build time and warehouse spend at the price of correctness risk: duplicates, missed late rows, schema drift and a build that no longer matches what a full rebuild would produce. If a table rebuilds in a couple of minutes, keep it a `table`. Reach for incremental when the full rebuild genuinely does not fit the run window or the budget.