skip to content

In ClickHouse, how does chaining materialized views turn one insert into writes across several tables?

level: seniorimportance: should knowfreq 45%

answer

  1. a view's write is itself an insert
  2. who else is watching that target table
  3. it all happens inside the client's INSERT
  4. one block, one new part per table
  5. sequential unless a setting says otherwise

basics

~20 s

A materialized view fires on any insert into its source table, including writes made by another view. So a view on a rollup table cascades: one client INSERT synchronously executes every view in the chain and creates a new part in each target table, multiplying insert latency and part count.

solid answer

~50 s

Chaining works because a materialized view triggers on *inserts into its source*, and a view writing into table B counts as an insert into B. That lets you build a ladder — raw events → per-minute rollup → per-hour rollup — where each level is a view on the level below. The whole chain runs synchronously inside the original `INSERT`, on the inserting thread. One client insert therefore writes a part into the raw table and into every downstream target. Views attached to the same source run one after another by default; `parallel_view_processing` changes that. Two operational consequences dominate. First, part creation multiplies, so small frequent inserts become part explosion across N tables — batch, or use async inserts. Second, if any view in the chain throws, the `INSERT` fails, but views that already ran have written their rows, leaving partial state. `system.query_views_log` shows per-view duration and errors.

code

sql · 13 lines
sql
-- level 1: raw -> minute
CREATE MATERIALIZED VIEW mv_minute TO minute_rollup AS
SELECT toStartOfMinute(ts) AS minute, country,
       uniqState(user_id) AS uniq_users,
       count()            AS events
FROM events GROUP BY minute, country;

-- level 2: minute -> hour, re-aggregating the states
CREATE MATERIALIZED VIEW mv_hour TO hour_rollup AS
SELECT toStartOfHour(minute) AS hour, country,
       uniqMergeState(uniq_users) AS uniq_users,
       sum(events)                AS events
FROM minute_rollup GROUP BY hour, country;

go deeper

for a junior

Know that a view attached to a table fires when anything inserts into it, including another materialized view, so views can be stacked into a chain.

for a middle

Explain that the cascade runs synchronously inside the client INSERT, creating a part per table per block, and that views on one source run sequentially by default.

for a senior

Show the operational picture: part explosion multiplied by the number of views, insert latency you can attribute via system.query_views_log, and partial state after a mid-chain failure.

for a principal

Decide how much of the read path belongs on the write path at all — chain depth, fan-out width, which derived tables may silently skip errors, and who owns replaying a level after a rebuild.

## How the cascade happens A ClickHouse materialized view is attached to a table and fires on inserts into it. Nothing in that rule says the insert must come from a client. When view `mv1` writes its output block into table `minute_rollup`, that is an insert into `minute_rollup`, and any view attached to `minute_rollup` fires in turn. That is the whole mechanism behind multi-level rollups: ``` events ──mv_minute──▶ minute_rollup ──mv_hour──▶ hour_rollup ``` When the second level aggregates data that is already stored as aggregate states, use the composed combinator: `uniqMergeState(uniq_users)` reads the incoming states, merges them, and emits a coarser state for the hourly table. Additive columns are simply summed again. ```sql CREATE MATERIALIZED VIEW mv_hour TO hour_rollup AS SELECT toStartOfHour(minute) AS hour, country, uniqMergeState(uniq_users) AS uniq_users, sum(events) AS events FROM minute_rollup GROUP BY hour, country; ``` ## Everything is synchronous and on the insert path The cascade is not a background job. It executes inside the original `INSERT` statement, on the thread doing the insert, before the client gets an acknowledgement. So a single insert of a block into `events`: 1. writes a part into `events`; 2. runs `mv_minute`'s `SELECT` over that block and writes a part into `minute_rollup`; 3. runs `mv_hour`'s `SELECT` over *that* block and writes a part into `hour_rollup`. Three tables, three new parts, one client insert. Add a fan-out of six views on the raw table and one insert becomes seven parts plus whatever their own children produce. ## The two costs that actually bite **Part explosion.** MergeTree's health depends on parts being large and few. Small frequent inserts are already the classic ClickHouse anti-pattern; with N attached views the damage is multiplied by N, and each target has its own merge backlog. The mitigations are the usual ones, applied harder: batch inserts into large blocks, or enable async inserts so the server assembles batches for you, and keep an eye on the parts count per target table. **Insert latency.** Every view's `SELECT` is real work on the insert path — a `GROUP BY`, possibly a dictionary lookup or a join that re-reads a table per block. Views attached to the same source are executed one after another by default; the setting `parallel_view_processing` runs them concurrently instead, trading CPU and memory for wall-clock latency. Measure before flipping it: parallel execution multiplies peak memory by the number of views. ## Failure semantics If a view's `SELECT` throws — a type overflow, a memory limit, a missing dictionary — the exception propagates and the client's `INSERT` fails. But views that already ran in this cascade have already written their rows. The result is a partially applied insert: the source may hold the block while a downstream rollup does not, or vice versa. There is no transaction wrapping the chain. The setting `materialized_views_ignore_errors` makes such failures logged and skipped instead of failing the insert. That keeps ingest flowing at the price of silently losing view output, so it is a deliberate choice for non-critical derived tables, never a default for the rollup a dashboard trusts. For observability, `system.query_views_log` records one row per view execution per query — the view name, rows read and written, duration, memory and any exception — which is the fastest way to find *which* view in a chain is costing your insert latency. It is populated when view logging is enabled for the query. ## Design guidance - Prefer a shallow ladder (raw → fine rollup → coarse rollup) over a wide fan-out of independent views over the raw table: each level reads far less data than the one below it, so total insert-path work drops. - Keep every view's `SELECT` cheap. Enrichment belongs in a `Dictionary` lookup, not a join. - Give every level an explicit `TO` table so you can backfill, `ALTER` or rebuild a single level without touching the rest of the chain. - Remember that a rebuild of level N must be replayed into level N+1 yourself; re-inserting into the intermediate table will cascade, so plan for the double counting that follows and use an append-safe design or a truncate-and-replay window. - Watch the depth. Every level adds latency to a synchronous path, and the deeper the chain the harder the partial-failure story becomes. ## The interview answer *"Views trigger on inserts into their source, and a view's own write is an insert — so chains cascade. All of it runs synchronously inside the client's INSERT, creating one part per table per block, sequentially unless `parallel_view_processing` is on. The costs are part explosion and insert latency, and a mid-chain failure leaves partial state because there is no transaction around it."*

  • How do you aggregate an hourly rollup from a minute table that stores AggregateFunction states?
    Use the composed `-MergeState` combinator in the second view: `uniqMergeState(uniq_users)` merges the incoming states and emits a coarser state for the hourly AggregatingMergeTree, while additive columns are summed again. Do not finalize with `-Merge` and then re-wrap with `-State`; the composed combinator exists precisely so states can be re-aggregated without a round trip through final values.
  • One of five views attached to an ingest table throws on a bad row. What is the state of the system?
    The client's INSERT fails, but views executed earlier in the sequence have already written their output, so the derived tables are partially updated with no transaction to roll back. A retry may then be deduplicated at the source and never reach the failed view. Treat views as fallible parts of the insert path: keep their SELECTs total, validate on ingest, and decide explicitly whether `materialized_views_ignore_errors` suits each target.
  • How would you find out which view in a chain is making inserts slow?
    Query `system.query_views_log`, which records one row per view execution per query with the view name, rows read and written, duration, peak memory and any exception. Correlate by `initial_query_id` with `system.query_log`. That immediately separates a slow dictionary lookup or an expensive join inside one view from generic ingest pressure such as merge backlog.

saying these in an interview costs you the question

  • Assuming materialized views run asynchronously after the insert
  • Believing the whole cascade is transactional and rolls back
  • Fanning out ten views over the raw table with tiny inserts
  • Enabling parallel_view_processing without checking memory use
  • Rebuilding one level and expecting downstream levels to follow

context