skip to content

A Snowflake table is clustered by event_date, yet queries still scan every micro-partition. Why?

level: seniorimportance: should knowfreq 58%

answer

  1. check what the query filters on first
  2. the stored ranges describe raw column values
  3. a function on the column breaks the comparison
  4. key column order changes what prunes
  5. someone may have suspended reclustering

basics

~20 s

Usually the predicate cannot be matched to the stored min/max metadata: it filters a different column, wraps the clustering column in a function or cast, or constrains only a trailing key column. Otherwise the clustering itself has decayed or the key is too granular to separate anything.

solid answer

~50 s

Work from the query backwards. First, **is the predicate on the key at all?** A table clustered by `event_date` prunes nothing for `WHERE customer_id = ...`. Second, **is the column bare?** `WHERE to_date(event_ts) = '2026-08-01'` compares a computed value, not the values whose min/max were recorded, so the ranges usually cannot be used — unless the clustering key is that same expression. Casts, `LIKE '%x%'` and wide `OR` chains have the same effect. Third, **is the leading column constrained?** With `CLUSTER BY (region, event_date)`, filtering only on `event_date` prunes far worse, because files are separated primarily by `region`. Only then look at the layout: run `SYSTEM$CLUSTERING_INFORMATION` — if depth is high relative to the partition count, either reclustering is suspended, the table churns faster than it converges, or the key is too high-cardinality to separate anything. Confirm every step against partitions scanned in Query Profile.

code

sql · 7 lines
sql
-- typically prunes nothing: the clustering column is wrapped
select count(*) from events
where to_date(event_ts) = '2026-08-01';

-- prunes: half-open range on the bare clustering column
select count(*) from events
where event_ts >= '2026-08-01' and event_ts < '2026-08-02';

go deeper

for a junior

Remember the two everyday causes: the query filters a column the table is not clustered by, or it wraps the clustering column in a function so the stored value ranges cannot be compared.

for a middle

Explain why a function or cast defeats the min/max comparison, why only the leading key column separates files strongly, and why a near-unique key cannot separate anything at all.

for a senior

Show a diagnostic order: confirm partitions scanned versus total, audit the predicate, then inspect clustering depth, whether recluster is suspended, and what reclustering has been costing. Fix the query before spending credits.

for a principal

Recognize when clustering is simply the wrong tool — a table churned faster than it can converge, or queried on an axis the key does not serve — and decide instead to change the write pattern, split the table, or accept the scan.

## Establish the fact first Before theorising, open **Query Profile** and read the TableScan operator's *Partitions scanned* against *Partitions total*, or pull `partitions_scanned` and `partitions_total` from `SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY` for that `query_id`. "Feels slow" is not a pruning diagnosis. If scanned equals total, pruning genuinely failed and the checklist below applies. If scanned is already tiny, the time is going somewhere else — a spilling join, an under-sized warehouse, an exploded intermediate result — and clustering is the wrong lever entirely. ## Cause 1: the predicate is not on the clustering key The most common answer and the most embarrassing. A table clustered by `event_date` gives you nothing for `WHERE customer_id = 918273`, because `customer_id` values are spread across every micro-partition and their recorded ranges all overlap the predicate. Clustering helps exactly the columns it names (and columns strongly correlated with them). Check what the important queries actually filter on before blaming the service. ## Cause 2: the column is wrapped ```sql -- typically prunes poorly: compares a computed value select count(*) from events where to_date(event_ts) = '2026-08-01'; -- prunes: compares directly against recorded min/max select count(*) from events where event_ts >= '2026-08-01' and event_ts < '2026-08-02'; ``` The micro-partition metadata holds min and max of the **stored** column values. A function or cast applied to the column produces something those ranges do not describe, so the optimizer generally cannot exclude files. The fix is either to rewrite the predicate as a range on the bare column — a *sargable* predicate, in the classic terminology — or, if every query genuinely filters by day, to make the clustering key *be* that expression (`CLUSTER BY (to_date(event_ts))`) so the recorded organization matches how the data is asked for. The same family of problems: `WHERE cast(order_id as varchar) = '123'`, `WHERE substr(code, 1, 3) = 'ABC'`, `WHERE col LIKE '%mid%'` (a leading wildcard has no usable range), and long `OR` chains across unrelated columns, where the union of ranges covers everything. ## Cause 3: only a trailing key column is constrained With `CLUSTER BY (region, event_date)`, Snowflake separates micro-partitions primarily by `region`, and by `event_date` only within a region. A query filtering only on `event_date` must consider files from every region, so it prunes far less than the same filter would on a table clustered by date first. Order the key columns by how your queries filter — the column present in the most selective, most common predicate goes first. ## Cause 4: the key is too granular A clustering key on a raw microsecond timestamp, a UUID, or a concatenation with near-unique values gives Snowflake an ordering it can never economically maintain: nearly every value is distinct, so almost every micro-partition ends up overlapping its neighbours anyway. Lower the cardinality with an expression — `to_date(ts)` instead of `ts`, a truncated prefix instead of a full identifier — and keep the key to a few columns. ## Cause 5: the organization has decayed or was never maintained Now the layout questions. Run: ```sql select system$clustering_information('analytics.public.events'); ``` If `average_depth` is large relative to `total_partition_count`, the files genuinely overlap. Ask: - **Is reclustering suspended?** Someone ran `ALTER TABLE t SUSPEND RECLUSTER` around a backfill months ago and never resumed. This is a real and frequent finding. - **Was the key added recently?** Convergence is incremental; a key declared yesterday on a 40 TB table has not finished. - **Does the table churn faster than the service converges?** Check `SNOWFLAKE.ACCOUNT_USAGE.AUTOMATIC_CLUSTERING_HISTORY`. Heavy credits *plus* persistently poor depth means DML is outrunning clustering, and the answer is to change the write pattern (batch it, append instead of update) or to drop the key. - **Do many small trickle loads keep arriving?** They produce files that straddle the key ranges until reclustering catches up. ## Cause 6: the table is too small to matter A table of forty micro-partitions cannot prune impressively, and it does not need to. Clustering is a large-table tool; on a small table the whole scan is cheap and a key just burns credits. ## The order to work in Measure partitions scanned → check the predicate names the key → check the column is bare → check the leading column is constrained → check key cardinality → only then examine depth, suspension and reclustering spend. Most incidents end at step two or three, and every one of those is fixed by editing the query rather than by paying Snowflake more.

  • How do you prove pruning failed rather than guessing from query duration?
    Read the TableScan operator in Query Profile: *Partitions scanned* against *Partitions total*. Equal numbers mean no pruning; a tiny ratio means pruning worked and the time is going elsewhere — spilling, a bad join order, an undersized warehouse. The same figures are in `SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY` as `partitions_scanned` and `partitions_total` for after-the-fact analysis.
  • With CLUSTER BY (region, event_date), why does filtering only on event_date prune badly?
    Because micro-partitions are separated primarily by `region`, and by `event_date` only within a region. Every region contributes files covering that date, so far fewer can be excluded than if the key led with the date. Order key columns by how your dominant queries filter, not by what reads nicely in the DDL.
  • When is redefining the clustering key as an expression the right fix?
    When every important query filters through that same expression. If dashboards always ask for a day of data from a microsecond-precision timestamp, `CLUSTER BY (to_date(event_ts))` both lowers the key's cardinality — making it cheaper to maintain — and matches how the data is requested. Do not do it for a shape only one query uses.
  • You find average_depth is high and reclustering credits are also high. What does that combination mean?
    The service is working hard and losing. The table is churning faster than clustering can converge — typically full reloads, or scattered updates across the whole history. The fix is on the write side: batch the DML, switch to appends, or split hot recent data from cold history. Paying more for reclustering will not close that gap.

saying these in an interview costs you the question

  • Blames the clustering service before checking the predicate
  • Thinks a clustering key makes any filter fast
  • Wraps the clustering column in a function and expects pruning
  • Never checks whether recluster was left suspended
  • Adds more columns to the key to fix a query that filters elsewhere

context