skip to content

A report query that ran in under a second for months suddenly takes several minutes, with no change to the query text, the schema, or the indexes. How would you determine whether inaccurate optimizer statistics are the cause, and what would you do about it?

level: seniorimportance: must knowfreq 56%

answer

  1. estimated vs actual, read bottom-up
  2. lowest divergent node = root cause
  3. last-analyzed timestamp + churn since
  4. growing max: predicate past recorded max → est. 1 row
  5. refresh → re-plan → deeper histogram / multi-column stats

basics

~20 s

Get an execution plan with both estimated and actual row counts, then walk it bottom-up looking for the first step where estimate and actual diverge by orders of magnitude. If the leaf estimates are wrong, refresh statistics on those tables, re-check the plan, and if it still misestimates, add deeper histograms or multi-column statistics.

solid answer

~60 s

First, capture a plan that shows **estimated versus actual** rows per step, not just an estimate. Then read it bottom-up and find the *lowest* node where the two diverge badly — an estimate of 1 row against 400,000 actual, for example. Everything above that point is a consequence, not the cause: a leaf underestimate makes the optimizer pick a nested-loop join or a small hash table, which then explodes. Establish why the estimate is wrong: - **Stale statistics** — check the last-analyzed timestamp and how much the table has changed since. Bulk loads, mass deletes, and steady growth all invalidate them; on time-series columns, predicates past the stale maximum estimate as ~1 row, which is the classic overnight regression. - **Missing detail** — skew with no histogram, or correlated predicates estimated as independent. The remedy is to refresh statistics on the affected tables, then re-plan and compare. If it is still wrong, raise the histogram resolution on the offending column, add multi-column/extended statistics, or gather statistics on the expression being filtered. Finally, fix the reason they went stale: auto-analyze thresholds that never fire on a huge table, or a bulk pipeline that doesn't analyze afterwards.

code

text · 6 lines
text
Nested Loop  (est. rows=2  actual rows=418,902  time=214,003 ms)
  ->  Index Scan on events e  (est. rows=1  actual rows=418,902)   <-- root cause
        Filter: created_at > now() - interval '1 hour'
  ->  Index Scan on users u  (est. rows=1  actual rows=1, loops=418,902)

Estimate of 1 row made a nested loop look free; 418,902 loops made it not free.

go deeper

for a junior

Say you would look at the execution plan, compare estimated to actual rows, and refresh the table statistics if they look stale.

for a middle

Add the bottom-up reading discipline, name concrete staleness causes such as a bulk load or the growing-maximum problem on timestamps, and describe re-checking the plan after refreshing.

for a senior

Own the full loop: reproduce, measure, isolate the lowest divergent node, distinguish stale statistics from insufficient detail from an atypical parameter, fix the estimate, then fix why it went stale.

for a principal

Frame it as reliability of the estimation pipeline: analyze thresholds for very large tables, statistics as a step in every bulk pipeline, monitoring for plan regressions, and an explicit policy on when pinned plans are acceptable debt.

## Why "nothing changed" is never true A plan is chosen from statistics, and statistics track data. If the query text, schema, and indexes are unchanged, then either the data changed, the description of the data changed, or the parameters changed. A sudden regression almost always means the optimizer crossed a cost boundary and switched plan shape — typically from a hash/merge join to a nested loop, or from an index scan to a scan, or the reverse. ## Step 1: get estimated versus actual An estimate-only plan tells you what the optimizer believed, not whether it was right. Every mainstream engine has a mode that executes the statement and reports actual row counts and timings per node. That comparison is the entire diagnostic. Read the plan **bottom-up**. Estimation errors propagate and multiply: if a scan at the bottom is estimated at 1 row when it returns 400,000, then the join above it is estimated at 1 × something, and the join above that inherits the fiction. Only the lowest divergent node is a genuine finding; the rest is fallout. A useful heuristic: an estimate within a factor of 2–3 is fine, a factor of 10 is suspicious, a factor of 1000 is your bug. ## Step 2: classify the estimation error **Stale statistics.** Check the metadata for when the table was last analyzed and how many rows have been inserted, updated, or deleted since. Common causes: - Automatic analysis is threshold-based on a *fraction* of the table. On a table with 500 million rows, a 10% threshold means 50 million changes before it fires — which can be months, or never. - A bulk load, restore, or partition swap wrote a large volume without a following analyze. - A mass delete left the row-count estimate wildly high. - **Growing maximum**: the table is append-only on `created_at`, statistics recorded a maximum from last month, and the query asks for the last hour. Range estimation interpolates inside [min, max], so anything past the recorded max estimates as about one row. This single mechanism causes a large share of "it was fast yesterday" incidents on time-series and event tables. **Fresh statistics but insufficient detail.** - Skew that the histogram resolution cannot represent — too few buckets on a column with a heavy distribution. - Correlated columns estimated as independent, so a multi-predicate filter is underestimated by the product of the individual selectivities. - A predicate wrapped in a function or a type cast, which makes the column statistics inapplicable and drops the engine onto a fixed default guess. **Not a statistics problem at all.** Before committing to the statistics story, rule out: a parameter value that is genuinely atypical (the plan is fine for most values and terrible for this one), data volume that has simply grown past what the old plan could handle, resource contention or a cold cache, or a change in the amount of memory available for sorts and hashes that flipped an in-memory operation to a spilling one. ## Step 3: act 1. **Refresh statistics** on the tables at the bottom of the plan. This is cheap and non-destructive; it samples the table and rewrites the summary. Then re-capture the plan and compare estimates to the same actuals. If the estimates snap into place and the runtime recovers, you have your answer. 2. **If it is still wrong, add detail.** Increase histogram resolution on the specific column driving the misestimate; add multi-column or extended statistics for correlated predicates; gather statistics on the expression if the filter is an expression. 3. **Fix the recurrence, not just the incident.** Add an explicit analyze step at the end of the bulk-load pipeline. Lower the automatic-analysis threshold for the specific huge table so it triggers on an absolute row count rather than an unreachable percentage. For append-only time-series tables, schedule frequent refreshes of the leading timestamp column so the recorded maximum stays close to reality. 4. **Have a containment option.** If the fix cannot land immediately and the query is business-critical, the short-term levers are rewriting the query so the estimate matters less (for example splitting a correlated filter, or materializing an intermediate result), or — where the engine supports it — pinning or hinting the plan. Treat a pinned plan as debt with an expiry date: it freezes a decision made against today's data. ## How to narrate this in an interview The structure interviewers want is: reproduce and measure → get estimated versus actual → find the lowest divergent node → explain *why* the estimate is wrong → fix the estimate rather than the symptom → prevent recurrence. Jumping straight to "I'd add an index" or "I'd hint the plan" without establishing the row-count error is the answer that fails.

  • You refreshed statistics and the estimate is still off by three orders of magnitude. What next?
    Look at what per-column statistics structurally cannot capture. Multi-column correlation is the usual culprit — two filters that are logically dependent get their selectivities multiplied — so add extended or multi-column statistics. Also check for a function or cast around the column, which makes the stored statistics inapplicable, and consider raising the histogram bucket count if the column is heavily skewed.
  • How would you tell an atypical parameter value apart from stale statistics?
    Run the same statement with a typical value and with the problem value and compare plans and estimates. If the plan is good for common values and only degrades for one outlier, the statistics are describing the data correctly and the issue is that a single plan has to serve very different selectivities — a different class of fix, such as splitting the query path or forcing per-value planning.
  • Why is pinning the plan a poor first response?
    It freezes a decision made against a snapshot of the data while the data keeps changing, so it converts a self-correcting system into a manual one. It also hides the real defect — the wrong row estimate — which is probably degrading other queries against the same table. It is a legitimate containment measure under incident pressure, but it needs an owner and a removal date.

saying these in an interview costs you the question

  • Reading an estimate-only plan and never comparing against actual row counts.
  • Attacking the topmost expensive operator instead of the lowest node where estimate and actual diverge.
  • Adding an index as the reflexive fix without establishing why the plan changed.
  • Assuming automatic statistics collection guarantees fresh statistics on very large tables.
  • Jumping to a plan hint or pinned plan as the primary solution rather than as temporary containment.

context