skip to content

How do you decide which warehouse metrics may use approximate distinct counts and which may not?

level: principalimportance: should knowfreq 38%

answer

  1. ask what breaks if the number is one percent wrong
  2. money and regulators do not accept estimates
  3. a sketch says how many, never which
  4. declare the choice where the metric is defined
  5. exactness is cheap when cardinality is small

basics

~20 s

Decide by what consumes the number, not by what it costs. Money, regulatory reporting, anything reconciled against a system of record, and threshold decisions inside the error band require exactness; indicators, trends and exploration do not. Declare the choice in the metric layer, not per query.

solid answer

~50 s

The test is the consequence of being a percent wrong. A number that is invoiced, filed with a regulator, reconciled against an upstream system, or used to decide a contractual threshold must be exact — the error is not a rounding difference, it is a discrepancy someone will litigate. A number that drives a trend line, a funnel, a monitoring panel or an exploratory cut is fine approximate, because no decision changes on a percent. Two second-order rules matter as much as the first. Publish the choice: mark each metric in the semantic layer as exact or approximate with its error bound, so consumers do not compare an approximate figure to an exact one and open an incident. And never let a metric switch implementation quietly — the step change looks exactly like a data bug. Finally, measure before you approximate: distinct counts on low-cardinality columns are cheap exactly, and paying for a sketch there buys nothing.

go deeper

for a junior

Know the bright line: numbers that are billed, filed or audited must be exact, and everything exploratory or trend-shaped can be approximate. Do not swap a function in a query without asking who reads the result.

for a middle

Explain the mechanics behind the rule — a relative error bound, the loss of drill-through, the impossibility of reconciling an estimate — and name concrete metric classes on each side of the line.

for a senior

Demonstrate operational judgment: a certified exact job alongside the fast approximate one, monitoring the gap, and warning consumers about consistency oddities like a subset estimating larger than its parent.

for a principal

Own it as governance. Classify metrics at the semantic layer, publish error bounds to consumers, version any change of implementation, and set the standard that no metric silently changes how it is computed.

## The decision belongs to the consumer, not the query Engineers tend to make this call at the point of pain: a dashboard query is slow, so someone swaps `COUNT(DISTINCT)` for the approximate function and the dashboard gets fast. That is the wrong place to decide, because the person editing the query rarely knows what the number is eventually used for. Six months later the same figure is feeding an invoice. The framing that survives contact with a real organisation is: **classify the metric by consequence, then implement accordingly.** ## Where exactness is non-negotiable **Money.** Anything that determines what a customer is charged or what a partner is paid — billed seats, billed unique devices, unique events under a usage plan, revenue-share counts. A 0.8% error on a large invoice is a real sum of money and an indefensible one, because there is no story that makes "we estimated your bill" acceptable. **Regulatory and audit reporting.** Filed figures must be reproducible on demand and defensible line by line. An estimate whose value depends on how the cluster happened to parallelise the query is neither. **Anything reconciled against a source of truth.** If the warehouse figure is compared with an operational system's figure, an approximation guarantees a permanent unexplained delta and burns analyst time forever. **Hard thresholds.** A contractual minimum, an SLA breach, a compliance limit — if the decision boundary sits inside the error band, the estimate can flip the decision. Either compute exactly or restructure the rule so the error cannot change the outcome. **Row-level or drill-through requirements.** A sketch answers how many, never which. If the consumer needs the list — for outreach, for investigation, for a cohort — the sketch cannot serve them and an exact path must exist. **Uniqueness and deduplication checks.** A data-quality test asserting that a key is unique is meaningless against an estimate. ## Where approximation is clearly correct Exploratory analysis, data profiling, trend charts, funnels, monitoring panels, capacity signals, A/B dashboards where the effect being measured is far larger than the error, and any cut where the analyst is looking at shape rather than magnitude. In these cases the alternative is not a more accurate number, it is a slower dashboard or no dashboard, and the accuracy nobody uses is not worth the cost. ## Make the choice visible in the metric layer The durable implementation is a property of the metric definition, not of any one query. Each metric in the semantic or metric layer carries a flag: exact or approximate, and if approximate, the error characteristics. Downstream tools show that alongside the number. This solves the two failure modes that actually bite organisations. The first is **cross-tool disagreement**: an approximate figure in one dashboard next to an exact one in another, differing by a percent, generating a support ticket every quarter. The second is **silent implementation drift**: a metric computed exactly for a year, switched to approximate during a cost-reduction push, showing a step change that a downstream team investigates as a pipeline incident. Both are process failures, not technical ones, and both are prevented by declaring and versioning the choice. ## A hybrid that works A pattern worth knowing: serve the fast approximate metric to interactive consumers, and run an exact **certification** job on a slower cadence — nightly or monthly — over the same definition. The certified number is what gets reconciled, filed and billed; the approximate one is what the dashboard shows intraday. Publish both under distinct names so nobody confuses them, and track the observed gap between them over time; a widening gap is a genuine signal that the sketch's precision no longer suits the data's cardinality. ## Consistency traps to warn people about Estimates carry independent error, so they do not respect the relationships people expect of exact numbers. A filtered subset, estimated separately, can come back larger than its parent. Segments that must add to a whole will not add exactly. A month-over-month change smaller than the error band is noise, not a movement. None of these are bugs, but every one of them looks like one to a consumer who was not told the number was approximate — which is why the declaration matters more than the choice. ## Measure before you approximate Finally, do not pay for approximation you do not need. Exact distinct counting is expensive only when the counted column's **cardinality** is large enough to make the deduplication state heavy and the redistribution wide. A distinct count over a few thousand product categories is cheap exactly, regardless of how many billion rows are scanned. Profile the actual cost of the exact version first; if it is acceptable, keep the guarantee. Reserve approximation for the cases where exactness genuinely costs something a business would notice.

  • A team wants the same metric approximate on the dashboard and exact on the invoice. Is that acceptable?
    Only if they are published as two clearly distinct metrics with different names, and consumers are told which is which. The same name resolving to two numbers guarantees a recurring dispute. A defensible version is a fast approximate indicator plus a slower certified figure, with the gap between them monitored as a health signal.
  • How would you handle a metric that has been exact for a year and now needs to be approximate for cost reasons?
    Treat it as a versioned change, not a query edit. Backfill the approximate series alongside the exact one so the step change is visible and explained, announce the switch with its error bound, and keep the exact definition available for reconciliation. A silent swap produces a step in the chart that downstream teams will investigate as an incident.
  • What early warning tells you a sketch's precision no longer fits the data?
    A widening gap between the approximate metric and a periodically computed exact one, particularly if it trends rather than oscillates. Since HyperLogLog's error is relative, growing cardinality does not raise the percentage error but does raise the absolute miss — so a figure that was comfortably within tolerance at ten million distinct users may not be at a billion.
  • Does the error bound let you state a confidence interval next to the number?
    Roughly. A HyperLogLog-style sketch has a known relative standard error set by its precision, so you can present a band rather than a point estimate. It is a distributional guarantee about typical behaviour, not a hard maximum on any single result, so present it as a tolerance rather than as a guaranteed range.

saying these in an interview costs you the question

  • Says approximation is always fine because the error is small
  • Chooses exactness per query rather than per metric definition
  • Switches a published metric to approximate without announcing it
  • Uses an estimate for a threshold that sits inside the error band
  • Assumes exact distinct counting is always expensive without measuring

context