How would you watch statement counts in production without logging every statement the service executes?

Statement logging is a development instrument. Turned on for every request in production it writes one line per statement, at the exact moment the service is issuing far too many of them, on the machine that is already struggling — and the lines carry parameter values, which is a data-handling problem on top of a volume problem. The production version of the same idea keeps the **number** and throws away the text, then buys the text back only for the requests that deserve it. ## Record the count as a measurement, not as text The per-request counter described for tests works unchanged in production; what changes is where the number goes. Two destinations, and they answer different questions: - **On the request's own record.** Whatever line or span the service already emits per request gets one more field: the statement count, next to the route or operation name and the duration. Cost is one integer on a record that was being written anyway, and it makes "why was this request slow" answerable after the fact. - **As an aggregate.** The same number fed into a distribution keyed by operation gives percentiles per route over time — the series you can actually alert on. Both are cheap because the expensive part of statement logging was never the counting, it was the formatting and the volume of the text. ## Keep the key space small A distribution keyed by operation is useful; a distribution keyed by anything that varies per request is a liability. Route templates, job names and message types are safe keys. Identifiers, parameter values, user names, raw paths with values interpolated into them are not: each distinct value creates its own series, the series count grows without bound, and the cost lands in the service's memory, in the export step and in the store at the other end. The rule is the same one that applies to any dimensional measurement, and it bites harder here because a per-row read is precisely the situation where identifiers are abundant. ## Alert on the shape, not on an absolute number An absolute threshold across a service is either so high it never fires or so low it fires constantly, because a legitimate count varies by an order of magnitude between a health check and a reporting screen. What is meaningful: | Signal | What it catches | Why it is stable | |---|---|---| | Per-route percentile compared with the same route last week | a change in the access pattern | the route is its own baseline | | A step change aligned with a release | a regression that shipped | correlates cause and effect immediately | | Count divided by rows returned, trending up | multiplication rather than more traffic | normalises away load and page size | | Count percentile far above the route's median | one input class taking a bad path | catches the tail an average hides | Total statements per second is a load metric and belongs on the capacity dashboard, not here: it rises with traffic and says nothing about whether a code path multiplied. ## Buy the text back by sampling The count tells you a route regressed; it never tells you which read multiplied. So arrange for the evidence to be captured only when it is worth capturing: 1. **Trigger on the count.** When a request's count crosses a threshold for its route, emit one record for that request containing the statements grouped by text with parameters stripped, and their repetition counts. 2. **Cap it.** A rate limit per route per interval, so a route that regresses for every request produces a handful of samples rather than a flood, and so the diagnostic path cannot become the outage. 3. **Redact.** Group by statement text with values removed; the shape and the repetition count are what diagnose the problem, and the values are what create the data-handling risk. 4. **Keep a slower unconditional trickle.** A small random sample, independent of the trigger, is what lets you see the baseline shape of a route that has never crossed its threshold. This gives the property that matters operationally: constant, negligible cost in the healthy case, full evidence in the pathological one. ## Know what this arrangement still cannot do - It shows a route regressed, not which line multiplied — the sampled statement shapes narrow that, a captured call path narrows it further, and neither is guaranteed to be present for the request you care about. - Counts alone will not distinguish "more statements" from "the same statements, now slower", so the count belongs beside the duration, never instead of it. - A route whose statements are grouped into fewer sends can regress in latency while its execution count sits still, and vice versa. - Sampling by definition misses the request the user complained about, unless the trigger happened to fire. ## Why interviewers like this one It separates people who have only ever debugged this locally from people who have had to find it in a running system. The local answer — turn on statement logging — is the wrong answer in production for reasons of cost, volume and data handling, and the good answer is a two-tier design: a cheap number always, expensive text rarely and deliberately.