An identical list endpoint is fast in one environment and slow in another — how do you prove a per-row emitter is why?

"Works here, not there, same commit" is the most common way a query-per-row emitter is first reported, and it is a consequence of the defect's defining property: **the statement count is a function of the rows walked, not of the code executed**. Two deployments running byte-identical code will diverge as soon as their data differs in size or shape. ## Why data volume, not the code path, sets the cost The walk visits whatever the read returned. A collection with three items is touched three times; the same field, in the same line, is touched three thousand times against a larger tenant. Nothing in the code changed and nothing needs to. Three practical consequences follow: - A small data set can never expose the defect, however carefully the code path is exercised. - The same endpoint can be healthy for most users and fatal for the largest one, because collection sizes are almost always skewed. - The cost grows with adoption over time, so an endpoint that has been fine for a year degrades without a deploy. ## Proving it by correlation The evidence you want is two numbers per request, captured in both environments: **the number of statements issued** and **the number of rows the response actually walked** (items in the list, plus items in any nested collection rendered). Then look at the relationship. | Observation | Reading | |---|---| | Count rises roughly one-for-one with rows; each statement fast | per-row emitter, effectively confirmed | | Count flat; one statement much slower in one environment | plan, index or statistics difference — not this defect | | Count flat; all statements slightly slower | resource or network difference between hosts | | Count rises faster than rows | nested walk, with more than one term multiplying | The second row of that table is the alternative most often confused with this one, and the two are cleanly separable: a bad plan makes one statement expensive while the count is unchanged, whereas an emitter leaves every statement cheap and multiplies how many there are. Total time can look identical in both cases, which is precisely why a total-time comparison proves nothing on its own. ## Then prove it causally Correlation across two environments still confounds data with everything else that differs between them — configuration, hardware, warmth of caches. Close that gap by making the data the only variable: 1. In the fast environment, construct a data set whose collection sizes match the slow one, for a single representative parent. Re-run the same request. 2. Watch the statement count follow the data. If the count moves with collection size while the deployment is held fixed, the emitter is established. 3. Bisect the output shape to name the field: drop one linked field from what is rendered, re-run, and see which removal makes the burst disappear. Step three is what turns a diagnosis into an actionable line number, and it is the step teams skip — leaving them with a proven pathology and no idea which walk produced it. ## Reading the request-level shape Two further tells help before you have instrumentation in place. A per-row emitter's latency scales with the *result size of that specific request*, so time per request plotted against items returned makes a straight line rather than a cloud; a per-statement problem makes a flat band. And in the log, the extras arrive as a run of near-identical statements differing in one bound value — a shape nothing else produces. One caution: some layers, on touching a link, opportunistically fill the same link for other loaded objects in one grouped statement. Where that happens the count stops rising one-for-one and rises in steps instead, which can mask the correlation at small sizes while the underlying walk is still there. Read the statement text, not only the count: grouped fills carry a list of bound values where a one-at-a-time fill carries a single one. ## Reporting it credibly The finding that persuades is a short chain: here is the statement count against rows in each environment; here is a single statement's timing showing the store is not the problem; here is the same code with the same data shape reproducing it in the fast environment; here is the field whose removal deletes the burst. Each link rules out an alternative, and together they name a line. "It is an N+1" without those links is a guess that happens to be right often enough to be dangerous — it sends people optimising the store, or the environment, when the emitting walk is what needs to change.