A stage in an export pipeline fetches ledger rows a page at a time but emits them one by one — how should it convert downstream demand into upstream requests?

## Demand is not conserved across a stage A naive mental model has a request signal travelling upstream unchanged, like a token passed hand to hand. That is true only of the simplest pass-through. Any stage whose input and output counts differ — a stage that batches, filters, expands, or fetches in pages — must **translate** the number, and how it translates is a design decision with real failure modes on both sides. The setting: a stage in a ledger export sits above a paged source and below a report writer. The writer commits one row at a time and therefore asks for one row at a time. The source answers in pages of sixty-four. A request signal per row would mean sixty-four round trips to fetch one page's worth of data, and on a remote source each of those is latency the export pays for nothing. ## The two counters a rewriting stage keeps 1. **Downstream outstanding** — how many values it is currently permitted to emit. Incremented by requests arriving from below, decremented by each value it emits. 2. **Upstream outstanding** — how many values it has asked its own source for and not yet received. Incremented by each request it sends up, decremented by each value that arrives. These are independent numbers, and the stage's job is to run a policy between them. Conflating them into one counter is the single most common way a rewriting stage is written wrong: the stage then either asks upstream for exactly what it was asked for (chatter) or forwards a large upstream request every time something small is asked below (it will be handed more than it can hold). ## Replenishing at a low-water mark The workable policy is: on subscribe, ask upstream for a full buffer's worth. Emit downstream whenever there is both demand below and a value in hand. Count the emissions, and once some fraction of the buffer has drained, ask upstream for exactly that many more. With a buffer of sixty-four and a replenishment threshold of forty-eight, at the moment the stage asks for forty-eight it still holds sixteen — so the most it can ever hold or be owed at once is sixty-four, exactly its capacity. That arithmetic is the whole safety argument, and it should be checkable by hand in any stage you write. The threshold trades two costs against each other: - **Threshold near the full buffer** — one request per large batch, minimal signalling, but the buffer runs closer to empty before the replacement arrives, so a slow source can leave the stage idle. - **Threshold near one** — the buffer stays full, but you are back to a request signal per value, which is the chatter the batching existed to avoid. ## Different stage shapes translate differently | Stage shape | Downstream asks for 10 | What it should ask upstream | |---|---|---| | one in, one out | 10 | 10, or a batch with a low-water mark | | fetches in pages | 10 | a page, buffered, replenished at a mark | | filters | 10 | 10, plus one more for every value it drops | | expands one into many | 10 | 1, and the next only once that expansion is drained | | combines N inputs into one output | 10 | 10 from each input it must pair | The filtering row is the one that breaks pipelines in production. A stage that receives a value and drops it has spent a unit of its upstream demand and produced nothing downstream. If it does not request a replacement, upstream demand ratchets down with every dropped value until it hits zero, and the stream goes quiet with the consumer still waiting. The symptom is a pipeline that works in testing, where few values are filtered, and stalls in production, where most are. The expanding row is the mirror hazard. A stage that turns one input into many outputs must not pull the next input while the current expansion is undelivered, or it accumulates output it has no permission to emit. ## The invariants worth stating out loud - Never emit more values downstream than the downstream count permits. - Never request upstream more than the stage can hold if all of it arrives at once. - Never let a consumed-but-not-emitted value silently reduce the upstream count. - Replenish from what has actually left the stage, not from what has arrived. Hold those four and backpressure survives the stage: a slow report writer at the bottom still eventually slows the paged source at the top, just at page granularity instead of row granularity. Break the second one and the stage becomes the place where flow control quietly ends, while every stage above and below it still looks correct in isolation.