What breaks when a pipeline fans every element out into its own inner operation with no limit on how many run at once?

## What unbounded actually means A fan-out with no limit subscribes to an inner operation for **every element as it arrives**. Nothing waits for anything. If the source can produce a million records, a million pieces of work are outstanding, and the pipeline's concurrency is a property of the input rather than a property of the system. That turns several independent things into functions of input size: - **Memory.** Every outstanding operation holds its record, its intermediate state and its result buffer. - **Connections or handles.** Remote work holds one per outstanding call, against a supply that is finite. - **Dependency load.** Whatever sits behind the stage now sees concurrency equal to the batch size, from a single caller. - **Latency.** Every outstanding call queues behind the others at the dependency, so per-call latency climbs while throughput does not. - **Failure distance.** The first thing that breaks is usually the dependency or the host, far from the pipeline that caused it. ## Why it passes the test suite The failure is proportional to input, and fixtures are small. A thousand records produce a thousand outstanding operations, which fits in memory and which the dependency absorbs without complaint. The same code at ten million falls over on the first real night. This is why the absence of a bound is a review finding rather than a test finding — nothing in a small run distinguishes bounded from unbounded. ## Choosing the bound The bound should name the **scarce resource**, not the workload: 1. For processor-bound work, roughly the number of cores the process can actually use — more in-flight work than that adds context switching and memory without adding throughput. 2. For work that waits on a dependency, what that dependency can absorb from this caller, which is a number the dependency's owner should recognise and agree to. 3. For mixed work, the tighter of the two, measured rather than guessed. 4. Never the record count, and never a number chosen because it made a benchmark look good on a machine with no other load. **Little's Law** makes the consequence precise: the average in-flight count equals arrival rate times average latency. With a bound of k and average per-element latency L, the stage cannot sustain more than about k/L elements per unit time. Raising k raises the ceiling only while the resource behind the work still scales; past that point latency rises in step and throughput flattens, which is the shape you should expect to see when you plot it. ## The bound is not only a safety device It is also what makes the pipeline **describable**: - Peak memory is a function of k and per-element size, so it can be reasoned about before the run. - The load the dependency sees is a number you can state in a conversation with its owner. - Performance work becomes a single-variable experiment: change k, measure, repeat. - A failure at k=8 reproduces at k=8; an unbounded pipeline's failures depend on the night's input. ## What a bound does not solve - It does not restore source order at the convergence point; that is a separate construction. - It does not help if the real bottleneck is elsewhere — a bounded fan-out feeding a single-writer sink simply queues k results at the writer. - It does not make blocking work safe on a context sized for non-blocking work; where the work runs is a separate decision from how much of it runs. - It does not remove skew: with uneven per-element costs, a bound of k can still leave k−1 slots waiting on one long straggler if the pipeline also restores order downstream. ## How to answer it Say what scales — in-flight work, memory, dependency load — then say what the bound is *for*: making concurrency a property of the system rather than of the input. Finish with the number you would pick and where it comes from. An answer that only says *you might run out of memory* has named the least interesting of the four consequences.