In a reactive media pipeline, how do you choose the execution context for a decode stage versus a stage that waits on remote storage?

## Where does the stage spend its time? A pipeline stage is work handed to a worker, and that worker runs it to completion before taking the next task. The decisive question when choosing an execution context is not how important the stage is or how much data it touches, but **where its wall-clock time goes** — into the processor, or into waiting for something else to answer. - A **compute-bound** stage — decoding an uploaded frame, re-encoding audio, hashing bytes already in memory — turns processor cycles into progress. Its processor time and its wall-clock time are nearly the same number. - A **waiting** stage — reading from a remote store, calling another service, waiting on a handle someone else holds — is idle for nearly all of its wall-clock time. It occupies a worker, but not a processor. The classification, not the stage's name, chooses the context. ## The kinds of worker a stage can run on | Context | Shape | Fits | Breaks when | |---|---|---|---| | Small fixed pool | workers roughly equal to available processors, no growth | compute stages | a waiting stage holds one of its scarce workers | | Elastic pool | opens a worker when all are busy, reclaims idle ones, capped by a ceiling | stages that spend their time waiting | compute is placed on it and workers multiply to contend for the same processors | | Single serial worker | one worker draining one queue | work that must happen one at a time | ordinary throughput work is queued behind it | | Caller-thread execution | no hop at all; the stage runs on whichever worker delivered the value | short non-blocking steps | the step is long or waits, occupying a worker chosen by somebody else | ## Why the compute pool is deliberately small Compute capacity comes from processors, not from workers. A pool with four times as many workers as processors does not perform four times the decoding: the same processors interleave the same total work, so throughput is flat while each individual item now finishes later, because its execution is sliced into more pieces. The extra workers are not free either — each carries a stack, appears in every scheduling decision, and pulls other work's data out of the caches it warmed. Keeping the pool near the processor count also makes the pipeline's compute capacity a number you know. When that pool is saturated, you have a capacity statement rather than a mystery: the host cannot decode faster, and the fix is more hosts or cheaper decoding. ## Why the waiting context is allowed to grow A worker parked in a remote call consumes a stack and a scheduler slot but almost no cycles, so many of them can coexist on one host. Growth is the correct response to waiting work: the number of workers that traffic needs follows how often calls arrive multiplied by how long each one is held. When a burst arrives, opening more workers lets more calls overlap; when it passes, idle workers are reclaimed. The ceiling on such a context is not a performance knob — it is the boundary at which unbounded growth becomes a failure you chose. ## Applying it to an ingest pipeline 1. **Decode the upload.** Pure computation over bytes in memory: the small fixed pool. 2. **Write the derived file to the remote store and await acknowledgement.** Almost entirely waiting: the elastic context. 3. **Compute a checksum over bytes already decoded.** Microseconds of arithmetic: usually no hop at all, because moving it costs more than running it. ## Both mismatches, briefly - **Waiting work on the small fixed pool.** Each waiting task holds one of a handful of workers while using no processor, so the pool's capacity disappears into idleness rather than into decoding. - **Compute work on the elastic context.** Tasks arrive, every worker is already busy computing, so the context opens another — but the processors were saturated before the growth started. Throughput does not rise; contention and memory do. ## What an interviewer is listening for They want to hear the classification step happen out loud, and then hear it checked against evidence rather than asserted. Processor time per stage against wall-clock time per stage tells you which kind of work you have: a stage whose wall-clock time dwarfs its processor time is waiting work, whatever the code looks like. Candidates who answer with a list of context names and no rule for choosing between them have memorised an inventory; candidates who start from where the time goes have the thing the inventory is for. One caution worth voicing: some stages are mixed, doing real computation and then waiting inside the same step. No single context fits such a step, and the answer is to split it rather than to compromise.