A batch transcoder's fetch stage fills its job queue faster than the encode stage drains it — what fails first?

## The claim being tested The wrong answer this scenario is built to catch is *"the queue is unbounded, so there is no back-pressure problem — the fetcher just keeps going."* That gets the mechanism right and the consequence exactly backwards. Removing the bound does not remove the mismatch; it removes the *symptom that would have told you about it*, and converts a visible stall into an invisible, monotonically growing memory consumer. Picture a single-threaded batch video transcoder: one pass reads a manifest and enqueues job descriptors (source location, target profile, and — the expensive part — any already-fetched payload), and a second pass dequeues descriptors and encodes them. There is no synchronisation to reason about; the passes simply alternate, and the queue is what carries work between them. ## Why growth is unbounded, not just large Over an interval, queue length changes by (jobs enqueued) minus (jobs dequeued). If the average enqueue rate exceeds the average dequeue rate by any margin at all, that difference accumulates linearly for as long as the run lasts. There is no equilibrium to settle into: the queue has no mechanism that slows the producer or speeds the consumer, so the only thing that ends the growth is exhaustion. This is the difference between *buffering* and *storing*. A buffer's job is to cover short-term variance — fetch stalls on a slow source, encode hits an unusually large file — so that a temporary imbalance in one direction is repaid by a temporary imbalance in the other, and the length returns to its baseline. A permanent surplus is not variance, and a queue cannot repay it. ## The three failures, in the order you meet them **1. Latency, first and quietly.** A job's wait is roughly the number of jobs ahead of it divided by the drain rate. As the queue grows linearly, so does the wait: a job that took twenty seconds end-to-end in the first minute of the run takes an hour by the thirtieth. Throughput — jobs finished per minute — is completely unchanged, so a dashboard that tracks only throughput shows nothing wrong. Worse, the work goes stale: by the time a descriptor is encoded, the source may have been deleted or the requested profile superseded, so effort is spent producing output nobody wants. **2. Memory, next and loudly.** Whether this is minutes or hours away depends entirely on what the descriptor carries. A queue of small records — an identifier and a profile name — can hold millions before it matters. A queue where the fetch stage has already downloaded the payload holds whole media files, and a few thousand entries is the whole address space. The single highest-leverage change is often not the queue at all: keep the payload out of it, enqueue a reference, and let the encode stage pull the bytes when it is ready. **3. Durability, whenever the process ends.** An in-memory backlog is unrecorded work. A crash, a deployment, or a machine losing power discards every queued job with no trace of what was lost, and a restart re-reads the manifest from the beginning — redoing the completed work or, worse, silently skipping the discarded jobs depending on how progress is tracked. ## How to confirm it Instrument the queue's depth and plot it against time. The shape answers the question immediately: | Shape of queue depth over time | Reading | | --- | --- | | Spikes, returns to near zero | Healthy buffering; the queue is doing its job | | Sawtooth around a stable band | Balanced rates with periodic bursts | | Monotonic upward trend | Sustained rate mismatch; capacity will not save it | | Pinned at a ceiling | The consumer is saturated and the bound is what is holding the line | A memory profile alone is a much weaker signal, because it tells you memory grew without telling you *which* accumulation grew. Depth over time names the culprit directly, and it is one counter. ## Why adding capacity is the wrong instinct The reflex when a backlog grows is to give it more room. More room buys time proportional to the added capacity and changes nothing about the trend; a run long enough to expose the mismatch will expose it again slightly later. The real remedies are all about rate or about what is stored: fetch fewer jobs per pass, do less work per job on the encode side, shrink each entry to a reference, or persist the backlog somewhere that is designed to hold it rather than the heap. Choosing a bound is a real decision with real consequences, but it is a decision about the queue's *mechanics* rather than about this failure — and it is worth being clear in an interview that a bound converts silent growth into a visible stall, which is a diagnostic improvement, not a throughput one. ## The sentence to land A queue between two stages is a shock absorber, not a warehouse. When the depth trend is up and not coming back, the queue is not buffering — it is quietly storing your entire backlog in memory, and the fix lives on one side of the queue or the other, never inside it.