A submitted program loops over a large collection and runs for two seconds; on a store that runs one operation to completion, what happens to every other caller?
answer
- the unit that protects you also holds you
- one program, everybody queues
- head-of-line blocking
- bounded work, decided when written
- fast at a hundred, fatal at a million
basics
~20 sThey all wait. The program is applied as one uninterleaved unit, so on a store that runs one operation to completion every other caller's work queues for the full two seconds, and their timeouts fire even though nothing has crashed.
solid answer
~50 sThe property that makes a submitted program safe — the store applies it as one unit nothing interleaves — is the same property that makes a long one the worst thing in the system to run. On a store that runs one operation to completion before starting the next, a two-second program is two seconds of **head-of-line blocking**: microsecond operations sit behind it, client deadlines expire, connection pools fill with waiting callers, retries add load, and the tier looks down while it is merely busy. The picture is narrower but not harmless where the server is multi-threaded and locks the entries an operation touches: only the callers needing those entries wait. Assume you cannot stop it — where a store offers any abort, it is safe only before the program's first write, since afterwards there is nothing to undo. So bound the work when you write the program.
go deeper
Recall that the store applies a submitted program as one unit, so while it runs other work waits. Keeping such a program short is the whole of the junior-level rule.
Explain the mechanism and its symptoms: head-of-line blocking, latency that is service time plus wait, deadlines expiring on operations that were never slow, and retries arriving as extra load.
Show production judgment — recognise a latency cliff spread across unrelated keys, know the picture differs where worker threads lock entries instead, assume the program cannot be stopped once it has written, and bound its work when writing it.
Treat the program as shared capacity rather than one caller's code. Decide what bound is allowed at the store, who reviews it, and what signal makes an over-long unit visible before its first incident.
## The guarantee and the hazard are the same property A submitted program is attractive because the store applies it as **one unit that no other caller interleaves**. Turn that sentence around and you have the hazard: for as long as the unit lasts, the store is doing your work and not theirs. A single operation is bounded by what the store's own implementation does; a program is bounded by what *you wrote*, including a loop whose length the data decides. This is why the two prices of shipping a decision to the store are determinism and **blast radius**, and why the second one is the operational concern rather than the theoretical one. ## What the waiting callers actually experience On a store that runs one operation to completion before starting the next, a two-second program means: - every other caller's operation is served after it, so their measured latency is their own service time **plus** their wait; - client-side deadlines expire in bulk, and they expire for operations that were never slow; - connection pools fill with callers parked on a reply, so upstream request threads block behind the pool rather than behind the store; - timeouts turn into retries, which arrive as extra load exactly when the tier has none to give; - the tier looks unavailable to its dependants even though nothing crashed, no memory ceiling was hit and nothing was evicted. The failure presents as a **latency cliff shared by unrelated keys**, which is the signature to recognise: if slow calls are spread across the whole keyspace rather than concentrated on hot entries, something is holding the store rather than contending for one entry. ## It does not look the same on every store This is where an answer built on one product goes wrong. Stores in this class split on execution model: | Execution model | Who waits for a long program | |---|---| | Runs one operation at a time to completion, server-wide | Everybody, for its whole duration | | Several worker threads, each locking the entries its operation touches | Only callers needing those entries; others proceed, and the symptom reads as contention on specific keys rather than a global stall | And not every store in this class offers submitted programs at all, so on some the question never arises. Saying which model you are assuming is most of the answer. ## Assume you cannot stop it Where a store offers any way to abort a running program, it can only be safe **before the program's first write**: once writes have landed there is no rollback and no save point, so stopping it would leave half an effect with no record of which half. Plan as though a program you start runs to its end. That single assumption is what turns "keep it short" from advice into a design rule. ## Bounding the work when you write it 1. **Give it a fixed, small amount of work.** The number of entries it touches should be decided by the program's shape, not by how large a collection has grown. 2. **Keep it to the decision, not the bulk.** The program exists to make a read, a branch and a write one unit. Moving, scanning or rewriting large amounts of data is the caller's job, done in bounded pieces. 3. **Cap anything data-driven.** If the program must walk a collection, take a bounded slice per call and let the caller come back, so each unit stays short. 4. **Review its cost like an operation's, not a request's.** The relevant number is not "is two seconds acceptable to this caller" but "is two seconds acceptable to every caller of this tier, at once." ## The growth trap The most common way a team arrives here is not writing a slow program; it is writing a fast one. A program that walks a collection of a hundred members is imperceptible, ships, and is correct for a year. The collection reaches a million members and the same unchanged program is a multi-second stall for everything on the node. Nothing changed in the code, nothing appeared in a review, and the blame lands on the tier. The defence is to make the bound explicit in the program rather than implicit in today's data size, and to watch the size of any collection a program iterates as an operational signal in its own right.
- The tier shows a latency spike across unrelated keys with no change in memory or eviction. How do you tell a long program from a large entry?Both stall callers, so the discriminator is what was running. A long program is your code, arrives with a deploy or with data growth in the collection it walks, and the stall repeats whenever it is called. A single oversized entry stalls on operations naming that key, and the blocked window tracks that key's traffic. Diagnosing the slow operation itself belongs to the tier's access and operations material; here the point is that the program is a stall you authored.
- Why not simply give the program a deadline so the store cuts it off?Because a deadline that fires mid-way is worse than the wait. There is no rollback: the writes already made stay, and no record says which part ran. Where a store offers an abort at all, it can be safe only before the first write. The bound has to be in the work the program does, not in a timer over it.
- Does the same hazard exist where the server is multi-threaded and locks entries?Yes, narrowed. Only callers needing the entries the program holds wait, so unrelated keys keep flowing and the symptom reads as contention on specific keys rather than a global stall. It is still an unbounded hold on shared state, and a hot entry makes the difference academic.
A counter with a single window: while the person at the front has a two-second errand, everyone behind waits, no matter how trivial their own errand is. A store whose worker threads lock only the entries an operation touches is more like several windows sharing a set of drawers — most people move, and only those needing the drawer that is currently open are stuck.
saying these in an interview costs you the question
- Says other callers get errors rather than waiting.
- Assumes every store in this class stalls every caller equally.
- Plans to abort the program if it runs long.
- Puts bulk data work in the program because it is atomic there.
- Sizes the loop by today's collection size and never revisits it.
- Reads the resulting latency spike as an eviction or memory problem.