Why is an input read twice when one distributed collection feeds two separate calls that each demand an answer, and which engine models never re-read it?

## A declared step is a description, not a result The variable you are holding after a chain of **assembly calls** — calls that only add a step to the graph and compute nothing — refers to a position in the **step graph**, the ordered set of steps the engine derives from the program, each naming the steps whose output it reads. It does not refer to any records, because none have been produced. So when two separate **demanding calls** — calls that ask for an answer or for output to be written — are made against that same position, the engine has no stored result to serve the second one from. It does what it did the first time: walks back to the sources of that position's sub-graph and runs the whole thing. Reading the input, applying the filter, performing whatever grouping the shared prefix contained — all of it happens twice, in two independent runs. The trap is that the program *looks* like it computed something once and used it twice. In the source text there is one chain and one name. In execution there are two runs that happen to share a shape. ## How you notice it This is a diagnosis question as much as a mechanism question, and the signals are ordinary: - **Bytes read.** The run reporting shows roughly the input's full size for each of the two runs. If the two demands together read twice what the input holds, the shared prefix ran twice. - **The source step appears twice.** Across the two runs, the same read of the same input is listed twice, with similar record counts. - **Wall time is roughly additive.** Two demands over a shared prefix take about twice as long as one, where a reader expecting reuse would predict a small increment for the second. - **Cost.** On storage charged per byte scanned, or on capacity rented by the minute, the duplicate shows up as a line on the bill before it shows up in a conversation. - **The two answers can disagree.** If anything inside the shared prefix is unstable — a value drawn from a clock, a random draw, a source being written to while you read it — the two runs saw different things. The general subject of when two runs of the same job agree is a separate one; here it is only a symptom that tells you the branch really did run twice. ## Where it does not happen Stating "a reused branch is always recomputed" as the model of distributed processing is the error this question is built to catch. It depends entirely on the engine model. | engine model | what a second consumer of the same branch does | why | |---|---|---| | defers assembly, finite job, nothing retained | recomputes the prefix from the sources | the position holds a description, not records | | runs one grouping step at a time, writing every intermediate to storage | reads the intermediate back from storage | the prefix was materialised by construction, not by choice | | keeps one fixed graph running, record at a time | evaluates the prefix once and sends each record down both branches | there is one running graph, not one run per answer | | runs continuous work as a succession of small finite jobs | recomputes within each small job unless something retains it | each small job is a complete run of the same graph | That table is the whole answer to the second half of the question, and it is also the reason the first half must be stated conditionally. In a model where every intermediate hits storage, duplicated computation is not the risk — the unavoidable write of every intermediate is. In a running graph, a branch consumed twice costs an extra downstream path, not an extra read of the source. ## What this question is not about Deliberately pinning a computed result so that downstream steps stop recomputing it — deciding what to pin, where it is held, what it costs in memory or local disk, and when it is a mistake — is a separate subject with its own material. The point here stops at: the recomputation happens, here is why, here is how you would see it, and here is where the model differs. In an interview, naming the remedy in one clause and moving on is the right proportion; explaining it at length usually means the candidate has substituted the remedy for the mechanism. ## Answering it in an interview Three beats. First the mechanism: the handle is a description, each demand runs the sub-graph from its sources, so the shared prefix runs twice. Second the evidence: bytes read roughly doubling and the source appearing twice across the runs, which is how you would prove it rather than assume it. Third the condition: this is the behaviour of a model that defers assembly and retains nothing, and it is *not* the behaviour of a model that writes every intermediate to storage or of one that keeps a single graph running. A candidate who supplies the third beat unprompted is telling you they have used more than one engine in this class.