Replanning at Runtime

## The correction this leaf is about **Runtime replanning** is the engine re-deciding a part of the plan it has not run yet, using the sizes a finished step actually produced instead of the estimate it started with. The situation in the question is its clearest case: the plan was built expecting 200 GB out of one branch, the branch has now completed, and the recorded output is 200 MB. Every decision downstream of that branch was made against a number that is wrong by three orders of magnitude, and some of those decisions have not been executed yet. Two words carry the whole mechanism. **Finished** - the evidence exists only once a step has completed and its output has been recorded somewhere the engine can count. **Actually** - this is measurement after the fact, not estimation before it, which is exactly what separates it from a query planner choosing a shape from statistics gathered about a table before anything ran. ## Why the opening estimate is wrong so often Something has to run first, so the initial plan is built from whatever the sources declare: file sizes, a schema, a recorded row count. Those describe the data as it sits in storage. They do not describe what a filter, an expansion or an earlier grouping will leave of it, and the error compounds with depth. A large input behind a selective predicate and a join arrives at the next step as a small fraction of itself, and nothing available at submission time knows the fraction. ## What a completed step hands back The evidence available at a step boundary is coarse but exact: - the total bytes and records the step emitted; - the same numbers **per destination** - the share of the output each unit of the next step will read. This is the figure that makes uneven work visible, as one destination far above the median of its peers; - in most designs, nothing about individual key values. The evidence is per destination piece rather than per key, which is why the available corrections are about the *shape of the pieces* and not about the key itself. ## What is open and what is closed | | can change after the measurement | cannot change | |---|---|---| | the step that finished | nothing | its records were produced and moved; that cost is paid | | the consumer's unit count | undersized destinations can be handed together to one worker, or an oversized one divided | the count the producing side already ran with | | an unrun join | may switch to copying the whole small side to every worker, so the large side never moves | a join whose redistribution has already happened | | the result | nothing | every move is meant to be result-preserving; the rows are the same | So the honest answer to *what can it change now* is: the join that has not started, and the number of units the next step runs with. Not the redistribution already paid for, and not the answer. ## Why a small measurement is the valuable one Of the corrections available, the one a 200 MB measurement unlocks is the largest. A join planned as a two-sided redistribution - both inputs sent across the network so records with equal keys meet on one worker - becomes **copying the small side**: the whole of the smaller input is sent to every worker, so the larger input is never grouped by key and never moves. The plan could not choose that at submission time, because it believed both sides were large. The measurement is what makes the choice both legal and safe. ## Where this does not exist at all This is the part that makes the subject answerable by an engineer who has used a different engine. The mechanism needs three things, and engines of this class supply different subsets: 1. **A boundary where a step finishes and its output can be measured.** A runtime that pushes records to their destination as they are produced, materialising nothing in between, never reaches one. 2. **A description the engine is allowed to rewrite.** Where the program describes what is wanted, the engine may re-decide how; where it is a chain of opaque per-record functions, there is nothing to re-decide even when the boundary exists. The oldest, two-phase disk-to-disk model in this family rewrites nothing. 3. **An implementation for the operator in front of it.** Even where the mechanism exists it typically covers groupings and joins rather than every step, and which corrections are enabled out of the box differs between engines. Treat it as a correction that *may* apply, never as a promise that the runtime will notice a problem and fix it. ## What an interviewer is listening for That you separate measurement after the fact from estimation before it; that you know only the unrun part of the plan is in play; and that you state the condition instead of the universal - 'on a runtime where steps finish and their output sizes can be measured'. A candidate who says 'the engine will just handle uneven pieces' has asserted something false on part of this market and only partly true on the rest.