Which three contracts can a job offer for emitting a grouped interval's value downstream, and what does each demand of the sink?
answer
- three contracts, not one
- insert, revise, or restate
- the sink's ability decides first
- freshness bought, corrections paid
- restatement rate follows the runtime
basics
~20 sEmit once when the group is declared finished, emit early and correct later, or restate the running value on every input. Only the first suits an insert-only sink; the other two need a sink that replaces a value it already holds.
solid answer
~50 sThree contracts, and the difference between them is what the receiving system has to be able to do. **Emit once on close**: one handoff per group, after the job's running assertion that no older record will arrive has passed the group's end — a sink may simply insert it. **Emit early and correct later**: a provisional value goes out on an elapsed span or a record count, then revisions follow — the sink must apply each revision to the row it already holds, on an identity that distinguishes one interval from the next. **Update on every input**: the group's running value is restated as it changes — the sink holds a latest-value view and absorbs far more writes. The restatement rate follows the runtime's smallest unit of progress, which differs by design. Latency and correction volume move in opposite directions across the three.
go deeper
Recall that a group's value can reach a consumer in three different shapes: once at the end, early and then corrected, or restated as it changes. Knowing the three by behaviour is enough at this stage.
Explain what each contract obliges the receiving system to do, and why an insert-only sink is compatible with only one of them. Say out loud that corrections are revisions to apply, not duplicates to drop.
Demonstrate that you pick the contract from the consumer's capability first and the freshness requirement second, and that you state how a consumer distinguishes a provisional number from a settled one.
Frame the contract as a published interface between two teams. Its latency, its correction volume and its settlement marker are commitments, and changing any of them is a breaking change with no schema diff to show for it.
## The boundary is not the contract A grouping interval decides which records produce one result. A separate rule, the **firing condition**, decides at which moments that group's current value is handed downstream; one such handoff is **an emission**. The contract is the shape of the emission sequence a consumer will see for one group, and it is the part of the design that reaches outside the job. There are three, and interviewers ask for them by behaviour rather than by name. ## The three contracts | Contract | What the consumer receives per group | What the sink must be able to do | Latency of the first number | |---|---|---|---| | **Emit once on close** | Exactly one value, after the group is declared finished | Insert a row; nothing more | The wait for the completeness claim, plus any grace period | | **Emit early and correct later** | A provisional value, then zero or more revisions, then a final one | Replace the value it already holds, keyed so intervals do not collide | As short as the early firing condition allows | | **Update on every input** | A restated running value whenever the group changes | Hold a latest-value view and absorb a high write rate | Effectively the runtime's smallest unit of progress | **Emit once on close** depends on the job's **completeness claim** — its running assertion that no record older than a stated moment will still arrive — because that is what lets a group be declared finished at all. The contract is the cheapest to consume and the slowest to see. **Emit early and correct later** decouples the first number from completeness: the firing condition is an elapsed span on the clock the worker reads, or a count of records folded in. The value is provisional by construction, because records that belong to the group may not have arrived. **Update on every input** goes furthest: the group's value is restated whenever it changes, so the consumer always holds the current answer and never holds a settled one. ## What each demands of the consumer 1. **An insert-only sink is compatible with exactly one of them.** Give it corrections and it accumulates rows: a provisional value and a final value sitting side by side, with nothing in the table saying which is which. 2. **The other two require replacement, not accumulation.** The receiving system must be able to overwrite a value it already holds for the same group. That is a capability question about the sink, decided before the job's contract is chosen. 3. **The consumer needs to know which emission was final**, if it cares. Either the emission carries a marker saying so, or the contract states that nothing follows once the grace period has passed. Without one of those, a reader cannot distinguish a number that is still moving from one that has settled. 4. **Corrections are not duplicates.** A consumer that deduplicates by content, or that discards a second arrival for a group it has already seen, silently keeps the provisional number and throws the corrected one away. ## Where the designs differ The contracts are conceptual; what a given runtime can actually offer is not: - Where records advance **one at a time through long-lived operators**, all three are natural, and restating per arrival is genuinely per arrival. - Where arrivals are **collected for a short span and one finite job runs over the collected set**, the finest emission is one per span, so "update on every input" is really "update once per collected span", and its write volume is bounded by the span rather than by the arrival rate. - Where the model is a **finite two-phase pass materialising to disk between phases**, there is no open group to correct: the whole pass is re-run over a wider input, and the consumer sees one result per run. Corrections in that world are a re-publication, not a revision stream. So "we will just emit updates" is not a portable design statement. Say what the runtime's smallest unit of progress is, and the cost lands in the right place. ## Choosing between them The requirement decides, and it decides in two steps. First, what can the consumer do — insert only, or replace by identity? That eliminates contracts outright. Second, how fresh must the first number be, and how wrong is it allowed to be while it is provisional? Emitting before the group is declared finished buys freshness and pays for it in corrections somebody else must absorb; waiting buys a single settled number and pays for it in latency that no amount of cluster capacity will reduce. The common production answer is a combination: fire early on a short elapsed span, fire again when the group is declared finished, and fire once more for each record accepted during the grace period — with the final emission marked, so a consumer that only wants settled numbers can filter for it.
- How does a consumer know which emission for a group was the last one?Only if the contract tells it. Either the emission carries a marker saying it is final, or the contract states that nothing follows once the grace period after close has passed. Absent both, every value a consumer holds is indistinguishable from a provisional one that has not yet been revised.
- Does 'update on every input' really produce one output per record?Only where the runtime advances a record at a time. Where arrivals are collected for a short span and one finite job runs over them, the restatement is once per span; where the model is a finite two-phase pass, there is one result per run. The contract is the same; its output volume is not.
- Can one job offer different contracts to two consumers?Yes, and it is often the cleanest design: one surface fires early and revises for a screen that can repaint, another emits once on close for a table that can only be inserted into. The cost is two write paths and the discipline of never letting the provisional surface be read as settled.
Election night. One office publishes a single certified total once counting is complete; another publishes precinct counts through the night and replaces them as later precincts report; a third runs a live tally that moves every few minutes. A newspaper that prints one edition can only take the first. A screen that can repaint can take any of them, provided it replaces the old number rather than stacking a new line underneath it.
saying these in an interview costs you the question
- Says a group always emits exactly once, on close
- Sends a revision stream to an insert-only sink
- Treats corrections as duplicates to be filtered out
- Assumes restating per record is available everywhere
- Cannot say which emission is the final one
- Picks the contract before checking what the sink can do