A payment arrives whose matching order never appears within the match bound. What becomes of that unmatched record, and who decides?
answer
- the contract decides, not the runtime
- dropped, or one null-padded row
- nothing is knowable at arrival
- the unmatched row waits a full bound
- two latencies in one output stream
basics
~20 sIts fate is a declared property of the join, not a runtime accident: dropped silently, or emitted once with a null counterpart after the bound passes. The pipeline author decides, and a design that does not state which has chosen the silent drop by default.
solid answer
~50 sOnce the bound has passed, the held payment can never match, so the job must either discard it or emit it with nothing on the order side. Under an inner contract it is dropped, and nothing anywhere records that it existed — if unmatched volume matters to the business, it has to be counted deliberately. Under an outer contract the row is emitted **once**, with a null counterpart, at the moment the bound passes; it cannot be emitted at arrival, because at arrival the absence of a partner proves nothing. That has a visible consequence: matched rows appear within seconds while unmatched ones appear a full bound later, so one output carries two very different latencies. Which contract applies is the author's decision to state, and it is worth confirming the runtime offers the one chosen, because several support the inner form only.
go deeper
Recall that a record whose partner never comes has to go somewhere: it is either discarded when its bound passes or emitted with nothing on the other side, and the pipeline says which.
Explain why the decision cannot be made at arrival — absence of a partner then proves nothing — and what each contract asks of the consumer downstream.
Demonstrate the operational consequences: counting expirations because the inner form is silent, and stating two latencies because matched and unmatched rows appear at very different times.
Own the decision as a product statement: what the business does with an unmatched record, whether it belongs in the main result at all, and what the miss rate is allowed to be.
## Three fates, not two, and none of them automatic When a record is held by a time-bounded match — a join whose predicate limits how far apart the two records' moments may be — and its bound passes with no partner, the job has three possible behaviours, and which one happens is decided by the pipeline, not discovered: 1. **Dropped.** The held record is released and nothing is emitted. This is the inner behaviour, and it is silent: no row, no error, no counter unless one was added on purpose. 2. **Emitted once with a null counterpart.** The output row carries the payment's fields and nulls where the order's fields would be. This is the outer behaviour. 3. **Routed to a separate output.** The unmatched record is sent somewhere other than the main result — a reconciliation path, a quarantine area, an alerting stream — so the main result stays inner while the misses are still visible. The third is the one senior candidates reach for and the one designs most often omit, because it gets the observability of the second without polluting the main output with half-empty rows. ## Why the null-padded row cannot be emitted at arrival The obvious shortcut — emit the unmatched row immediately, and correct it if a partner turns up — is not available as a plain fact of the join, and the reason is instructive. At the moment a record arrives, the absence of a partner in the held set proves nothing at all: the partner may be produced a minute from now, and the whole reason both inputs are held is that this happens routinely. The **earliest** honest moment to declare a record unmatched is when the bound has passed, as judged by the job's running claim that no record older than a stated moment will still arrive. Designs do differ here, and the difference matters. Where the output is a stream of revisions rather than a stream of insertions, a job may deliberately publish a provisional unmatched row and later retract or correct it. That is a **stated contract** with the consumer, not the join knowing something at arrival — and it obliges the consumer to apply revisions by key rather than appending rows. Where the output is insert-only, the wait for the bound is unavoidable. ## The cost of the outer contract | Aspect | Inner contract | Outer contract | |---|---|---| | Unmatched record | dropped, no trace | one row with a null counterpart | | When output appears | matched pairs only, at match time | matched at match time, unmatched a full bound later | | Retained size | rate × bound per side | the same; the record was held either way | | Demand on the consumer | none beyond the normal result | must handle nulls, and must tolerate two latencies in one stream | | Detecting a broken upstream | needs an explicit counter | visible in the output itself | Two observations follow from the table. First, the outer contract is not more expensive in memory — the record was held for the full bound regardless, so the only difference is whether anything is emitted when it expires. Second, the latency of the output stops being a single number. A downstream service-level objective written as "rows appear within ten seconds" is simply false for the unmatched rows, and the honest statement is two numbers: match latency for pairs, bound-length latency for the rest. ## What an interviewer is listening for - That the candidate **states the contract** rather than assuming their usual one. "Unmatched payments are dropped" and "unmatched payments appear after thirty minutes with a null order" are both defensible; not knowing which is happening is not. - That they notice the silence of the inner form. A steady stream of unmatched records usually means something upstream is broken, and under an inner contract nothing says so. A counter of expirations is a cheap, deliberate signal. - That they connect the bound to the delay. Lengthening the bound to catch more matches also lengthens how late every unmatched row shows up, which can matter more to the consumer than the extra matches. - That they check the runtime supports what they designed. Not every system in this class offers an outer time-bounded match; where it does not, the behaviour is built by hand — keep the unpaired records, and emit when their bound passes. ## What is not decided here How the claim that expires the bound advances, and what stalls it when a source goes quiet, is a neighbouring subject; this leaf relies on it and does not explain it. Neither does the emission contract for an aggregate over an interval — early value, corrected value, final value — belong here; that is about a group's value being restated, whereas the unmatched record is emitted exactly once whichever way it goes, unless the output is explicitly a stream of revisions.
- Under an inner contract, how do you find out that records are going unmatched?By counting expirations on purpose. The join itself emits nothing when a held record's bound passes, so add a counter of records that expired unmatched, per side, and alert on its ratio to arrivals. A sudden rise usually means an upstream input stopped, a key changed shape, or the moments are being assigned differently on the two sides — none of which appears anywhere in the output.
- Can the unmatched row ever be published before the bound passes?Only as a deliberate revision contract, on runtimes whose output is a stream of changes: publish a provisional row with a null counterpart and retract or correct it if the partner arrives. It buys latency and costs the consumer an obligation to apply revisions by key rather than appending them. Where the output is insert-only, waiting for the bound is the only correct option.
saying these in an interview costs you the question
- Assumes an unmatched record is always dropped silently
- Says the null-padded row can be emitted at arrival time
- Treats a row arriving a bound later as a bug in the job
- Believes downstream filtering can recover rows never emitted
- Assumes every runtime offers an outer time-bounded match