skip to content

Aligning Two Differently-Sampled Sequences

Two feeds never tick together, so the match has to be inexact. Nearest, latest-at-or-before, or a shared grid first: the choice decides whether the joined table can see the future.

on this pageshow

questions

4

A quote feed and a price feed never share a timestamp, so how is each quote matched to the price in effect then?

level: juniorimportance: must knowfreq 48%

answer

  1. the stamps never coincide
  2. ordering, not equality
  3. one driving side, one row each
  4. at most one match per record
  5. absent before the other feed starts

basics

~20 s

Match each quote to the most recent price record whose stamp is at or before the quote's stamp - an inexact ordered match rather than an equality match. The driving side sets the row count: at most one matched record each.

solid answer

~50 s

Two feeds sampled on different clocks share no equal key, so the match has to be inexact and ordered. For each record on the driving side - the feed you want one output row per record of - you take the most recent record on the other feed whose stamp is at or before it: an *as-of match*, meaning matched as of that instant rather than on equality. Some designs offer this as a primitive that takes the direction, an age cap and an entity restriction as arguments. Designs with no such operation assemble the same result: re-space both feeds onto one shared regular grid, carrying each feed's last observed value forward, and then match on equal grid positions; or match on a range condition and keep the latest qualifying record per driving record. Either way the shape is identical - one row per driving record, absent matched columns where nothing earlier exists.

code

pseudocode · 10 lines
pseudocode
# one output row per record on the driving side
result = []
for q in quotes sorted by stamp:
    p = last record in prices, in stamp order, with p.stamp <= q.stamp
    if p exists:
        result.append(q fields + p fields + matched_stamp = p.stamp)
    else:
        result.append(q fields + absent price fields + matched_stamp = absent)

assert len(result) == len(quotes)

go deeper

for a junior

Recall that the two feeds have no equal key, so the match is on ordering: each record takes the most recent record on the other side at or before its own stamp, and the driving side keeps one row per record.

for a middle

Explain the decisions hidden in the match - direction, whether an exactly equal stamp counts, how a tie on the matched side is broken - and why the output's row count is the driving side's rather than anything derived from both.

for a senior

Show that you assert the row count and the count of unmatched records immediately, and that you can build the same result where no primitive exists, naming what the shared-grid construction assumes that the direct match does not.

for a principal

The tradeoff to argue is whether the pipeline commits to a shared regular grid for every feed up front, paying an assumption about spacing everywhere, or matches each pair of feeds on their own stamps and carries the matched stamp through for audit.

## Two clocks, no equal key A quote feed and a price feed are produced by different systems, sampled on different schedules and buffered differently on the way to you, so their timestamps coincide only by accident. An **equality match** - pair records whose key values are the same - returns almost nothing here, because almost no stamp on one side equals a stamp on the other. The business question is not an equality question at all. It is: *for each quote, what was the price in effect at that moment?* That is a statement about **ordering**. The operation that answers it is an **as-of match**: each record on one side is paired with the record on the other side whose stamp is the latest one at or before it - its value *as of* that instant. Nothing is equal; what matters is that the matched stamp is not later, and that no other record of the other feed sits between the two. ## The match, step by step 1. Fix a **driving side**: the feed whose records you keep, one output row each. Here, the quotes. 2. For each driving record, find the last record on the other feed whose stamp is at or before the driving stamp. 3. Attach that record's columns. Where no such record exists - a quote earlier than the first price ever recorded - the attached columns come back absent. Four decisions hide inside step 2, and an interviewer expects the first two unprompted: - **Which direction the search may take.** Latest at or before is the only one that cannot attach a value that did not exist yet at the driving stamp. - **How old a matched value may be** before the match should be abandoned rather than honoured. - **Whether an exactly equal stamp counts.** Some designs let you exclude it, which matters when both feeds stamp the same underlying event. - **Which record wins a tie** among several matched-side records carrying the identical stamp. A common rule is the last one in the order the input was given, so the answer can depend on an ordering nobody chose deliberately; deduplicate that side, or make the rule explicit. ## When the primitive is not there This is where a candidate who has used one ecosystem answers half the question. Some designs ship the inexact ordered match as a first-class operation with the direction, an age cap and an entity restriction as arguments. Others have nothing of the kind, and the same result is assembled by hand. | construction | what you do | what it costs you | |---|---|---| | the primitive, where it exists | name the driving side, the stamp on each side, the direction and any age cap | you still supply the entity restriction yourself; it is never inferred | | one shared grid first | re-space both feeds onto a single regular sequence of positions, carry each feed's last observed value forward into positions it was never measured at, then match on equal positions | the grid's step is a fresh assumption, and a carried value is indistinguishable from a fresh observation unless you keep its original stamp beside it | | range, then latest | pair each driving record with the other feed's earlier records by a range condition, then keep the latest one per driving record | the intermediate can grow toward the product of the two sides before it is reduced; cost is what rules this out, not correctness | Saying "you use the inexact ordered match" and stopping is the answer of someone who has used exactly one tool. Saying "the requirement is the latest value at or before each stamp, and here is how to get it where no single operation does it" is the answer of someone who has ported this work. ## Which side sets the row count An inexact ordered match selects **at most one** record per driving record, so the result carries the driving side's row count - no more when the other feed is dense, no fewer when it is sparse. That makes the row count the cheapest assertion available: capture it before the match, compare it after, and treat any change as evidence that the operation you ran was not the one you meant. What does vary is how many driving records came back with absent matched columns. That count is worth keeping rather than discarding: it is the leading edge of a feed that started late, stopped, or has a hole in it. ## What an interviewer is listening for - That you restate the requirement in terms of ordering before reaching for an operation name. - That you name the direction deliberately instead of letting an unstated one decide. - That you know the result's row count is the driving side's, and say so before being asked. - That absence at the beginning of the span is expected behaviour, not a defect. - That you can describe a construction for tools that have no such primitive.

  • Why does rounding both feeds' stamps to the nearest second and then matching on equality not solve this?
    It converts an ordering question into an equality question and loses on both sides. Records that were seconds apart still land on different values and never match, while unrelated records that happen to round together now match by accident. It also silently rewrites the stamps that every later check depends on.
  • Several records on the matched feed carry the identical stamp. Which one is attached?
    One of them - typically the last in the order the input was supplied, though that is a design's rule rather than a law. The output still has one row per driving record, but the value depends on an ordering you may not have chosen. Deduplicate the matched side first, or reduce those records to one deliberately.
  • What single number would you check immediately after the match?
    The row count against the driving side's row count before the match; they must be equal. Then the number of rows whose matched columns came back absent - expected at the start of the span, suspicious in the middle of it.

saying these in an interview costs you the question

  • Rounds both stamps to the nearest second so the keys become equal
  • Assumes every tool has this as a built-in, so it is one call everywhere
  • Takes whichever record is closest in time without asking whether it is later
  • Expects the result to hold both feeds' records added together
  • Assumes the matched columns can never come back absent
  • Treats a match against a record from hours earlier as good as a fresh one
open as a page

A match to the latest earlier reading always succeeds while any earlier reading exists, so what is a staleness tolerance for?

level: middleimportance: should knowfreq 38%

basics

~20 s

A staleness tolerance caps how old a matched value may be. Beyond that age the match is abandoned and an absent value is produced instead, so a reading from hours ago is never silently presented as the value in effect now.

open as a page

A feature table matched each label row to the nearest sensor reading by stamp, and the model scored far better offline than live - what did the match do?

level: seniorimportance: should knowfreq 52%

basics

~20 s

Nearest takes whichever reading is closer, earlier or later. Every row where the later reading was closer carries a value that did not exist at the label's stamp, so the table contains information from the future. Only the latest-at-or-before direction cannot.

open as a page

An inexact ordered match over a table of many sensors returned a full, plausible result with no error - which two preconditions went unchecked?

level: seniorimportance: nice to knowfreq 34%

basics

~20 s

That the match stayed inside one sensor, and that both inputs were in stamp order. Neither is inferred: without an entity restriction one sensor's reading attaches to another's record, and an out-of-order input is walked as if ordered, producing a plausible table.

open as a page