skip to content

questions

6

A merge pipeline buckets rows by a grouping key: what must that key's equality and its hash satisfy for lookups to stay correct?

level: middleimportance: must knowfreq 72%

answer

  1. the contract has two halves
  2. equality must be an equivalence
  3. one direction only
  4. collisions legal, disagreement fatal
  5. mutating a filed key hides the row

basics

~20 s

Equality must be an equivalence relation — reflexive, symmetric, transitive — and must stay stable while the key is stored. Equal keys must produce equal hashes; unequal keys are allowed to collide, and a collision never means equality.

solid answer

~40 s

Two obligations, and they pull in different directions. First, the equality check must itself be an equivalence relation over key values: reflexive, symmetric and transitive, and consistent over time unless the compared state changes. Second, the hash must **agree with equality in one direction only** — equal keys must hash the same, while unequal keys are free to collide. The one-way direction matters: requiring distinct hashes for distinct keys is impossible, but allowing equal keys to hash differently breaks lookups outright, because the probe lands in a bucket that does not hold the entry. The stability clause is what forbids mutating a field the key compares while that key is filed: the entry keeps the bucket it was filed under, and the row becomes present but unreachable.

code

pseudocode · 13 lines
pseudocode
key = make_key(email: "a@example")
store.put(key, row)              // hash(key) -> bucket 7, filed there

key.email = "b@example"          // a compared field, mutated in place

lookup(key)                      // hash -> bucket 12, nothing there -> miss

probe = make_key(email: "a@example")
lookup(probe)                    // hash -> bucket 7, entry found,
                                 // compare "a@example" vs stored "b@example"
                                 // -> unequal -> miss

// the row is present, and neither value can reach it

go deeper

for a junior

Remember the direction: equal values must hash the same, and values that hash the same still have to be compared. Never edit a value you have already used as a key.

for a middle

Explain why the direction is one-way — a finite digest range over an unbounded key space makes collisions unavoidable — and why equality must be reflexive, symmetric and transitive for a bucketed lookup to mean anything.

for a senior

Recognise the signature in production: a row a scan can see and a lookup cannot, duplicates on re-insert, counts that disagree between two code paths. Trace it back to a mutated key or a digest reading a field equality ignores.

for a principal

Set the rule at the platform level: grouping keys are immutable derived values with a specified digest function, so that durable routing and re-runs on other machines cannot depend on a digest the runtime chose for itself.

## Two halves of one contract A grouping key is a value used to decide 'same customer' and to file the row under that decision. Any structure that buckets by a digest of the key relies on two separate promises, and interviewers probe them together because breaking either one produces the same symptom: a row that is in the structure and cannot be found. 1. **The equality check is an equivalence relation.** Reflexive — a key equals itself. Symmetric — if `a` equals `b` then `b` equals `a`. Transitive — if `a` equals `b` and `b` equals `c` then `a` equals `c`. 2. **The hash agrees with equality, in one direction.** Equal keys must produce the same hash. Unequal keys may produce the same hash, and inevitably will. ## Why equality itself must be an equivalence Lookup answers the question 'is there a stored key equal to this probe?', and a bucketed structure answers it by comparing the probe against *some* of the stored keys — the ones filed in the same bucket, in whatever order they sit there. That answer is only meaningful if 'equal to' partitions the key space into classes: - Break **symmetry** — the classic case is a case-folding key that considers itself equal to a plain text key but not the reverse — and the verdict depends on which value happens to be the probe and which the stored entry. Two structures built from the same rows then disagree. - Break **transitivity** — a key that compares 'close enough' rather than exactly — and membership depends on which stored key is compared first, which is an artefact of insertion order. - Break **reflexivity** — a key whose comparison short-circuits to false on an absent field — and the structure cannot find a row using the very key it was filed under. ## The one-way hash rule | Statement | Does the contract require it? | Why | |---|---|---| | Equal keys produce equal hashes | **Yes** | otherwise the probe is sent to a bucket that does not hold the entry, and the lookup misses a row that is present | | Unequal keys produce unequal hashes | **No** | the digest range is finite and the key space is not, so collisions are unavoidable by counting alone | | Equal hashes imply equal keys | **No** | a shared bucket only means 'compare these'; the equality check is still what decides | | The hash reads only fields equality reads | **Yes, in effect** | a digest that mixes in a field equality ignores can differ for two equal keys, which breaks the first row of this table | The asymmetry is the whole point: the digest is a **necessary** filter, not a **sufficient** one. It narrows the candidates; equality decides. A candidate who says 'different values must hash differently' has inverted the requirement into one no function can satisfy. ## Stability while the key is in use The contract is not only about a single comparison; it is about a comparison repeated over time. A stored key must keep answering the same way for as long as it is filed. Two ways teams break this: - **Mutating a compared field in place.** The entry keeps the bucket computed at insert time. A probe carrying the new value hashes elsewhere and finds nothing; a probe carrying the old value reaches the right bucket and then compares unequal against the mutated stored key. The row is unreachable by either value, re-inserting creates a duplicate, and a full scan still shows it — which is exactly the confusing signature that gets escalated. - **Letting the digest depend on something outside the key's own content.** Ecosystems differ here: some environments derive digests for text in a way that varies between processes, others do not. That is harmless inside one run and fatal the moment a digest is persisted or used to choose a shard, because the same key routes to different destinations on different machines. A durable partition key must be derived from the key's content by a specified function, not from whatever digest the runtime happens to hand you. ## Where this bites in a merge pipeline Grouping keys are usually small derived values: a normalized email, a normalized phone, a composite of a few cleaned fields. Two consequences follow from the contract rather than from any particular structure. First, normalization must happen **before** the key is built, not inside the comparison, so that equality stays a plain comparison of stored content. Second, the key should be treated as immutable once created — build a new key rather than editing a filed one — because immutability is the cheapest way to satisfy the stability clause and the only way that survives a later refactor by someone who has not read this contract.

  • Why must a key's hash never read a field that its equality check deliberately ignores?
    Because two keys that equality calls equal could then hash differently and be filed in different buckets. A probe reaches only one of them, so a lookup misses an entry the structure already holds. The digest must be a function of exactly the state equality compares — a subset would be safe but wasteful, a superset is a defect.
  • A grouping key is mutated after it has been filed. What is the observable symptom?
    The row is present but unreachable: probing with the new value hashes to a different bucket and finds nothing, probing with the old value reaches the right bucket and compares unequal. A full scan still lists the row, re-inserting produces a duplicate, and counts drift apart between the scan path and the lookup path.

saying these in an interview costs you the question

  • Says unequal values must produce different hashes
  • Thinks equal values may hash differently if equality is right
  • Treats a hash collision as proof that two keys are equal
  • Defines equality that matches another kind of value one way only
  • Mutates a compared field while the key is filed
  • Uses a process-local digest as a durable partition key
open as a page

A merge pipeline groups customer rows by a 'same person' rule: which three properties must that rule satisfy for the groups to be well defined?

level: middleimportance: must knowfreq 64%

basics

~20 s

Reflexivity, symmetry and transitivity. A rule with all three is an equivalence relation, and only then does it cut the rows into disjoint groups where every row lands in exactly one group, whichever row you start from.

open as a page

A 'same customer' rule matches names within one edit, so Jon matches Jan and Jan matches Ian: what breaks when you group with it?

level: seniorimportance: should knowfreq 55%

basics

~20 s

The rule is reflexive and symmetric but not transitive, so it has no equivalence classes. 'The group of a row' stops being a property of the data and becomes a property of the traversal order, and chains of near matches fuse distinct people.

open as a page

A matcher emits only direct 'same entity' pairs between customer rows: what does the transitive closure of those pairs give you?

level: seniorimportance: should knowfreq 40%

basics

~20 s

The smallest transitive relation containing every reported pair: two rows are related when a chain of reported pairs links them. Closing reflexively and symmetrically too gives the smallest equivalence containing the pairs, which is the grouping the matcher implies.

open as a page

A near-match matcher can never be made transitive without fusing distinct people: how should the platform define what 'same customer' means?

level: principalimportance: should knowfreq 34%

basics

~20 s

Keep two relations. Identity is defined so that it is an equivalence by construction — equality of a normalized key — and the near-match rule stays advisory, feeding review and search but never partitioning the data or naming anything other systems store.

open as a page

A merged customer group must expose one stable identifier: what property must the choice of canonical representative have?

level: seniorimportance: nice to knowfreq 30%

basics

~20 s

The representative must be determined by the class's members alone — never by arrival order, worker or wall-clock time. Formally it is a function whose value is equal for two rows exactly when they are in the same class.

open as a page