A merge pipeline buckets rows by a grouping key: what must that key's equality and its hash satisfy for lookups to stay correct?
answer
- the contract has two halves
- equality must be an equivalence
- one direction only
- collisions legal, disagreement fatal
- mutating a filed key hides the row
basics
~20 sEquality must be an equivalence relation — reflexive, symmetric, transitive — and must stay stable while the key is stored. Equal keys must produce equal hashes; unequal keys are allowed to collide, and a collision never means equality.
solid answer
~40 sTwo obligations, and they pull in different directions. First, the equality check must itself be an equivalence relation over key values: reflexive, symmetric and transitive, and consistent over time unless the compared state changes. Second, the hash must **agree with equality in one direction only** — equal keys must hash the same, while unequal keys are free to collide. The one-way direction matters: requiring distinct hashes for distinct keys is impossible, but allowing equal keys to hash differently breaks lookups outright, because the probe lands in a bucket that does not hold the entry. The stability clause is what forbids mutating a field the key compares while that key is filed: the entry keeps the bucket it was filed under, and the row becomes present but unreachable.
code
pseudocode · 13 lineskey = make_key(email: "a@example")
store.put(key, row) // hash(key) -> bucket 7, filed there
key.email = "b@example" // a compared field, mutated in place
lookup(key) // hash -> bucket 12, nothing there -> miss
probe = make_key(email: "a@example")
lookup(probe) // hash -> bucket 7, entry found,
// compare "a@example" vs stored "b@example"
// -> unequal -> miss
// the row is present, and neither value can reach itgo deeper
Remember the direction: equal values must hash the same, and values that hash the same still have to be compared. Never edit a value you have already used as a key.
Explain why the direction is one-way — a finite digest range over an unbounded key space makes collisions unavoidable — and why equality must be reflexive, symmetric and transitive for a bucketed lookup to mean anything.
Recognise the signature in production: a row a scan can see and a lookup cannot, duplicates on re-insert, counts that disagree between two code paths. Trace it back to a mutated key or a digest reading a field equality ignores.
Set the rule at the platform level: grouping keys are immutable derived values with a specified digest function, so that durable routing and re-runs on other machines cannot depend on a digest the runtime chose for itself.
## Two halves of one contract A grouping key is a value used to decide 'same customer' and to file the row under that decision. Any structure that buckets by a digest of the key relies on two separate promises, and interviewers probe them together because breaking either one produces the same symptom: a row that is in the structure and cannot be found. 1. **The equality check is an equivalence relation.** Reflexive — a key equals itself. Symmetric — if `a` equals `b` then `b` equals `a`. Transitive — if `a` equals `b` and `b` equals `c` then `a` equals `c`. 2. **The hash agrees with equality, in one direction.** Equal keys must produce the same hash. Unequal keys may produce the same hash, and inevitably will. ## Why equality itself must be an equivalence Lookup answers the question 'is there a stored key equal to this probe?', and a bucketed structure answers it by comparing the probe against *some* of the stored keys — the ones filed in the same bucket, in whatever order they sit there. That answer is only meaningful if 'equal to' partitions the key space into classes: - Break **symmetry** — the classic case is a case-folding key that considers itself equal to a plain text key but not the reverse — and the verdict depends on which value happens to be the probe and which the stored entry. Two structures built from the same rows then disagree. - Break **transitivity** — a key that compares 'close enough' rather than exactly — and membership depends on which stored key is compared first, which is an artefact of insertion order. - Break **reflexivity** — a key whose comparison short-circuits to false on an absent field — and the structure cannot find a row using the very key it was filed under. ## The one-way hash rule | Statement | Does the contract require it? | Why | |---|---|---| | Equal keys produce equal hashes | **Yes** | otherwise the probe is sent to a bucket that does not hold the entry, and the lookup misses a row that is present | | Unequal keys produce unequal hashes | **No** | the digest range is finite and the key space is not, so collisions are unavoidable by counting alone | | Equal hashes imply equal keys | **No** | a shared bucket only means 'compare these'; the equality check is still what decides | | The hash reads only fields equality reads | **Yes, in effect** | a digest that mixes in a field equality ignores can differ for two equal keys, which breaks the first row of this table | The asymmetry is the whole point: the digest is a **necessary** filter, not a **sufficient** one. It narrows the candidates; equality decides. A candidate who says 'different values must hash differently' has inverted the requirement into one no function can satisfy. ## Stability while the key is in use The contract is not only about a single comparison; it is about a comparison repeated over time. A stored key must keep answering the same way for as long as it is filed. Two ways teams break this: - **Mutating a compared field in place.** The entry keeps the bucket computed at insert time. A probe carrying the new value hashes elsewhere and finds nothing; a probe carrying the old value reaches the right bucket and then compares unequal against the mutated stored key. The row is unreachable by either value, re-inserting creates a duplicate, and a full scan still shows it — which is exactly the confusing signature that gets escalated. - **Letting the digest depend on something outside the key's own content.** Ecosystems differ here: some environments derive digests for text in a way that varies between processes, others do not. That is harmless inside one run and fatal the moment a digest is persisted or used to choose a shard, because the same key routes to different destinations on different machines. A durable partition key must be derived from the key's content by a specified function, not from whatever digest the runtime happens to hand you. ## Where this bites in a merge pipeline Grouping keys are usually small derived values: a normalized email, a normalized phone, a composite of a few cleaned fields. Two consequences follow from the contract rather than from any particular structure. First, normalization must happen **before** the key is built, not inside the comparison, so that equality stays a plain comparison of stored content. Second, the key should be treated as immutable once created — build a new key rather than editing a filed one — because immutability is the cheapest way to satisfy the stability clause and the only way that survives a later refactor by someone who has not read this contract.
- Why must a key's hash never read a field that its equality check deliberately ignores?Because two keys that equality calls equal could then hash differently and be filed in different buckets. A probe reaches only one of them, so a lookup misses an entry the structure already holds. The digest must be a function of exactly the state equality compares — a subset would be safe but wasteful, a superset is a defect.
- A grouping key is mutated after it has been filed. What is the observable symptom?The row is present but unreachable: probing with the new value hashes to a different bucket and finds nothing, probing with the old value reaches the right bucket and compares unequal. A full scan still lists the row, re-inserting produces a duplicate, and counts drift apart between the scan path and the lookup path.
saying these in an interview costs you the question
- Says unequal values must produce different hashes
- Thinks equal values may hash differently if equality is right
- Treats a hash collision as proof that two keys are equal
- Defines equality that matches another kind of value one way only
- Mutates a compared field while the key is filed
- Uses a process-local digest as a durable partition key