A service fetches a list of records and deduplicates it with new Set(records), but duplicates still get through. Explain why, and how you would deduplicate by content instead.

## Why the Set does nothing Set membership for object values is reference identity. Two objects are "the same value" only when they are literally the same object in memory: ```js const a = JSON.parse('{"id":1}'); const b = JSON.parse('{"id":1}'); new Set([a, b]).size; // 2 ``` Every record that comes out of a parse, a mapper, a spread (`{...r}`) or a merge is a new object. So a `Set` of records deduplicates only in the one case that is never true for fetched data: the very same reference appearing twice in the array. The symptom in production is a Set-based dedupe that silently does nothing and a UI showing repeated rows. ## Step one: decide what identity means The real question is not which container to use but **what makes two records the same record**. Usually there is a domain answer — the primary key, the message id, the normalized email — and the dedupe should be written against it explicitly rather than inherited from reference semantics. ### First-wins with a seen-Set ```js function dedupeBy(items, keyOf) { const seen = new Set(); return items.filter(item => { const k = keyOf(item); if (seen.has(k)) return false; seen.add(k); return true; }); } dedupeBy(records, r => r.id); ``` This keeps the **first** record for each key and preserves the original order. It is the right shape when earlier data is more trustworthy, or when you are streaming and cannot look ahead. ### Last-wins with a Map ```js const byId = new Map(records.map(r => [r.id, r])); const unique = [...byId.values()]; ``` Re-setting an existing key overwrites the value while leaving the key's original position, so this keeps the **last** record for each key in first-seen order. That is usually what you want when later payloads are fresher — a paginated feed where a record was updated between pages. Choosing between first-wins and last-wins is a real decision with visible consequences, and an interviewer will ask which one your code implements. Be able to answer without re-reading it. ## When there is no natural key Sometimes the records genuinely have no id and identity is "all the fields". The tempting one-liner is a serialized key: ```js const seen = new Set(records.map(r => JSON.stringify(r))); ``` It works often enough to be dangerous. The failure modes are worth naming: - **Property order is significant.** `{a:1,b:2}` and `{b:2,a:1}` serialize differently and will not dedupe, even though nothing about the data differs. - **Values are lost.** Properties whose value is `undefined`, a function, or a symbol are omitted entirely, so two records that differ only there collapse into one. - **Some values change shape.** `NaN` and `Infinity` become `null`; a `Date` becomes an ISO string; a `Map` or `Set` becomes `{}`. - **Cycles throw.** A self-referential record raises a `TypeError` rather than producing a key. If you must serialize, canonicalize first — pick the identity fields explicitly and build the key from them in a fixed order: ```js const keyOf = r => `${r.tenant}|${r.type}|${String(r.externalId)}`; ``` An explicit template also documents, in one readable line, exactly what your system considers a duplicate. Guard against separator collisions if a field can contain the delimiter. ## Operational considerations A `seen` Set holds every key it has ever been shown for as long as it lives. That is harmless for a one-shot dedupe of a page of results, and it is a slow leak when the same Set spans a long-running stream or a server process handling an unbounded feed. In that shape you need a bound: reset the Set per batch or per window, or cap it and evict. Deciding the window is a design choice — it defines how far apart two occurrences can be and still count as duplicates. Finally, ask whether the dedupe belongs in the client at all. Duplicates arriving from an API are frequently a symptom upstream — a join fanning out rows, pagination overlapping, an at-least-once delivery guarantee. Client-side dedupe is the right patch while you fix the source, but a senior answer names the source as the real fix and treats the Set as the mitigation.