When is {**base, **overrides} the wrong way to layer configuration dictionaries in Python?
answer
- One promise: later operand wins
- Top level only, nothing recursive
- Order is the policy, not the data
- New dict, same value objects
- ChainMap is lazy and first wins
basics
~10 sIt is wrong whenever the layers nest, whenever precedence should follow something other than operand order, or whenever the result must not share objects with its sources. The merge is shallow, positional and eager.
solid answer
~50 sA dict display merge encodes exactly one policy: the mapping written last wins, key by key, at the top level only. That breaks in three ways. It is **shallow** — a nested settings dict from the later layer replaces the earlier one wholesale instead of merging inside it. Its precedence is **positional**, so it knows nothing about which record is actually newer or more authoritative; if you order the operands by a timestamp you have made clock accuracy part of correctness. And it is **eager and shallow-copying** — a fresh top-level dict per merge whose values are still the same objects as the sources, so a nested list mutated later is visible through both. Reach instead for `collections.ChainMap` when you want a lazy layered lookup, a hand-written recursive merge when the layers nest, `copy.deepcopy` when sharing is unsafe, or a validated settings type when the precedence rules are real policy.
code
python · 3 linesbase = {'locale': 'de', 'limits': {'batch': 10, 'retries': 3}}
incoming = {'limits': {'batch': 50}}
print({**base, **incoming})go deeper
Recall the one rule the merge follows: entries are inserted left to right, so a key in both mappings takes the value from the one written last, and only the top level is considered.
Explain the three mechanics behind the failures: the merge is shallow, it copies only the top-level container while sharing the values, and its precedence is fixed by operand order rather than by anything in the data.
Diagnose the silent cases — a nested sibling key vanishing, an older record winning because it happened to be written last, a cached layer mutated through a merged view — and name the alternative for each: a recursive merge, an explicit field comparison, copy.deepcopy or ChainMap.
Own whether ad-hoc dict merges should carry configuration policy at all: where precedence is defined, whether it is auditable, and what a validated settings type with an explicit resolution order buys over merge expressions scattered across the codebase.
## What the merge actually promises `{**base, **overrides}` builds a new dict by inserting every entry of `base` and then every entry of `overrides`, so a key present in both ends up with the later value. That is the whole contract. It is a fine default, and for two flat mappings of scalars it is the clearest thing you can write. The failures all come from reading more into it than it says. ## Failure one: the merge is shallow Consider a translation-memory updater that keeps per-locale entries and applies incoming records over a cached base. If the entries nest — `{'locale': 'de', 'limits': {'batch': 10, 'retries': 3}}` merged with `{'limits': {'batch': 50}}` — the result is `{'locale': 'de', 'limits': {'batch': 50}}`. The whole `limits` mapping was replaced, so `retries`, which only the base carried, is silently gone. Nobody gets an error; a field simply stops existing. When layers nest you need a recursive merge that walks both sides and replaces only leaves, and you have to decide explicitly what happens when one side has a mapping and the other a scalar. ## Failure two: precedence is positional, not semantic The operand order *is* the policy. That is fine when the order encodes a real hierarchy — built-in defaults, then a file, then the environment, then flags. It is not fine when the order stands in for something else. Suppose the updater merges records from two mirrors and orders them by an embedded `updated_at`, so the newer one is written second. A clock-skew artefact on one mirror — a host a few seconds ahead — silently inverts the order and the older record wins. The merge did exactly what it promised; the *policy* was wrong, because "newer wins" was expressed as "whatever I put last wins". If recency is the rule: - resolve it explicitly by comparing fields, - prefer a monotonic version counter to a wall clock, - and record which source won so the decision is auditable. On a three-week release train a mis-ordered merge that produces no error can ride a full cycle before anyone notices. ## Failure three: absent and explicitly-null are indistinguishable A merge cannot tell "this layer does not mention the key" from "this layer sets it to `None`". The second silently clobbers a good base value. If your layers need to express "leave this alone", they need a sentinel of their own — a module-level `MISSING = object()` that the merge step filters out — because `None` is real data. ## Failure four: the result shares its values The new dict is a top-level copy only. Every value in it is the same object that was in `base` or `overrides`. If the updater later mutates a list inside the merged config, the cached base mutates with it, and the bug surfaces on the *next* record, far from the mutation. `copy.deepcopy` fixes it at a real cost; freezing the layers, or only ever building fresh values, avoids it more cheaply. ## Failure five: it is eager Every merge allocates a dict sized to the union of the layers. Merging a large mapping per record is a real allocation on a hot path. `collections.ChainMap` is the lazy alternative: it holds references to the layers and searches them on lookup. Note its precedence is the mirror image — the **first** mapping in the chain wins, not the last — and writes land in the first mapping only. A ChainMap is a view, so a later change to an underlying layer is visible through it, which is sometimes the feature and sometimes the bug. ## How to talk about the alternatives - A **recursive merge** answers nesting. - An **explicit field comparison** answers precedence that is semantic rather than positional. - A **sentinel** answers absent-versus-null. - `copy.deepcopy` answers sharing. - `collections.ChainMap` answers cost and lets you keep the layers distinct and inspectable, which also makes "where did this value come from?" answerable — something a flattened dict destroys. Each has a price, and naming the price is what separates a senior answer from a syntax answer. ## When the display merge is right Flat mappings, scalar values, a precedence that genuinely is "the caller's overrides beat my defaults", and a merge that happens once at startup rather than per record. That covers most small uses, and the display form is the most readable thing available for them. It stops being right the moment the data nests, the ordering means something, or the merge runs on every item — and the failure mode in all three cases is silence, which is why a senior engineer is expected to name the boundary rather than the syntax.
- How does collections.ChainMap differ from building {**a, **b}?ChainMap copies nothing: it keeps references to the mappings and searches them in order on each lookup, so later mutations of an underlying layer show through. Its precedence is reversed — the **first** mapping wins, where a display merge lets the **last** operand win — and writes and deletions apply to the first mapping only. It suits a large or frequently rebuilt layering; the display merge suits one flat snapshot.
- Why is 'key absent' versus 'key explicitly None' a problem for this merge?The merge inserts every entry of the later mapping, including one whose value is `None`, so a layer meaning "I have no opinion" clobbers a good base value the moment it spells that opinion as `None`. The fix is a dedicated sentinel — a module-level `MISSING = object()` that the merge step filters out — leaving `None` free to mean a genuine null.
- What breaks if you order the merge operands by an embedded timestamp?You have made clock accuracy part of the correctness of the merge. A host whose clock runs ahead makes an older record sort last and win, with no error anywhere. If recency is the policy, compare the fields explicitly at merge time, prefer a monotonic sequence or version counter over a wall-clock time, and log which source won so the choice can be audited afterwards.
saying these in an interview costs you the question
- Assumes the merge combines nested dictionaries recursively
- Thinks precedence follows recency rather than operand order
- Believes the merged dict is independent of its sources
- Uses ChainMap expecting the last mapping to win
- Merges large mappings per record without weighing the copy cost
- Lets None from a later layer mean absent