How do you nest collections.defaultdict, and what breaks when you do?

## Building the nest `default_factory` is called with no arguments and its return value becomes the stored value, so nesting is just a factory that returns another mapping. `defaultdict(defaultdict)` almost works and is the classic near-miss: `defaultdict()` with no argument produces an inner mapping whose own factory is `None`, so the second level raises `KeyError`. The two shapes that do work: ```python # fixed two levels by_date = defaultdict(lambda: defaultdict(list)) by_date["2026-03-11"]["K"].append(4.1) # arbitrary depth, by recursion def tree(): return defaultdict(tree) ``` The recursive form is often called autovivification, borrowed from another language's mappings: any chain of subscripts you write into existence simply exists. ## Breakage 1: the factory has to be picklable Pickling a `defaultdict` pickles `default_factory` **by reference**, i.e. by module and qualified name. A `lambda` has no findable name, so `pickle.dumps` raises `PicklingError` — `Can't pickle <function <lambda>>`. `list`, `int`, `set` and any module-level `def` are all findable and pickle fine. This matters more than it sounds, because pickling happens implicitly. Sending the structure to a worker process serializes it: on 3.14 the `multiprocessing` default start method is `forkserver` on Unix other than macOS and `spawn` on macOS and Windows, and both of those transfer arguments by pickling them — only the explicitly-requested `fork` method sidesteps it by inheriting memory. Writing the structure to a disk cache serializes it. Passing it across a process pool boundary serializes it. The failure therefore shows up far from the line that chose the `lambda`, and the fix is a one-line rewrite to a module-level factory function. ## Breakage 2: a lookup now creates a whole subtree The insert-on-miss behaviour compounds with depth. On a single-level accumulator, a probing lookup costs you one empty list. On `defaultdict(lambda: defaultdict(list))`, `by_date[some_date]` creates an entire empty inner mapping, and `by_date[some_date][some_analyte]` creates the inner mapping *and* an empty list inside it. A clinical-lab loader ingesting a 6,800-row batch keyed by collection date and analyte will, after one report routine probes a handful of dates that a locale-dependent date format never produced, hold phantom branches indistinguishable in shape from real ones. And nothing downstream objects. `defaultdict` is a `dict` subclass, so serializers accept it and write the phantom branches out as genuine empty result sets. The structure has no schema and no validation layer, so a typo in either key level is invisible until a human reads the output and asks why a date nobody collected on has an entry. ## Breakage 3: it is hard to read and hard to hand over `repr` on a nested `defaultdict` interleaves factory objects with data and becomes unreadable past two levels. More importantly the type communicates nothing about the intended shape — `defaultdict(lambda: defaultdict(list))` names neither level, so a reader has to reconstruct "date to analyte to values" from the accumulation code. ## What to do instead, and when to accept it **Convert at the boundary.** Recursively rebuild as plain `dict`s once accumulation is finished — walk the structure and call `dict()` on every level. What leaves the function then has ordinary lookup semantics, an ordinary `repr`, and no factory to serialize. **Name the factory.** Even for a fixed two levels, a module-level `def` beats a `lambda`: it is picklable, it is greppable, and its name (`analyte_index`, `per_date_buckets`) documents the level it builds. **Stop at two levels.** Beyond that, a flat mapping keyed by a tuple — `results[(date, analyte)]` — is usually easier to reason about, easier to aggregate, and immune to half-created branches. A dedicated small class, or a mapping of dataclass values, beats both when the shape is fixed and known. **Accept the nest** when the work is genuinely a short-lived in-function accumulation that never crosses a process boundary and is converted before it is returned. That is a real and common case, and there the nested form is the clearest thing to write. `copy.deepcopy` handles nested `defaultdict`s correctly, factories included, so in-process copying is not among the hazards — it is specifically serialization and the silent branch creation that bite. ## Other factory shapes worth knowing The factory is only required to be callable with no arguments, which leaves more room than the `lambda` habit suggests. `functools.partial` binds arguments ahead of time and is picklable when the underlying function is, so `partial(dict.fromkeys, ANALYTES)`-style defaults travel fine. A class works directly, since instantiating it takes no arguments when `__init__` has defaults, and it gives the inner level a name and a schema in one move. And `dict` itself is the right factory when the inner mapping should behave normally — the outer level fills in, the inner level raises `KeyError` on a bad key, which is often exactly the boundary you want.