What breaks in a recursive `yield from` flattener on strings or deeply nested data?

## The idiom, and why it is tempting Recursive flattening is the canonical showcase for `yield from`: a generator that walks a nested structure and, for each child that is itself nested, delegates to a fresh call of itself. It is short, it streams (nothing is materialised), and it reads exactly like the shape of the data. In a payroll CSV import that groups rows by department, then by cost centre, then by employee, a flattener like this is the natural way to turn the nested groups back into a flat row stream for validation. It is also where three separate Python-specific traps meet. ## Trap one: strings iterate forever The test 'is this child nested?' is usually written as 'is this child iterable?', either with `iter()` in a `try`/`except TypeError` or against `collections.abc.Iterable`. Both say yes to `str`. Now follow a one-character string through the recursion: iterating `"a"` produces `"a"`. The recursive call receives the identical object, asks the same question, gets the same answer, and descends again. There is no base case, and the run ends with `RecursionError: maximum recursion depth exceeded`. A multi-character string merely takes one extra level to get there. The fix is a whitelist of atomic types checked *before* the iterable test — `str` and `bytes` at minimum. The same argument applies to any type that is technically iterable but semantically a leaf in your data: iterating a `dict` yields its keys and silently drops every value, which does not raise anything at all and is therefore the worse bug. A flattener should decide what counts as a container by naming those types explicitly, not by asking whether iteration happens to work. ## Trap two: recursion depth is a hard limit Every level of nesting is a real Python frame, created while the delegation chain is built on the first advance. CPython's default recursion limit is 1000 (`sys.getrecursionlimit()`), so structures nested deeper than roughly that raise `RecursionError` — and for a flattener the depth is a property of the *input data*, not of your code, so hostile or merely surprising input decides whether you crash. `sys.setrecursionlimit()` raises the ceiling, but the C stack is the real constraint and pushing it far enough can end the process outright rather than raising, so it is not a fix for unbounded input. Where depth is genuinely unbounded, convert to iteration: push the top-level iterator on an explicit list used as a stack, loop while the stack is non-empty, push a child's iterator when you meet a container, and yield leaves. It is less elegant, it is depth-limited only by memory, and it is what you want when the nesting comes from data you do not control. ## Trap three: delegation depth is not free A popular half-truth is that `yield from` 'short-circuits' the chain, so a deep pipeline costs no more than a shallow one. It does not. Delegation is faster than an equivalent re-yielding loop, because resuming a level is handled inside the interpreter rather than by loop bytecode in the delegating frame, but every level in the chain is still resumed for every item that passes through it. Per-item cost grows roughly linearly with depth: streaming a fixed number of items through a fifty-level chain costs several times what the same items cost through a five-level chain. For flattening real data, where the depth is small and constant, this is irrelevant. For a recursive descent over deeply nested data with millions of leaves, it is the difference between a job that finishes and one that does not. ## The laziness trap on top All three failures are deferred. Calling the flattener runs none of its body — it returns a generator object — so a regression pack of a few hundred cases that merely calls it and asserts no exception passes every case vacuously, including the ones that would recurse forever. Any test of a generator must consume it, and the same applies to your validation stage: if the import pipeline only builds the generator and hands it on, the `RecursionError` surfaces in whatever code eventually iterates, far from the flattener. ## The anti-pattern worth naming The recursive-with-accumulator version is what people write instead, and its most common form carries its own Python-specific bug: a default argument holding a list is evaluated once, when the function is defined, and is therefore shared by every call that does not pass one. The second import run appends to the same list and returns the first run's rows as well. Generators sidestep this entirely — each call gets a fresh frame and nothing is shared — which is a real argument for the `yield from` version once you have handled the string and depth traps.