Why does a generator expression still yield the old list after the source name is rebound in a document-conversion queue?

## The split: one eager part, everything else lazy A generator expression compiles to an implicit generator function. The compiler evaluates the outermost iterable — the iterable of the *first* `for` clause — in the enclosing scope, calls `iter()` on it, and passes the resulting iterator into the hidden function as its single argument. Only then do you get a generator object back. Nothing else has run: the output expression, any `if` filter, and the iterables of any second or later `for` clause are all evaluated inside the generator, on demand, once per step. That one eager evaluation is what the scenario turns on. A document-conversion queue builds `pending = (convert(p) for p in batch)`, then rebinds `batch` to the next batch of pages before draining `pending`. Rebinding a name changes what the name refers to; it does not reach into an object that already holds a reference to the old list. The generator therefore converts the original pages. A four-person team can lose most of a day to this, because the symptom — per-page scale totals that do not match the pages they think they queued — looks exactly like a floating-point rounding drift, and rounding is the first thing anyone suspects when totals are close but wrong. The tell that it is not rounding: the discrepancy is not small and it is not proportional to the number of pages. It is a whole wrong batch. ## Name capture versus object capture The distinction that actually predicts behaviour: ```python batch = ["a", "b"] pending = (name.upper() for name in batch) batch = ["c"] # rebinding the name: invisible to the generator print(list(pending)) # ['A', 'B'] batch = ["a"] pending = (name.upper() for name in batch) batch.append("b") # mutating the captured object: visible print(list(pending)) # ['A', 'B'] ``` The generator holds the iterator over the object that existed at creation. Rebinding is invisible, mutation is not. This is the same rule as anywhere else in Python, but it surprises people here because the surrounding construct *looks* lazy end to end. The inverse also holds, and it bites in the other direction: any *other* name in the expression is read late. `factor` in `(p * factor for p in batch)` is looked up when each item is produced, so a change to `factor` between creation and consumption does change the output. A generator expression is therefore half-frozen — its source object is fixed, its free variables are not — and that mixture is the real trap. ## Error timing Because the outermost iterable is evaluated and `iter()`-ed at creation, its failures happen at creation: ```python try: g = (x for x in None) except TypeError as exc: print(exc) # 'NoneType' object is not iterable ``` The same is true of a `NameError` for an undefined source name, or an exception raised by a function call that produces the source. But a failure anywhere *else* — in the filter, the output expression, or a later `for` clause — does not surface until iteration, and then it surfaces at whatever call site happens to be draining the generator, often far from where the expression was written. Splitting error handling across creation and consumption like this is the main reason generator expressions are harder to debug than the list comprehension they replace. ## Interaction with mutation during consumption Since the generator keeps a live iterator over the source object, mutating that object *while* the generator is being drained behaves exactly as it would in a `for` loop over it: a list can shift items under the iterator and silently skip or repeat entries, and a `dict` or `set` raises `RuntimeError` if its size changes during iteration. Producer-consumer queues where one side appends while the other drains a generator expression are a reliable source of this. ## How to write it so it cannot happen Three defences, in order of preference. Build the generator expression where it is consumed rather than passing it across a boundary, so there is no window in which the source can change. Where it must be passed, take the source as a *parameter* of a real generator function, which makes the capture explicit at the call site. And where the source is genuinely volatile, snapshot it deliberately — `tuple(batch)` or `list(batch)` — and say in a comment that the snapshot is the point. Materialising the whole comprehension with `list(...)` is the fourth option and is the right one whenever the source is small and the laziness was never buying anything. ## Does this apply to a list comprehension? The same eager evaluation of the outermost iterable happens, but a list, set or dict comprehension runs to completion immediately, so there is no window between creation and consumption in which the difference could be observed. The one place it *is* observable is a class body, where only the outermost iterable is evaluated in the class namespace.