skip to content

Why does a Python list comprehension's loop variable not leak into the enclosing scope?

level: juniorimportance: must knowfreq 55%

answer

  1. Where does the loop name live afterwards?
  2. Python 2 behaved differently here
  3. The comprehension gets a scope of its own
  4. Implicit function scope, discarded on exit
  5. PEP 709 inlined it in 3.12

basics

~20 s

Since Python 3, a comprehension evaluates its body in its own implicit function scope, so the name bound by its for clause is local to the comprehension and gone afterwards. Python 2 list comprehensions leaked it; generator expressions never did.

solid answer

~40 s

A list, set or dict comprehension in Python 3 runs its loop body in an **implicit function scope** of its own. The name bound by the `for` clause is local to that scope, so after the comprehension finishes the name is unbound, and a same-named variable outside is untouched even though the comprehension shadowed it during the loop. Reading still works in the normal direction: the comprehension can see enclosing locals, globals and builtins; only *binding* is contained. Python 2 list comprehensions shared the enclosing scope and leaked the loop variable, which is why the question is still asked; generator expressions have been isolated since they were introduced. A plain `for` statement is not a scope, so its target does leak and survives the loop.

code

python · 15 lines
python
x = "outer"
squares = [x * x for x in range(4)]
print(squares)   # [0, 1, 4, 9]
print(x)         # outer

try:
    print(item)
except NameError:
    print("item was never created")

total = sum(item for item in range(4))
try:
    print(item)
except NameError:
    print("the generator expression did not leak either")

go deeper

for a junior

Recall the one-line rule: the comprehension's loop name is local to the comprehension and gone afterwards, and a variable of the same name outside keeps its value. Be ready to predict what a short snippet prints.

for a middle

Explain the mechanism, not just the outcome: the body runs in an implicit function scope, which is why shadowing is harmless and why reading enclosing names still works. Contrast it with a for statement, whose target is not scoped.

for a senior

Show you know the rule survived an implementation change: 3.12 inlines list, set and dict comprehensions, so the guarantee is now enforced by save-and-restore rather than a call frame, and tracebacks lost the <listcomp> frame. Say why nothing in production should depend on which mechanism runs.

for a principal

Frame it as an API-stability argument: a language guarantee stayed fixed while its implementation was replaced for speed, and the only user-visible fallout was in tracebacks and profiles. That is the standard to hold your own internal libraries to when you optimise them.

## The rule In Python 3, every list, set and dict comprehension, and every generator expression, evaluates its body in a scope of its own — an *implicit function scope*. Any name bound by its `for` clauses, including names produced by tuple unpacking such as `for k, v in pairs`, lives and dies inside that scope. Code in a comprehension cannot create or overwrite a variable in the code around it. There are exactly two documented ways out, both of which are separate topics: an assignment expression (`:=`), which is specified to bind in the containing scope, and the outermost iterable, which is evaluated in the enclosing scope before the comprehension's own scope exists. ```python row = "header" widths = [len(row) for row in ("aa", "bbb")] print(widths) # [2, 3] print(row) # 'header' — untouched ``` The comprehension shadowed `row` for the duration of the loop and then vanished, leaving the outer binding exactly as it was. ## Why it works this way Python 2's list comprehensions were defined as syntactic sugar for a `for` statement executed in the current scope, so they leaked: after `[x for x in range(3)]` the name `x` survived, bound to `2`. That was a frequent source of quiet bugs — a comprehension in the middle of a function silently clobbered a variable defined above it. Generator expressions, added later, were compiled to a real generator function from the start and therefore never leaked, so Python 2 shipped two comprehension-like constructs with different scoping. Python 3 resolved the inconsistency by giving list, set and dict comprehensions the same isolation the generator expression already had. ## How CPython implements it Through Python 3.11 the implementation was literal: the compiler synthesised an anonymous function — you can see it in `dis` output and in tracebacks under the name `<listcomp>`, `<setcomp>`, `<dictcomp>` or `<genexpr>` — and called it immediately, passing the already-evaluated outermost iterable as its single argument. The loop variable was simply a local of that hidden function. Python 3.12 changed the implementation without changing the rule. PEP 709 *inlines* list, set and dict comprehensions into the enclosing code object: there is no function object to create and no call to make, which makes short comprehensions substantially faster (the PEP measured up to about twice as fast). Isolation is preserved by the compiler instead of by a call frame — if the enclosing scope already has a variable of the same name, it is saved before the loop and restored afterwards. The observable effects of the change are that a traceback raised inside a comprehension no longer shows an extra `<listcomp>` frame, and that profiling no longer attributes time to a separate function. Generator expressions are *not* inlined; they still compile to a real generator function, because their body must be able to suspend. So the isolation is a language guarantee that has held since Python 3.0, and the 3.12 work is an optimisation underneath it. Nothing you write should depend on which mechanism is in use. ## Isolation is only about binding The comprehension's scope is nested inside the code that contains it, so reading names works normally: enclosing function locals are reachable as closure variables, module globals and builtins resolve as usual. ```python def scale(values): factor = 10 return [v * factor for v in values] # factor read from the enclosing function ``` Only *creating or rebinding* a name is contained. That is precisely the asymmetry that makes comprehensions safe to drop into existing code: they consume the surrounding namespace but never mutate it. ## What still leaks A `for` statement is not a scope. After a plain loop, its target is still bound, holding the last value produced — and if the iterable was empty, it is not bound at all, which is why code that reads the loop variable after the loop can raise `NameError` on empty input. `while` loops, `if` blocks and `with` blocks are likewise not scopes. In Python, only modules, functions (including the implicit ones behind comprehensions), classes and, since 3.12, the annotation and type-parameter scopes introduce a new namespace. ## What interviewers are checking Three things, in order. First, that you know the loop variable does not survive — so you cannot use the Python 2 idiom of reading it afterwards. Second, that you know the mechanism is a scope rather than a `del` at the end, which is what predicts the shadowing behaviour: an outer variable of the same name keeps its value rather than being deleted. Third, that you do not overgeneralise the rule to `for` statements, which is the mirror-image mistake and shows up in real code as a variable that unexpectedly still holds the last item.

  • Does the same isolation apply to generator expressions and to a plain for statement?
    Generator expressions have always been isolated — they compile to a real generator function, so the loop name is a local of that function and never leaks, in Python 2 or 3. A `for` statement is not a scope at all: its target is an ordinary variable in the surrounding namespace, still bound to the last item after the loop ends, and left untouched (possibly unbound) if the iterable was empty.
  • What did PEP 709 change about comprehensions in Python 3.12?
    It inlines list, set and dict comprehensions into the enclosing code object instead of creating and calling a hidden function. Short comprehensions get substantially faster, and tracebacks no longer show a separate `<listcomp>` frame. The scoping guarantee is unchanged: the compiler saves and restores any same-named enclosing variable, so the iteration name still cannot leak. Generator expressions are not inlined.
  • Can a comprehension read a variable from the function that contains it?
    Yes. The implicit scope is nested inside the containing scope, so enclosing locals, module globals and builtins all resolve normally, exactly as they would inside a nested function. Only binding is contained — the comprehension cannot create or rebind a name outside itself, apart from an assignment expression, which is specified to bind outward.

The comprehension borrows the room it runs in but brings its own whiteboard: it can read everything already on the walls, and everything it writes is wiped when it leaves.

saying these in an interview costs you the question

  • Says the loop variable leaks, as it did in Python 2
  • Claims a for statement's target is scoped to the loop
  • Thinks the comprehension deletes a same-named outer variable
  • Says generator expressions leak but list comprehensions do not
  • Believes global or nonlocal is needed just to read outer names
  • Assumes 3.12 inlining removed the scoping guarantee

context