How does a Python generator preserve its local variables across a yield?
answer
- Ordinary calls throw something away
- Generators keep what calls discard
- Ownership moves off the call stack
- The pause point is stored, not replayed
- gi_frame is None once finished
basics
~20 sThe generator object owns the call's frame. A yield suspends that frame instead of destroying it, so locals and the resume point survive. Resuming continues in the same frame; the frame is released only when the generator finishes.
solid answer
~40 sAn ordinary function's frame is thrown away when it returns, which is why its locals vanish. A generator is different: the generator object *owns* its frame for the whole life of the generator, exposed as `gi_frame`. Hitting `yield` suspends that frame — locals, loop counters and the position in the body are all left exactly as they are — and control returns to the caller. The next `next()` re-enters the same frame and continues from the statement after the `yield`. You can watch the transitions with `inspect.getgeneratorstate()`, which reports `GEN_CREATED`, `GEN_SUSPENDED`, `GEN_RUNNING` and `GEN_CLOSED`; `gi_running` is true only while the body is actually executing. Once the generator finishes or is closed, `gi_frame` becomes `None` and the locals it held are released.
code
python · 14 linesimport inspect
def scan(limit):
seen = 0
while seen < limit:
yield seen
seen += 1
gen = scan(2)
print(inspect.getgeneratorstate(gen))
next(gen)
print(inspect.getgeneratorstate(gen), gen.gi_frame is None)
list(gen)
print(inspect.getgeneratorstate(gen), gen.gi_frame is None)go deeper
Know that a generator remembers where it stopped and what its variables held, so a counter inside it keeps counting across next() calls rather than resetting.
Explain the mechanism: the generator object owns the frame, yield suspends rather than unwinds it, gi_frame exposes it and becomes None at completion, and inspect.getgeneratorstate names the four states.
Demonstrate the lifetime consequences in production code — held file handles and connections, abandoned suspended generators pinning memory, cleanup that depends on close rather than collection, and the re-entrancy ValueError.
Own the tradeoff of shipping suspended state across a system: generators are cheap to hold but tie resource lifetime to consumer behaviour you do not control. Decide where that is acceptable and where an eager, closed-over-nothing result is safer.
### Frames are the unit that dies, or does not Every Python call gets a **frame**: a small object holding that call's local variables, the position of the next instruction, and the working values of the expression being evaluated. For an ordinary function, the frame is created on call and dropped on return, which is exactly why locals do not survive: nothing references the frame any more. A generator changes the ownership. When you call a generator function, the created frame is handed to a **generator object**, and that object keeps it. The frame's lifetime is now tied to the generator, not to any single `next()` call. ### What yield actually does `yield` is not a return. Evaluating it hands the yielded value back to whoever called `next()` and then **suspends** the frame: the instruction pointer is left pointing just past the `yield`, the locals stay bound, any partially-evaluated state is retained, and control leaves the frame without unwinding it. Nothing is copied or serialized — the frame simply stops being executed. Calling `next()` again re-enters that same frame and resumes at the saved position. From the body's point of view, the `yield` expression "returned" and execution continues on the next line. A loop counter therefore keeps counting, an accumulator keeps accumulating, and an open file object bound to a local stays open. ```python def running_total(values): total = 0 # runs once, on the first next() for v in values: total += v # total survives every suspension yield total ``` ### Introspecting the suspended state The generator object exposes its state directly: * `gi_frame` — the frame being kept alive, or `None` once the generator has completed or been closed. * `gi_running` — `True` only while the body is currently executing; `False` while it sits suspended. * `gi_code` — the code object of the generator function. Higher level, `inspect.getgeneratorstate()` collapses those into four readable states: `GEN_CREATED` (built, never advanced), `GEN_SUSPENDED` (parked at a `yield`), `GEN_RUNNING` (executing right now), `GEN_CLOSED` (finished or closed). In a debugging session that is usually what you want to print. ### One frame, not a thread A suspended generator is not a thread and not a task on a scheduler. There is exactly one frame, and it only executes while somebody is inside a `next()` call — on the *caller's* stack, in the caller's thread. Between calls, nothing about the generator is running: it consumes memory, not CPU. That is what makes generators cheap compared with a thread per producer. The single-frame model also explains re-entrancy. Because a frame cannot be executing in two places at once, advancing a generator from inside its own body raises `ValueError: generator already executing` rather than corrupting the frame. This is a real hazard when a generator's body calls code that, directly or indirectly, iterates the same generator. ### Lifetime consequences worth knowing The kept-alive frame holds strong references to everything its locals name. A generator suspended on line 3 of its body pins whatever it opened or built there for as long as the generator object is reachable. Two practical effects: * **Resources stay held.** A generator that opens a file and yields lines keeps the handle open while it is parked. If a consumer breaks out of the loop and drops the reference, the generator is eventually closed and its `finally` blocks run — but the timing depends on collection, so an explicit `with` inside the body, or closing the generator deliberately, is the reliable route. * **Abandoned generators are not free.** A suspended generator that nobody will ever advance again still holds its frame and its locals until it is collected. When the body finally completes, the frame is released and `gi_frame` drops to `None`. At that point the generator is a small, inert object: its state is gone, and the locals it was protecting are collectable. ### The interview shape of this The question is really "where does the state live?" The answer — in a frame owned by the generator object rather than by the call stack — explains everything downstream: why locals persist, why `gi_frame` exists, why re-entrancy is an error, why a generator pins resources, and why resuming is cheap. ### Why this is the cheap way to keep state The alternative to a suspended frame is writing the state down yourself: an iterator class with `__next__`, instance attributes holding the counter, the position and any partially-built value, and a hand-rolled state machine to decide what to do on each call. Generators exist so the compiler and the interpreter do that for you — the body reads as ordinary straight-line code, and the frame *is* the state machine. Anything you would have stored on `self` is simply a local that survives the pause. That is also the honest answer to "is a generator just syntax sugar?" It is sugar over an iterator class, but the sugar buys real correctness: the resume point is tracked by the interpreter rather than by a `self.stage` integer you have to keep in sync with the branches of your own `__next__`.
- What happens if code inside a generator's body calls next() on that same generator?It raises `ValueError: generator already executing`. There is one frame and it cannot be re-entered while it is running, so CPython refuses rather than corrupting the state. `gi_running` is the attribute that is `True` during that window. In practice this shows up indirectly — a helper called from the body ends up iterating the same generator object.
- Does a suspended generator consume CPU while it waits between next() calls?No. It holds memory — its frame and everything its locals reference — but nothing executes between calls. A generator is not a thread and has no scheduler behind it; the body only runs on the caller's stack, inside a `next()` call. That is why thousands of suspended generators are cheap where thousands of threads are not.
- If a generator opens a file in its body and the consumer breaks out of the loop early, what happens to the handle?The generator stays suspended with the handle held by a local, so the file remains open until the generator object is collected or explicitly closed, which then runs any `finally` or `with` cleanup in the body. Relying on collection timing is fragile; put the resource in a `with` block inside the generator so the cleanup is bound to the body's exit.
A normal call is a scratch pad thrown away when the work ends. A generator is the same pad with a bookmark clipped in: the writing stays, the bookmark says where to carry on, and both are kept by whoever holds the pad.
saying these in an interview costs you the question
- Says locals are copied out and restored on resume
- Thinks the body re-runs from the top each next()
- Believes a suspended generator runs on its own thread
- Cannot name where the paused state lives
- Assumes gi_frame stays set after exhaustion
- Thinks a suspended generator holds no memory