yield and Suspended State

**"Generator function" and "generator object" are two different things**, and most confusion about `yield` starts by collapsing them into the single word "generator". ### The function is decided at compile time A generator function is a `def` whose body contains `yield` (or `yield from`) *anywhere* — even inside a branch that never executes. The compiler notices the keyword while compiling the body and flags the resulting code object as a generator; there is no decorator and no runtime check involved. This is why adding one `yield` to an ordinary function silently changes what every existing caller receives: ```python def load(rows): if not rows: return [] # this no longer returns a list to the caller yield rows[0] ``` After that edit, `load([])` returns a generator object that yields nothing, not `[]`. Nothing raises; callers that expected a list simply see an empty iteration. ### The call builds an object, not a result Calling a generator function does three things: it binds the arguments to parameters, it creates a frame for that call, and it wraps the frame in a generator object which it returns. It does **not** execute any statement of the body. Argument *binding* is the one eager step — `g()` on `def g(a): yield a` raises `TypeError` at call time, because binding happens before the generator object exists. Anything inside the body — a type check, a `print`, an `open`, a counter increment — waits. ### The object is an iterator The returned object is of type `types.GeneratorType`. It implements `__iter__` (returning itself) and `__next__`, which is the whole iterator protocol, so it can be used anywhere an iterable is accepted. `for value in gen:` is sugar for calling `iter()` on it and then `__next__()` repeatedly until `StopIteration` is raised, which the loop catches and turns into a normal exit. The first `next(gen)` transfers control into the frame at the first statement of the body and runs until a `yield` expression is evaluated. The value on the right of `yield` becomes the return value of `next()`, and execution **suspends** at exactly that point. The frame is not discarded; it is parked. The next `next(gen)` resumes on the line after the `yield`. This alternation continues until the body falls off the end or executes `return`, at which point `StopIteration` is raised and the generator is finished. ### Why deferred start matters in real code Because nothing runs until consumption, three things follow that trip people up: 1. **Side effects are lazy.** A generator function that logs "starting export" logs nothing when called. If the caller builds the generator and never iterates — say, an early `return` on an empty input list — the body never executes. 2. **Errors surface at the consumer.** An exception raised in the body propagates out of the `next()` call, so the traceback points at the loop, not at the line that created the generator. 3. **Partial consumption means partial execution.** `next(gen)` once on a ten-step body runs only up to the first `yield`; the remaining statements never happen unless iteration continues. ### Checking which one you hold The stdlib distinguishes the two directly: `inspect.isgeneratorfunction(f)` is true of the function, `inspect.isgenerator(g)` is true of the object it returns. `inspect.getgeneratorstate(gen)` reports `GEN_CREATED` for a generator that has been built but never advanced, and `GEN_SUSPENDED` once it is parked at a `yield` — a precise way to answer "has this body started yet?" during debugging. ### The mental model Calling the function writes down *how* to produce values; `next()` is what actually produces one. The generator object is a resumable call that has not started. Every question about generator behaviour — why a print did not appear, why a validation error came from the wrong place, why a counter is one behind — is answered by asking: has anyone called `next()` on it yet, and how many times? ### Driving it by hand versus a for loop Manual advancing makes the timing visible in a way a `for` loop hides: ```python def pair(): yield 1 yield 2 gen = pair() # nothing has run next(gen) # runs to the first yield next(gen) # resumes, runs to the second yield next(gen) # body ends -> StopIteration ``` The third call raises `StopIteration`, because the body reached its end. A `for` loop performs exactly these calls and catches that exception as its exit condition, which is why the laziness is invisible inside a loop: the loop calls `next()` immediately, so the body appears to start "straight away". The deferral only becomes observable when the generator object is created in one place and consumed in another — stored on an object, passed to a helper, or built inside an `if` that the consumer may skip. Exhaustion is also final. A generator that has raised `StopIteration` will keep raising it; it does not rewind. Iterating the same values again means calling the generator function a second time to get a fresh generator object.