Why might an async generator's `finally` cleanup not run when a consumer breaks out of the `async for`, and how do you make it deterministic?

### What `break` actually does An async generator suspended at a `yield` is a live frame. Its locals are alive, its open `async with` blocks are unexited, and its `try`/`finally` has not reached the `finally`. When the consumer does `break` — or returns, or propagates an exception out of the loop body — none of that changes. The loop simply stops asking for items. The generator is not told anything, and its cleanup code has not run. The generator finishes only when it is *closed*, which means `GeneratorExit` is thrown in at the suspension point so the `finally` can run. Three things can cause that: an explicit `aclose()`, garbage collection of the object, or the event loop's shutdown sweep of the async generators it knows about. ### Why garbage collection is not enough Synchronous generators are easy: when the last reference goes, CPython throws `GeneratorExit` in right there, on the spot, in the collector. An async generator cannot be finalized that way, because its cleanup may `await` — closing a connection, flushing a buffer — and a garbage collector has no event loop to await on. CPython solves this with a pair of hooks (installed through `sys.set_asyncgen_hooks`). asyncio sets them for you: the first-iteration hook registers each live async generator with the running loop, and the finalizer hook, called when the object is collected, *schedules* its `aclose()` on that loop instead of running it inline. The consequences are exactly what you would expect from something scheduled: * The `finally` runs at some later turn of the loop, not at the `break`. * If the loop has already stopped or closed, the cleanup never runs at all, and you may see warnings about a pending task being destroyed or an async generator ignoring `GeneratorExit`. * You are relying on refcount timing, which changes if the generator is caught in a reference cycle, and which is not something to lean on in any case. Run the flight-schedule differ and you can watch it: with a `print` in the `finally`, the message lands *after* the code following the `break`, often only when `asyncio.run()` performs its end-of-run sweep of async generators. In a long-lived service, that sweep never comes, and each abandoned stream holds its decoder and socket until collection happens to catch up. An encoding mismatch on one page in a hundred is enough to leak steadily; across a three-week release train the connection count climbs until something upstream starts refusing. ### The deterministic fix ```python import asyncio from contextlib import aclosing async def main(): async with aclosing(schedule_pages()) as stream: async for page in stream: if bad_encoding(page): break # the generator's finally has already run here ``` `contextlib.aclosing`, added in 3.10, is a tiny async context manager whose `__aexit__` awaits the object's `aclose()`. That makes cleanup lexical: on `break`, on `return`, on an exception, the generator is closed before control leaves the block, on the consumer's own task, with the consumer's own exception handling and timeouts around it. Before 3.10 you wrote the same thing by hand as a `try`/`finally` awaiting `aclose()`. Note the trap: `contextlib.closing` is not a substitute. It calls a synchronous `close()`, which async generators do not have. ### What `aclose()` does, precisely Awaiting it throws `GeneratorExit` at the suspension point. The generator is expected to let that propagate after running its cleanup; if the body catches it and yields again, CPython raises `RuntimeError: async generator ignored GeneratorExit`. Calling it while another task is mid-resume raises `RuntimeError` about the generator already running, so closing is the consumer's job, not a watchdog's. A second `aclose()` on an already-finished generator is a harmless no-op. ### The design that avoids the problem The deeper fix is ownership. A generator that acquires a connection, opens a file, or builds a decoder ties that resource's lifetime to an object whose end is not under the consumer's control. Acquire the resource in the caller with an `async with`, pass it into the generator, and the generator becomes a pure transformation whose abandonment costs nothing. Where the generator must own the resource, treat `aclosing` at every call site as part of its published contract and say so in its docstring. ### Operating it Log a line in the `finally` with a correlation id and you can see, in production, how long after the consumer stopped the cleanup actually happened — the gap between the two timestamps is the leak window. Watch connection and file-descriptor counts rather than heap size, since the objects involved are small and the pain is in the handles. And do not replace asyncio's async-generator hooks to instrument this: the loop needs them for its own shutdown sweep, and taking them over is how prompt cleanup turns into no cleanup.