How do you guarantee a generator's `finally` cleanup runs when an image-thumbnail worker abandons it half-consumed?

## Why a suspended generator has unfinished business A generator that acquires a resource before its loop and releases it in a `finally` is idiomatic: ```python def thumbnail_chunks(path): handle = open(path, "rb") try: while chunk := handle.read(65536): yield chunk finally: handle.close() ``` While the generator is suspended at the `yield`, the `finally` has *not* run. It cannot: the frame is alive and parked inside the `try`. Cleanup happens only when the frame finally unwinds — because the body finished, because an exception propagated out of it, or because someone closed the generator. Consumers that iterate to exhaustion are fine. The problem is the consumer that stops early: a `break` after the first thumbnail passes a size check, a `next()` used for a peek, an exception in the loop body. The generator is left suspended, holding an open file, waiting for a resume that never comes. ## What `close()` does `gen.close()` raises `GeneratorExit` at the suspension point. It is not an `Exception` subclass — it inherits directly from `BaseException`, so a well-written `except Exception:` inside the generator does not swallow it. The `finally` runs, the frame unwinds, and the generator ends in the closed state. Calling `close()` on an already-closed or already-exhausted generator is a harmless no-op, and closing a generator that was never advanced simply marks it closed without running any of the body — there was nothing to clean up. One rule the generator itself must respect: it may catch `GeneratorExit` to do work on the way out, but it must not `yield` again. If it does, CPython raises `RuntimeError: generator ignored GeneratorExit` at the `close()` call. Refusing to die is not a supported option. ## Why "the garbage collector handles it" is only half true In CPython, when the last reference to a suspended generator goes away, its finalizer calls `close()`, and reference counting normally makes that immediate — which is exactly why the failure is intermittent rather than constant. Cleanup slips when: * the generator is part of a **reference cycle**, so it waits for a cyclic collection pass; * a **traceback holds its frame alive** — an exception raised inside the loop keeps the frame chain, and the whole generator with it, referenced from the exception object for as long as you keep that object; * something **holds the object on purpose**: a list of in-flight workers, a cache, a profiler or debugger, a closure; * you are not on reference-counted CPython at all, or the object is reachable only from another thread's stack. And when the finalizer does eventually run, it runs at an arbitrary point in some other thread's execution, with exceptions inside the `finally` swallowed and printed as `Exception ignored in:`. That is the worst kind of cleanup: late, unordered, and quiet. The observable symptom in an image-thumbnail worker is precisely the annoying kind. Under a 27-minute suite the worker opens far more source images than it closes, file descriptors accumulate, and somewhere past the middle of the run an unrelated open or connect starts failing — an intermittent timeout that reproduces on the whole suite and never on the single test. ## Making it deterministic Do not rely on lifetime. Bind cleanup to a scope: ```python with contextlib.closing(thumbnail_chunks(path)) as source: for chunk in source: ... break ``` `contextlib.closing` calls `close()` on exit regardless of how the block ends, which converts "eventually" into "now". An explicit `try`/`finally: gen.close()` is the same thing written by hand. Two design-level alternatives are worth having ready. First, **do not let the generator own the resource**: open the file in the caller's `with` block and pass the already-open stream in, so the resource's lifetime is a lexical scope rather than a generator's lifetime. Second, when a stage genuinely must own a resource across suspensions, give the consumer a context manager rather than a bare generator, so the API makes the close mandatory instead of optional. ## How to prove it in production Count descriptors over the run rather than guessing: sample the process's open handles periodically, or fail the suite deliberately by tightening the descriptor limit so the leak reproduces early instead of at minute 27. `warnings` in development mode will also surface unclosed file objects finalized by the collector, which points straight at the generator that was never closed.