skip to content

Why does weakref.finalize(obj, obj.close) never fire while the program runs?

level: middleimportance: must knowfreq 40%

answer

  1. Something is still holding the object
  2. The registry outlives the object
  3. What does obj.close carry with it
  4. A bound method stores its instance
  5. Capture the descriptor, never self

basics

~20 s

A bound method holds a strong reference to its instance, so storing obj.close as the callback keeps obj permanently reachable. The object is never collected, and the finalizer only runs during the interpreter's exit pass. Capture the raw resource instead.

solid answer

~50 s

`weakref.finalize` keeps `func`, `args` and `kwargs` in a module-level registry that lives for the whole process. `obj.close` is a **bound method**, and a bound method holds a strong reference to its instance through `__self__`, so the registry now transitively holds `obj`. Its refcount never reaches zero, the collector never reclaims it, and the callback is deferred to the exit pass — cleanup that was supposed to be prompt happens only when the process is already shutting down, all at once. The rule is that neither `func` nor any argument may reference the referent, directly or indirectly. Capture what the cleanup actually needs instead: a file descriptor number, a path, a socket, a lock — anything but the owner. A module-level function, a `staticmethod`, or a closure over those values all work; a bound method, a lambda that names `obj`, and a `functools.partial` carrying `obj` all fail the same way.

code

python · 14 lines
python
import gc
import weakref


class Handle:
    def close(self):
        print("closed")


leaky = Handle()
trap = weakref.finalize(leaky, leaky.close)
del leaky
gc.collect()
print(trap.alive)

go deeper

for a junior

Remember the rule of thumb: never pass the object, or one of its methods, as the cleanup callback. Pass the plain thing that needs closing, such as a path or a file descriptor number.

for a middle

Explain the mechanics precisely: a bound method stores its instance, the finalizer registry lives for the whole process, so the referent stays reachable and only the exit pass ever runs the callback.

for a senior

Describe how you would find this in a running service and prevent it: a collect-and-assert regression test around every class that registers a finalizer, and a review rule that a callback may capture resources but never owners.

for a principal

Frame it as an ownership question. Decide where cleanup logic lives so that it never needs the owner, and set the team convention on when non-deterministic finalization is acceptable at all versus explicit scoped release.

## The trap in one line `weakref.finalize` holds a *weak* reference to the referent and *strong* references to everything else you hand it. If what you hand it can reach the referent, you have quietly built a strong reference from a process-lifetime registry back to the object you wanted collected. ## Why a bound method is the usual culprit Writing `obj.close` does not name a function on the class — it builds a new **bound method** object on the spot, and that object stores the instance in `__self__`. So `weakref.finalize(obj, obj.close)` stores, in the registry: * a weak reference to `obj` (harmless), and * a bound method that strongly references `obj` (fatal). The second one wins. `obj`'s reference count never drops to zero and the cyclic collector cannot help either, because the registry is a live root, not garbage. The finalizer stays `alive` forever. ```python import gc, weakref class Handle: def close(self): print("closed") h = Handle() f = weakref.finalize(h, h.close) # bound method pins `h` del h gc.collect() print(f.alive) # True - nothing was collected ``` The same shape hides in `lambda: obj.close()`, in `functools.partial(some_func, obj)`, in a default argument such as `func=obj.flush`, and in an object graph one level deeper: a logger you pass as an argument that happens to hold a back-reference to `obj`. ## What actually happens instead of nothing The failure is not "the callback is lost" — it is worse, because it looks like it works. The finalizer is still registered, so at interpreter shutdown the exit pass calls it. So the resource does get released, five hours late, during shutdown, along with every other deferred finalizer at once. In a long-running process the symptom is a slow climb in descriptors, temp files or memory that vanishes the moment you kill the process for investigation — the least helpful bug shape there is. ## The fix: capture the resource, not the owner Ask what the cleanup genuinely needs. Almost always it is a small, plain value: an integer file descriptor, a path string, a socket, a connection handle from a lower layer. Pass those. ```python import os, tempfile, weakref class Scratch: def __init__(self): fd, self.path = tempfile.mkstemp() self._finalizer = weakref.finalize(self, os.close, fd) ``` Note what is safe here. Storing the finalizer *on* the instance is fine: the arrow points from the object to the finalizer, and the finalizer holds only a weak reference back, so there is no cycle and no pinning. `os.close` is a module-level function that knows nothing about `Scratch`. The captured `fd` is an `int`. When the cleanup is more than one call, use a module-level function or a `staticmethod` that takes exactly the pieces it needs, and pass those pieces. If you find yourself wanting `self` in the body, that is the signal that the cleanup logic should be split: the part that touches the resource moves out of the class, and the part that touches the object's state cannot run at finalization time anyway. ## How to catch it It is directly testable, which is why interviewers like it. Drop the last strong reference, force a collection, and assert the finalizer is dead: ```python obj = MyThing() f = weakref.finalize(obj, cleanup, obj.resource_id) del obj gc.collect() assert not f.alive ``` A regression test of that shape around any class that registers a finalizer costs three lines and catches the day someone "simplifies" the registration back to a bound method. `weakref.ref(obj)` plus `gc.collect()` gives the same assertion for objects you did not register anything on. ## The mirror-image mistake The opposite error is capturing something the callback needs and then having it die first. Anything the callback touches at exit time — a module global, an open stream, a logger — may already be torn down when the exit pass runs. Keep finalizer callbacks tiny, dependency-free, and tolerant of being called into a shutting-down interpreter.

  • Is it safe to store the finalizer object on the instance it finalizes?
    Yes, and it is the idiomatic pattern. The instance holds a strong reference to the finalizer, but the finalizer holds only a weak reference back, so there is no cycle and nothing is pinned. Finalizer objects carry no per-instance state of their own — their data lives in the module registry — so keeping one as an attribute costs almost nothing and gives you `.alive`, `detach()` and early invocation.
  • How would you write a test that proves a class's finalizer is not pinning its instance?
    Create the object, register or let the constructor register the finalizer, drop the last strong reference, call `gc.collect()`, and assert the finalizer is no longer `alive` — or hold a `weakref.ref` to the object and assert it returns `None`. It is a three-line test and it fails loudly the first time someone reintroduces a bound method or a lambda that closes over `self`.
  • Does using functools.partial instead of a bound method avoid the problem?
    Only if the partial does not carry the referent. `functools.partial(cleanup, obj)` is exactly as fatal as a bound method — it stores `obj` in its `args`. The tool is irrelevant; what matters is whether any object reachable from `func`, `args` or `kwargs` can reach the referent. A partial over a path or a descriptor is fine; a partial over the owner is not.

It is like leaving a forwarding address that is the house you are trying to move out of: the instruction to clean up can never take effect, because the instruction itself is what keeps you there.

saying these in an interview costs you the question

  • Thinks a bound method holds no reference to its instance
  • Says the callback never runs, missing the exit pass
  • Blames the garbage collector rather than the strong reference
  • Swaps the bound method for a lambda over the same object
  • Believes storing the finalizer on the instance causes the leak
  • Passes the object itself so the callback can inspect it

context