How do you use gc.get_referrers() to find what still holds a leaking object?

### The question this answers A type census tells you *what* is accumulating. It never tells you *who* is holding it, and that is the answer you actually need, because the fix lives at the holder. `gc.get_referrers(obj)` is the direct tool: given an object, it asks the collector which tracked objects contain a reference to it. A worked case makes the shape clear. A clinical-lab result loader runs as a long-lived service; RSS climbs steadily across a shift while throughput is flat, and the batch cache reports a healthy 83% hit rate. A census shows result instances growing linearly with samples processed. So take one instance and climb. ```python import gc gc.collect() leaked = next(o for o in gc.get_objects() if isinstance(o, LabResult)) holders = gc.get_referrers(leaked) print([type(h).__name__ for h in holders]) # ['dict', 'dict'] ``` ### Why the first level is almost always a dict Python objects rarely refer to each other directly. An instance attribute lives in that instance's `__dict__`; a cache entry lives in a mapping; a module-level name lives in the module's globals dict. So the referrer you get back is the *dict*, not the thing that owns the dict, and the name of the type tells you nothing. The move is to climb again: ```python for holder in holders: if isinstance(holder, dict): print([type(up).__name__ for up in gc.get_referrers(holder)]) ``` One or two more levels typically land on something identifiable: a `module` (a module-level registry), an instance of your own class (an attribute on a long-lived object), a `function` or a `cell` (a closure that captured the payload), or a `frame` (something is still on a stack). In the lab-loader case the chain ends at the module holding a memo dict that keyed results by sample id and never evicted -- the same dict responsible for the 83% hit rate, which is why nobody suspected it. The cache was doing its job; nobody had given it a ceiling. ### The noise, and how to control it `gc.get_referrers()` is honest to a fault: it reports *your* references too. - The frame you are running in refers to `leaked`, so a `frame` (or the module globals dict, at module level) appears. - Any list you built while searching -- the result of `gc.get_objects()`, a filtered list of candidates -- is a referrer. - The argument tuple of the call itself can appear. So: collect first, keep the search code in a small function so the noise is one identifiable frame rather than your whole session, compare by identity against the locals you know about, and `del` intermediate lists as soon as you are done with them. Holding those lists is not a theoretical concern -- they pin everything they contain and will distort the next census you take. ### The limits worth stating out loud **Only tracked referrers are visible.** The collector knows about container objects. An object reachable only from a slot on an untracked object, from a C-level structure inside a native extension, or from a thread-local in an extension module will show fewer referrers than the refcount implies -- sometimes none at all. Comparing `sys.getrefcount(obj)` with the number of referrers found is a cheap sanity check: a large gap says something invisible to the collector is holding it. **It is expensive.** Finding referrers means scanning the tracked heap, so treat it as a targeted probe on one or two objects, never a loop over thousands. **`gc.get_referents()` goes the other way.** Given a suspected holder, it lists what that object refers to directly. It is the tool for confirming a hypothesis ("does this cache still contain results from yesterday's batch?") and for writing a small breadth-first walk from a known root down to the payload, which is often easier to read than climbing up through anonymous dicts. ### Interpreting what you find The holder chain names the fix, and the fix is a policy at the holder, not a call to the collector: - module globals or a class attribute -> an unbounded registry; cap it, drain it per batch, or delete it; - a closure cell -> a callback, retry handler or partially applied function captured a large payload; capture the small key instead of the object; - a frame -> something is still on a stack, often a generator that was never exhausted or closed, or an exception being held; - a long-lived instance of your own class -> an attribute that outlives the request it belongs to. One more corroborating signal from the same case: because each cached result also held an open file object from the load path, the file-descriptor count climbed in lockstep with RSS. When a leak has a matching resource that was never closed, the descriptor or handle count is often the faster and cheaper alarm.