Why does `functools.lru_cache` on a method keep its instances alive?

seniorshould knowfreq 35%

answer

The cache outlives the instance
One cache per function, not per instance
The instance is part of the key
A strong reference pins every cached object

basics

~20 s

The decorator is applied once, to the plain function, so one cache is shared by the whole class. The instance arrives as part of the cache key, and the cache holds it by a strong reference — so every instance ever scored stays reachable.

solid answer

~50 s

@functools.lru_cache above a method decorates the function in the class body, not anything per-instance, so a single cache lives on that one function object and is shared by every instance. Because the instance is the method's first argument, it becomes part of the cache key and is retained by a strong reference for as long as the entry survives — unbounded with functools.cache, or up to maxsize entries otherwise. Instances are therefore reachable from a class-level structure and are never collected, taking everything they hold with them. Two secondary hazards come along: the instance must be hashable, and a hit is decided by __eq__/__hash__, so a mutable instance can serve stale results. Fixes are a per-instance cache built in __init__, functools.cached_property for a zero-argument value, a module-level cached function keyed by an immutable identifier, or a weak-keyed mapping.

`@functools.lru_cache` above a method decorates the function in the class body, not anything per-instance, so a single cache lives on that one function object and is shared by every instance. Because the instance is the method's first argument, it becomes part of the cache key and is retained by a strong reference for as long as the entry survives — unbounded with `functools.cache`, or up to `maxsize` entries otherwise. Instances are therefore reachable from a class-level structure and are never collected, taking everything they hold with them. Two secondary hazards come along: the instance must be hashable, and a hit is decided by `__eq__`/`__hash__`, so a mutable instance can serve stale results. Fixes are a per-instance cache built in `__init__`, `functools.cached_property` for a zero-argument value, a module-level cached function keyed by an immutable identifier, or a weak-keyed mapping.

code

python · 22 lines

python

import functools
import gc
import weakref

class Bidder:
    def __init__(self, name):
        self.name = name

    @functools.lru_cache(maxsize=128)
    def score(self, slot):
        return len(self.name) + slot

b = Bidder("alpha")
probe = weakref.ref(b)
b.score(17)
del b
gc.collect()
print("alive after del:", probe() is not None)

Bidder.score.cache_clear()
gc.collect()
print("alive after clear:", probe() is not None)

go deeper

for a junior

Recall the headline rule: do not put a cache decorator on a method without thinking about lifetimes, because the object itself becomes part of the cache key and the cache is shared by the whole class.

for a middle

Explain the mechanics: one decorator application in the class body means one class-level cache, the instance is the first argument and therefore part of the key, and keys are strong references. Name a concrete fix such as building the cache per instance.

for a senior

Show the production loop. Describe the symptom, the weak-reference probe that proves the cache is the retainer, why a bounded size only caps the damage, and how you would re-key the cache on immutable values so the object never enters it.

for a principal

Own the policy. Decide where caches may live in a long-lived service, require every cache to have a declared owner and lifetime, and treat any cache keyed on a live object as retention that has to be justified rather than as a free latency win.

The scenario

An ad-auction bidder builds one bidder object per campaign, and each one carries a resolved 17-service dependency graph — pricing, targeting, budget, fraud scoring and the rest. Someone notices score() is recomputed constantly and adds @functools.lru_cache(maxsize=1024) above it. Latency improves. Resident memory then climbs in a straight line for as long as the process runs, and heap dumps show every bidder object ever created still alive, dependency graphs included.

Why it happens

The decorator is applied where it is written: in the class body, to the plain function. There is one function object for the whole class, so there is exactly one cache for the whole class, not one per instance. The instance is just the method's first positional argument, so it is part of the key of every entry — and a cache key is held by an ordinary strong reference. As long as an entry lives, the instance in its key lives.

That turns a class-level attribute into a root that keeps objects reachable. Reference counting will never free them, and the cycle collector will not either: these are not unreachable cycles, they are genuinely reachable. With functools.cache (or maxsize=None) nothing is ever evicted and the growth is unbounded. With a bounded maxsize the growth stops, but you still pin up to maxsize distinct instances plus everything each one owns — which for a bidder holding a 17-service graph is the expensive part, not the cached integers.

The two hazards that ride along

Hashability and equality. Keys are compared with __hash__ and __eq__. A class that defines __eq__ without __hash__ becomes unhashable, and the very first call raises TypeError: unhashable type. Worse in the other direction: a class with value-based equality makes two distinct instances share cache entries, and a mutable instance can be mutated after a value is cached and then keep getting the stale answer, because the key no longer reflects the state the value was computed from.

Key identity is exact. A cache hit requires equal keys, and equality is type-sensitive. If a country code reaches score() sometimes as str and sometimes as bytes — the classic encoding mismatch, where one call site decoded a payload and another did not — "US" and b"US" are different keys. You get half the hit rate you expected and twice the retained instances, and neither symptom points at the encoding bug.

Confirming it in a live process

The cheap confirmation is a weak reference: take a weakref.ref to an instance, drop your own reference, force a collection, and see whether the referent is still alive. If clearing the cache makes the weak reference die, the cache is your retainer. That is a two-minute check and it distinguishes this from every other retention story.

The fixes, in the order you should offer them

Per-instance cache. Build the cache in __init__ — self.score = functools.lru_cache(maxsize=128)(self._score). Now each object owns its cache, and the cache dies with the object. It does create a reference cycle (instance → cache → bound method → instance), which the cycle collector handles, so collection is deferred to a gc pass rather than immediate.
functools.cached_property when the value takes no arguments beyond the instance: it stores the result in the instance __dict__, so it is naturally per-instance and naturally short-lived.
Move the cache off the instance entirely. Make the cached callable a module-level function taking only the immutable inputs that actually determine the result — a campaign id and a slot, not the whole bidder. This is usually the right answer: the instance was never part of the computation's identity, only of its call syntax.
A weak-keyed mapping (weakref.WeakKeyDictionary keyed by the instance) when you truly want a shared, class-level cache that does not extend lifetimes. It requires weak-referenceable, hashable instances and costs you the LRU eviction policy.

The line to say in an interview

"Caching on a method caches the instance too. The decorator runs once for the class, the cache is class-level, and self is part of the key — so the cache's lifetime becomes every cached instance's lifetime. Put the cache on the instance, or key it on values instead of on the object."

**The scenario** An ad-auction bidder builds one bidder object per campaign, and each one carries a resolved 17-service dependency graph — pricing, targeting, budget, fraud scoring and the rest. Someone notices `score()` is recomputed constantly and adds `@functools.lru_cache(maxsize=1024)` above it. Latency improves. Resident memory then climbs in a straight line for as long as the process runs, and heap dumps show every bidder object ever created still alive, dependency graphs included. **Why it happens** The decorator is applied where it is written: in the class body, to the plain function. There is one function object for the whole class, so **there is exactly one cache for the whole class**, not one per instance. The instance is just the method's first positional argument, so it is part of the key of every entry — and a cache key is held by an ordinary strong reference. As long as an entry lives, the instance in its key lives. That turns a class-level attribute into a root that keeps objects reachable. Reference counting will never free them, and the cycle collector will not either: these are not unreachable cycles, they are genuinely reachable. With `functools.cache` (or `maxsize=None`) nothing is ever evicted and the growth is unbounded. With a bounded `maxsize` the growth stops, but you still pin up to `maxsize` distinct instances plus everything each one owns — which for a bidder holding a 17-service graph is the expensive part, not the cached integers. **The two hazards that ride along** *Hashability and equality.* Keys are compared with `__hash__` and `__eq__`. A class that defines `__eq__` without `__hash__` becomes unhashable, and the very first call raises `TypeError: unhashable type`. Worse in the other direction: a class with value-based equality makes two distinct instances share cache entries, and a mutable instance can be mutated after a value is cached and then keep getting the stale answer, because the key no longer reflects the state the value was computed from. *Key identity is exact.* A cache hit requires equal keys, and equality is type-sensitive. If a country code reaches `score()` sometimes as `str` and sometimes as `bytes` — the classic encoding mismatch, where one call site decoded a payload and another did not — `"US"` and `b"US"` are different keys. You get half the hit rate you expected and twice the retained instances, and neither symptom points at the encoding bug. **Confirming it in a live process** The cheap confirmation is a weak reference: take a `weakref.ref` to an instance, drop your own reference, force a collection, and see whether the referent is still alive. If clearing the cache makes the weak reference die, the cache is your retainer. That is a two-minute check and it distinguishes this from every other retention story. **The fixes, in the order you should offer them** 1. **Per-instance cache.** Build the cache in `__init__` — `self.score = functools.lru_cache(maxsize=128)(self._score)`. Now each object owns its cache, and the cache dies with the object. It does create a reference cycle (instance → cache → bound method → instance), which the cycle collector handles, so collection is deferred to a gc pass rather than immediate. 2. **`functools.cached_property`** when the value takes no arguments beyond the instance: it stores the result in the instance `__dict__`, so it is naturally per-instance and naturally short-lived. 3. **Move the cache off the instance entirely.** Make the cached callable a module-level function taking only the immutable inputs that actually determine the result — a campaign id and a slot, not the whole bidder. This is usually the right answer: the instance was never part of the computation's identity, only of its call syntax. 4. **A weak-keyed mapping** (`weakref.WeakKeyDictionary` keyed by the instance) when you truly want a shared, class-level cache that does not extend lifetimes. It requires weak-referenceable, hashable instances and costs you the LRU eviction policy. **The line to say in an interview** "Caching on a method caches the instance too. The decorator runs once for the class, the cache is class-level, and `self` is part of the key — so the cache's lifetime becomes every cached instance's lifetime. Put the cache on the instance, or key it on values instead of on the object."

Does a bounded `maxsize` solve the retention problem?
It bounds it, it does not solve it. You still pin up to `maxsize` distinct instances, plus everything each one holds — and for an object graph of any size the retained payload dwarfs the cached values. Bounding turns unbounded growth into a fixed high-water mark, which may be acceptable, but the instances are still alive.
What breaks if the class defines `__eq__` for value equality?
Two things. Defining `__eq__` without `__hash__` makes instances unhashable and the first cached call raises `TypeError`. Defining both makes distinct instances share entries, so a mutation after a value is cached leaves the key unchanged and the stale value keeps being served.
Two call sites pass the same country code, one as `str` and one as `bytes`. What does the cache do?
It stores two entries. Keys are compared by equality and `"US" != b"US"`, so the encoding mismatch halves the hit rate and doubles the number of pinned instances. The symptom looks like a cache-sizing problem and is actually a decode-boundary bug at one of the call sites.
How would you confirm the cache is the retainer rather than something else?
Hold a `weakref.ref` to a suspect instance, drop every other reference you control, force a collection, and check whether the referent is still alive. Then clear the cache and collect again — if the weak reference dies at that point, the cache was the root keeping it reachable.

saying these in an interview costs you the question

Thinks each instance gets its own cache
Says the cycle collector will eventually free the instances
Believes a bounded maxsize removes the retention entirely
Claims the cache holds keys weakly
Ignores that a mutable instance can produce stale hits
Proposes only raising maxsize when hit rates disappoint

Why does `functools.lru_cache` on a method keep its instances alive?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

▾common follow-ups4 questions

▾red flags6 flags

§ context

▾related questions6 questions

▾related topics4 topics

answer

go deeper

context