skip to content

What does functools.lru_cache do to a function, and how does functools.cache differ?

level: juniorimportance: must knowfreq 70%

answer

  1. A decorator that remembers past calls
  2. Arguments in, stored result out
  3. A cap, and the least recent goes first
  4. cache means maxsize=None
  5. cache_info() reports hits and misses

basics

~20 s

functools.lru_cache stores each call's result in a dictionary keyed by the arguments, so a repeat call returns the stored value without running the body. functools.cache, added in 3.9, is lru_cache(maxsize=None): unbounded, and it never evicts.

solid answer

~40 s

`@functools.lru_cache` replaces a function with a wrapper holding a dictionary from the call's arguments to that call's result. The first call runs the body and stores the value; later calls with the same arguments return the stored value. `maxsize` caps the number of entries, and when the table is full the least recently used entry is discarded. `functools.cache`, added in 3.9, is exactly `lru_cache(maxsize=None)`: unbounded, no eviction, and marginally faster because it skips the recency bookkeeping. Both wrappers expose `cache_info()`, which reports hits, misses, the configured maxsize and the current size, and `cache_clear()`, which drops everything. Memoization is only correct for a function that is deterministic in the arguments it is keyed on and free of side effects worth repeating: the stored value never notices that the world changed underneath it.

code

python · 17 lines
python
import functools

calls = []

@functools.lru_cache(maxsize=2)
def slow_square(n):
    calls.append(n)
    return n * n

slow_square(2)
slow_square(3)
slow_square(2)
print(calls)                  # [2, 3] - the third call was a hit
print(slow_square.cache_info())

slow_square(4)                # table is full, 3 was least recently used
print(slow_square(3), calls)  # 9 [2, 3, 4, 3] - 3 had to be recomputed

go deeper

for a junior

Be ready to define memoization in one sentence and to name the decorator that gives it to you for free. Knowing that functools.cache is the unbounded variant of functools.lru_cache is enough at this level.

for a middle

Explain how the key is built, what maxsize caps and which entry eviction picks, and show yourself reading cache_info() to judge whether the cache is earning the memory it holds.

for a senior

An interviewer expects you to say which functions are safe to memoize in a long-lived process - deterministic, cheap to key, bounded argument space - and how you would bound or clear the cache in production.

for a principal

Own whether an in-process cache belongs in the design at all, weighed against recomputation, precomputation at startup, or a cache other processes can see and invalidate.

## What the decorator actually builds `functools.lru_cache` is a decorator factory. Applied to a function, it returns a wrapper object that owns a dictionary. Each call builds a key from the arguments, looks the key up, and either returns the stored result (a *hit*) or calls the original function, stores the result and returns it (a *miss*). The original function is untouched and still reachable through the wrapper, which is how documentation and testing tools recover the real signature. There are two forms. `@functools.lru_cache` applied bare uses the default `maxsize=128`; since 3.8 the bare form is legal, and before that the parentheses were mandatory. `@functools.lru_cache(maxsize=1024)` sets the cap explicitly. `maxsize=None` removes the cap entirely, and `functools.cache`, added in 3.9, is a name for exactly that: `cache` and `lru_cache(maxsize=None)` are the same behaviour, with `cache` reading better at the call site. ## What LRU means here LRU is *least recently used*. The wrapper tracks the order in which keys were touched. When the table already holds `maxsize` entries and a new key arrives, the entry that has gone longest without being read is discarded to make room. That is the only reason an entry ever leaves the table: there is no timeout, no size-in-bytes budget, no pressure signal from the allocator, and no notification when the data the function read has changed. An entry that is read often survives forever; an entry that is never read again is evicted only once something else needs its slot. With `maxsize=None` there is no recency bookkeeping to do at all, which is why `cache` is slightly cheaper per call than a bounded cache — and why it can only ever grow. ## Reading the counters Both wrappers expose `cache_info()`, returning a named tuple with four fields: `hits`, `misses`, `maxsize` and `currsize`. That tuple is the whole observability story for this decorator, and it is worth reading in a running process rather than once in a REPL. A hit ratio near zero says the arguments essentially never repeat, so the cache is buying nothing and paying memory for the privilege. A high ratio with `currsize` pinned at `maxsize` says the working set is larger than the cap and raising it may pay. `cache_clear()` empties the table and resets the counters; it is the only invalidation the decorator offers. ## When memoization is correct The cache answers from the past, so the function must be one whose past answers stay right. Three conditions matter. First, determinism in the keyed arguments: given the same arguments the function must produce the same result, which rules out anything reading a clock, a random source, a mutable global or a database. Second, side effects: if the body writes a log line, increments a counter or sends a message, memoizing silently drops those effects on every hit. Third, a bounded space of distinct arguments, or a `maxsize` chosen deliberately, because every distinct key that arrives is one more entry held by strong reference — both the arguments and the result stay alive as long as the entry does. ## Where it fits The decorator's virtue is that it is one line and needs no plumbing, which makes it the right reach for a pure, cheap-to-key computation whose inputs repeat: parsing a configuration string, resolving a lookup table, normalizing a small value in a hot path. Its vice is that it is invisible from the call site, process-local and un-invalidatable except wholesale. That is a fair trade for a pure helper and a poor one for anything that reads changing state. A quick mental rule: reach for `functools.cache` when the set of possible arguments is small and fixed, reach for `functools.lru_cache(maxsize=N)` when the arguments repeat but the space is open-ended, and reach for neither when the answer can go stale. ## One more property worth knowing The wrapper's own bookkeeping is safe to use from several threads, but it does not serialize the wrapped call. If two threads miss on the same key at the same moment, both run the function and one result simply overwrites the other. That is harmless for a pure computation and wrong if you were counting on the work happening exactly once.

  • What does cache_info() return, and how would you use it to choose a maxsize?
    A named tuple of hits, misses, maxsize and currsize. Read it from a running process, not once at startup. A hit ratio near zero means the arguments rarely repeat and the cache is pure cost. A high ratio with currsize sitting at maxsize means the working set exceeds the cap, so raising it may pay. A currsize that never approaches maxsize means the cap is not the constraint and the memory is bounded by the key space instead.
  • When is functools.cache the wrong choice even though the function is expensive?
    When the result can go stale, when the variety of arguments is unbounded, or when the arguments and results are large objects the cache will then keep alive indefinitely. It never evicts and holds strong references, so an open-ended key space turns it into steady memory growth. A bounded lru_cache with a deliberate maxsize, or an explicit cache with expiry, fits those cases better.
  • Is the undecorated function still reachable after applying functools.lru_cache?
    Yes. The wrapper copies the original function's metadata and keeps a reference to the original callable, so tests can call the uncached version deliberately and introspection tools can report the real signature rather than the wrapper's. That reference is also why the decorator does not lose the docstring or the qualified name.

It is a receptionist keeping a notepad of answers she has already looked up: the same question gets the same answer with no second trip to the archive, and the notepad never learns that the archive was updated.

saying these in an interview costs you the question

  • Says functools.cache and functools.lru_cache are unrelated tools
  • Thinks cached entries expire on a timer
  • Believes the cache notices when the underlying data changes
  • Claims maxsize=None still evicts under memory pressure
  • Memoizes a function whose side effects matter

context

resolving links…