skip to content

What do the four numbers from functools.lru_cache's cache_info() tell you about a live cache?

level: middleimportance: should knowfreq 40%

answer

  1. Ask the wrapper what it has been doing
  2. Four counters, and none of them is bytes
  3. hits, misses, maxsize, currsize
  4. Ratio is hits over hits plus misses
  5. currsize pinned at maxsize means eviction churn

basics

~20 s

cache_info() reports hits, misses, maxsize and currsize since process start or the last clear. The hit ratio is hits over hits plus misses: near zero means the keys never repeat, and currsize stuck at maxsize means the cache is evicting.

solid answer

~50 s

`functools.lru_cache` replaces your function with a wrapper that counts its own traffic, and `cache_info()` returns those counters as a named tuple of `hits`, `misses`, `maxsize` and `currsize`, accumulated since the process started or the last `cache_clear()`. The number you derive is the hit ratio, `hits / (hits + misses)`. Near-zero hits with misses tracking the call count means the argument tuples never repeat, so the cache is pure overhead and should be removed, not enlarged. `currsize` pinned at `maxsize` with misses still climbing means the working set is bigger than the bound - that is the case where raising it helps. With `maxsize` of `None` nothing is ever evicted, so a steadily rising `currsize` is unbounded growth. Note that `currsize` counts entries, not bytes, and a lifetime ratio hides a recent regression - sample the counters and compare deltas.

code

python · 18 lines
python
from functools import lru_cache


@lru_cache(maxsize=2)
def score(user_id: int) -> int:
    return user_id * 7


for uid in [1, 2, 1, 3, 1]:
    score(uid)

info = score.cache_info()
print(info)
print("hit ratio:", info.hits / (info.hits + info.misses))
print("full:", info.currsize == info.maxsize)

score.cache_clear()
print(score.cache_info())

go deeper

for a junior

Be ready to name the four fields cache_info() gives you - hits, misses, maxsize, currsize - and to compute the hit ratio as hits divided by hits plus misses. Knowing that cache_clear() empties the cache is enough at this level.

for a middle

Explain the mechanics: the decorator returns a wrapper that owns the dictionary and the counters, the totals run since process start or the last clear, and maxsize of None means nothing is ever evicted so currsize only grows. Read each shape of the numbers and say what it implies.

for a senior

An interviewer expects you to diagnose from the counters on a running service: near-zero hits means remove the cache rather than enlarge it, currsize at maxsize with rising misses means the working set exceeds the bound, and lifetime ratios hide regressions so you sample deltas. Say out loud that entries are not bytes.

for a principal

Own the policy question: which caches are worth instrumenting at all, whether these per-process counters are exported alongside the service's other metrics, and how a team decides that a hit ratio justifies the resident memory. Per-process counters do not aggregate across workers, and that shapes what the dashboard can honestly claim.

### What the call actually returns A function decorated with `functools.lru_cache` is not the function you wrote: the decorator returns a *wrapper* object that owns a dictionary of results and forwards to your function on a miss. That wrapper carries its own instrumentation. Calling `cache_info()` on it returns a small named tuple with four integer fields: * **hits** - calls that were answered from the stored dictionary. * **misses** - calls that had to run the wrapped function. * **maxsize** - the bound you asked for, or `None` if you asked for no bound. * **currsize** - how many entries the dictionary holds right now. All four are cumulative since the process started, or since the last `cache_clear()`. They are counters on one wrapper in one process; nothing is aggregated across worker processes, and a restart resets them. ### Reading the numbers The number an interviewer wants you to derive is the **hit ratio**, `hits / (hits + misses)`. Guard the division: before the first call both are zero. Four shapes come up in real services. **A healthy cache.** A hit ratio comfortably above zero with `currsize` below `maxsize` means the whole working set fits and the cache is doing exactly what you asked. Memory is bounded by `currsize` entries and is not going anywhere. **A useless cache.** `hits` near zero while `misses` tracks the call count means the argument tuples almost never repeat - you are keying on a request id, a timestamp, or an object whose hash is its identity, so every call is a new key. This cache is pure cost: the memory for the entries, the hashing of every argument tuple, and an extra Python-level call on the hot path. The fix is to delete the decorator or change the key, never to raise `maxsize`. **Eviction churn.** `currsize` pinned at `maxsize` while `misses` keeps climbing steadily means the working set is larger than the bound, so entries are evicted before they are reused. Here raising `maxsize` genuinely helps - but it is a memory decision, and you should size it against measured entry cost, not guess. **Unbounded growth.** With `functools.cache`, or `lru_cache(maxsize=None)`, `maxsize` is `None` and nothing is ever evicted. Then `currsize` is not a diagnostic, it is the *alarm*: it is the number of distinct keys the process has ever seen, and on a long-running service it only goes up. `currsize` climbing without limit on a service that runs for days is the classic memoization leak. ### What the numbers do not tell you `currsize` counts **entries, not bytes**. Four counters cannot tell you whether the cache costs 40 KB or 4 GB; to value it you multiply entries by an estimate of one stored value, or measure the process. Nor does the hit ratio tell you time saved: a 99% hit rate on a function that costs 200 nanoseconds saves nothing measurable, while a 40% hit rate in front of a call that costs 50 milliseconds is a large win. Hit ratio measures *reuse*; you still have to pair it with the cost of one miss. The other trap is **lifetime totals**. A process up for a week with a 96% lifetime ratio can have been missing on everything since yesterday's deploy and the lifetime figure will barely move. If you export these counters, export `hits`, `misses` and `currsize` as raw values and compute the ratio from the **delta between two scrapes**, the way you would with any monotonic counter. ### `cache_clear()` `cache_clear()` drops every stored entry **and** resets `hits` and `misses` to zero. Two uses. In tests, a memoized function is process-global mutable state: a value computed under one test's configuration will happily be served to the next test, so clearing it in setup or teardown is what keeps test cases independent. In diagnosis, it lets you measure a *window* - clear, drive a known amount of traffic, read the counters - instead of squinting at lifetime totals. Be aware that on a method it clears entries for every instance at once, since there is one cache per decorated function, not per object. `functools.cached_property` has none of this: no counters, no clear. It stores its value in the instance's own attribute dictionary, so you inspect and invalidate it by looking at or deleting that attribute. ### In an interview Say the four field names, derive the ratio, and then say what you would *do* with each shape: near-zero ratio means remove the cache; `currsize` at `maxsize` with rising misses means raise the bound deliberately; `maxsize` of `None` with rising `currsize` means put a bound on it. That sequence - measure, interpret, act - is the whole answer.

  • Your cached function shows a 96% lifetime hit ratio, yet latency regressed after yesterday's deploy. How would you tell?
    A lifetime ratio is dominated by history, so a week of hits drowns a day of misses. Export `hits` and `misses` as raw monotonic counters, scrape them periodically, and compute the ratio from the difference between two scrapes. That windowed ratio would show the regression immediately. `cache_clear()` gives you the same effect manually: clear, drive known traffic, read the counters.
  • The hit ratio is 99%. Why is that not automatically a reason to keep the cache?
    Hit ratio measures reuse, not time saved. If the wrapped function costs a couple of hundred nanoseconds, the wrapper call and the argument hashing cost about as much as the work, so a 99% hit rate buys nothing while still holding memory. The number that decides it is the cost of one miss multiplied by the misses avoided, weighed against the entries retained.
  • Why call cache_clear() between test cases?
    A memoized function is process-global mutable state that outlives a single test. A value computed under one test's configuration or patched dependency is served straight to the next test, producing passes that depend on test order. Clearing the cache in setup or teardown restores independence. It resets the counters too, which is convenient if a test asserts on hits and misses.

The counters are the cache's utility meter: they tell you how much it consumed and how full it is, but not whether the appliance was worth buying - for that you still need the price of one miss.

saying these in an interview costs you the question

  • Thinks currsize reports the cache's memory footprint in bytes
  • Raises maxsize when the hit ratio is near zero
  • Treats a high hit ratio as proof the cache saves time
  • Believes cache_clear() drops entries but keeps the counters
  • Reads the lifetime ratio and misses a recent regression
  • Expects functools.cached_property to expose the same counters

context

resolving links…