Why would a WeakValueDictionary cache in a billing run show a near-zero hit rate?

This is the most common production disappointment with weak references, and it usually surfaces in a batch job. Picture a subscription-billing run whose loader memoises customer records in a mapping shared across calls as a default argument: ```python import weakref _CACHE = weakref.WeakValueDictionary() def load_customer(cid, cache=_CACHE): hit = cache.get(cid) if hit is not None: return hit record = fetch_customer(cid) cache[cid] = record return record ``` It reviews cleanly, it shipped in a three-week release train, and the hit-rate metric added afterwards reads about one percent. ### Why it misses A `weakref.WeakValueDictionary` holds its values weakly by definition: an entry exists only while some *other* part of the process holds a strong reference to that value. The caller's loop is: ```python for cid in due_today: customer = load_customer(cid) invoice(customer) ``` The single strong reference is the local name `customer`. On the next iteration that name is rebound, the refcount drops to zero, the record is deallocated and the mapping quietly removes the entry. By the time the same customer id comes round — on a retry, on a second pass, in the next batch — the entry is long gone. The cache is not broken; it is a cache with no retention policy of its own, and the program supplied no owner. Note the mutable default argument is a red herring here in the usual sense — sharing one mapping across calls is exactly what a memo cache wants — but it is worth naming in review, because a shared mutable default is also how a *strong* dict in the same position becomes an unbounded leak that nobody notices until RSS climbs over the run. ### Confirming it rather than guessing The diagnosis is cheap and does not need a profiler. Count insertions and compare with `len(cache)` at intervals: an insert count in the millions against a length that hovers in single digits is conclusive. A second confirmation is to bind one known record to a module-level name and watch the hit rate for that one id go to a hundred percent while every other id still misses. If you want to see the eviction as it happens, register a callback on a `weakref.ref` to one record and log when it fires — you will see it fire inside the loop, not at the end of the run. ### The fix is an ownership decision There are three honest resolutions, and choosing between them is the actual interview content. **Give the working set a strong owner.** Keep a small, explicitly bounded structure — held by the run itself — containing the records currently in play, and let the weak mapping remain as the identity index. Now a lookup hits while the record is in the working set, and the weak mapping guarantees that two parts of the run handling the same customer share one object rather than building two. This is the arrangement weak values were designed for: the index never extends anyone's life, the owner decides lifetime. **Restructure so the object is genuinely still in use.** If the run processes each customer exactly once, there is nothing to cache and the cache should be deleted. A hit rate near zero sometimes means the workload has no reuse, and the honest response is to stop paying for a lookup that never pays back. **Switch containers.** If entries must survive with no strong owner, you want retention, and retention is a plain `dict` plus a deliberate eviction rule. A weak-valued mapping cannot express "keep this for a while" because it has no notion of a while. ### The mirror-image failure Worth naming, because interviewers often follow up with it: the same design fails in the opposite direction when the values are not as unowned as you think. Add a callback list, an audit trail, or an in-flight index that holds each record strongly, and the weak mapping now never drops anything — the hit rate goes to a hundred percent and memory grows without bound for the whole run. Both failure modes have the same root cause: nobody wrote down who owns the object. A weak mapping does not answer that question; it makes the absence of an answer visible.