What are the pitfalls of building a cache out of SoftReferences, and why are bounded caches like Caffeine usually preferred?

## Why people reach for soft-reference caches The appeal is simple: wrap each cached value in a `SoftReference`, drop it in a `Map`, and let the **garbage collector (GC)** decide when memory is tight enough to evict. No size tuning, no eviction policy to write — the JVM 'handles it.' For a quick memoization that's acceptable. As a production cache it has several real problems. ## Pitfall 1: non-deterministic, coarse clearing (the cliff) The GC clears softly-reachable referents only under **memory pressure**, and when it does, it can clear a large fraction — sometimes *all* — of them in one collection to avoid an `OutOfMemoryError`. The result is a **latency cliff**: the cache is full and fast, then suddenly empty, so every request misses and recomputes simultaneously, driving CPU and allocation up *at the exact moment the heap is already stressed*. A bounded cache evicts steadily, one entry at a time, so behavior degrades gracefully. ## Pitfall 2: no size, no TTL, no smart eviction Soft references give you *one* lever — 'keep until memory is low' — tuned only globally by `-XX:SoftRefLRUPolicyMSPerMB`. You cannot say 'at most 10,000 entries,' 'expire after 5 minutes,' or 'keep the most frequently used.' All entries are treated by a rough recency heuristic; a critical hot entry can be cleared alongside cold ones. Real caches offer **maximumSize**, **expireAfterWrite/Access**, **weight-based** bounds, and frequency-aware admission (Caffeine's **W-TinyLFU**) that keeps high-value entries. ## Pitfall 3: the map leaks keys and empty references A `Map<K, SoftReference<V>>` softly holds only the *values*. The **keys** and the `SoftReference` wrapper objects are held *strongly* by the map. When the GC clears a value, the entry remains: a live key mapping to a `SoftReference` whose `get()` now returns `null`. Without active **reaping** — polling a `ReferenceQueue` or sweeping on access — these dead entries accumulate, so a 'self-cleaning' cache actually grows. You must write the cleanup the GC won't do for you. ## Pitfall 4: GC overhead and surprise retention Every `Reference` object must be **discovered and processed** by the collector during each GC; large numbers of soft references lengthen reference-processing time and can extend pauses. Worse, a soft referent that transitively holds a big object graph keeps *all of it* alive until cleared, so heap usage stays high longer than intuition suggests — and a heap dump can look like a leak when it is just deferred soft-ref clearing. ## Pitfall 5: weak interaction with modern collectors and sizing Because soft references survive across collections, they fight against keeping the heap small; on collectors tuned for low pause times, leaning on soft refs to size the cache couples your cache capacity to GC behavior you don't otherwise want to tune. Cache capacity should be an explicit product decision, not an emergent side effect of the collector's pressure heuristic. ## Why bounded caches win Libraries like **Caffeine** (the modern standard) and **Guava Cache** give: explicit size/weight/time eviction; frequency- and recency-aware replacement; asynchronous loading and refresh; and hit/miss/eviction **statistics** for tuning. Behavior is predictable and observable. Caffeine can even *optionally* combine bounded eviction with soft/weak values when you want a GC safety valve on top of a hard bound — the right way to use soft references is as a *backstop*, not the whole policy. ## When soft references are still fine A tiny memoization, a single large recomputable object you'd keep if you could, or a deliberate last-line OOM backstop under a bounded cache. For anything load-bearing, reach for a real cache and treat soft references as the low-level primitive they are.