With the allkeys-lfu maxmemory policy, Redis tracks each key's access frequency in only 8 bits. How can 8 bits represent millions of accesses, and what does the lfu-log-factor setting control?
answer
- 8-bit Morris counter, not a hit count
- new keys start at 5, protects fresh entries
- p = 1 / ((counter - 5) * log_factor + 1)
- saturates at 255; default factor 10 ~ 1M hits
- OBJECT FREQ, LFU policy only
basics
~20 sThe 8 bits hold a probabilistic logarithmic counter, not a hit count. Each access increments it only with probability decreasing as the counter grows, controlled by lfu-log-factor, so the counter saturates at 255 after millions of hits. OBJECT FREQ reads it.
solid answer
~50 sUnder LFU, Redis reuses the object header's 24 bits as a 16-bit last-decay time in minutes plus an 8-bit counter. A plain counter in 8 bits would saturate at 255 hits, useless for ranking hot keys, so Redis uses a Morris-style probabilistic counter. New keys start at 5, not 0, so a freshly inserted key is not evicted instantly before it can prove itself. On each access Redis increments with probability roughly `1 / ((counter - 5) * lfu-log-factor + 1)`: cheap increments while the counter is low, exponentially rarer as it climbs. With the default `lfu-log-factor 10`, reaching the ceiling of 255 takes on the order of a million accesses, so the counter is a compressed logarithmic view of popularity. A lower log factor makes the counter climb faster and discriminate better among low-traffic keys; a higher one stretches the scale so that very hot keys stay distinguishable. `OBJECT FREQ key` reports the raw counter.
code
text · 8 linesredis-cli CONFIG SET maxmemory-policy allkeys-lfu
redis-cli CONFIG SET lfu-log-factor 10
redis-cli SET hot:1 v
redis-cli OBJECT FREQ hot:1 # 5 - the initial value for a new key
for i in $(seq 1 10000); do redis-cli -x GET hot:1 </dev/null >/dev/null; done
redis-cli OBJECT FREQ hot:1 # climbed a little, far below 10000go deeper
Know that LFU ranks keys by how often they are used, that the counter is approximate, and that OBJECT FREQ shows it.
Explain the 8-bit probabilistic counter, the initial value of 5, saturation at 255, and that lfu-log-factor sets how many accesses map onto that scale.
Reason about tuning from observed counters, explain why LFU resists scan-shaped traffic that poisons LRU, and note that eviction machinery is shared with LRU apart from scoring.
Frame it as bounded-metadata probabilistic counting: constant memory per key traded for a logarithmic, tunable popularity rank, and state the workloads where that ranking beats recency.
## Why LFU exists at all LRU answers "who was touched longest ago", which is the wrong question for some caches. A key read once by a nightly report looks as fresh as a key read ten thousand times a second, so a full scan or a burst of one-off requests can evict genuinely hot data. LFU (Redis 4.0, policies `allkeys-lfu` and `volatile-lfu`) ranks by how often a key is used rather than how recently, which is more robust against scan-shaped traffic. ## The bit budget LFU has to fit in the same 24 spare bits of the object header that LRU used, because Redis will not pay extra memory per key. Those bits are split as: - 16 bits: the last decay time, in minutes. - 8 bits: the frequency counter, range 0 to 255. Eight bits obviously cannot count real accesses; a hot key sees millions. The trick is to stop counting exactly. ## The probabilistic (Morris) counter A Morris counter stores the logarithm of a count. Instead of incrementing on every event, it increments with a probability that falls as the stored value rises, so the stored number grows roughly like the logarithm of the true number of events. Redis implements it as follows: - New keys are created with the counter at 5 rather than 0. This is important: a key inserted a moment ago has no access history, and starting at 0 would make it the top eviction candidate immediately, so it would be evicted before it ever had a chance to be used again. - On access, Redis computes `baseval = counter - 5` (floored at 0) and increments the counter with probability `1 / (baseval * lfu_log_factor + 1)`. - The counter saturates at 255. So the first accesses after creation increment almost every time, and by the time the counter is in the hundreds each further increment requires thousands of accesses. With the default factor of 10, a key needs on the order of a million accesses to reach 255; with a much larger factor the same ceiling corresponds to roughly ten million; with a factor of 0 the counter behaves close to a linear count and saturates almost immediately, which destroys any ability to rank popular keys. This is also why the counter is *not* a hit count: an interviewer's favourite trap is asking what `OBJECT FREQ` returns. It returns the logarithmic counter, a rank, not a number of accesses. ## How eviction uses it Eviction machinery is shared with LRU: random sampling into the same bounded candidate pool. Only the score changes - instead of "most idle wins", the candidate with the *lowest* frequency counter wins, so it is scored as the inverse of the counter. Everything else - `maxmemory-samples`, the pool, the split between `allkeys-*` and `volatile-*` candidate sets, eviction happening inline in the command path - is identical. ## Reading and observing it `OBJECT FREQ <key>` is only valid while `maxmemory-policy` is one of the LFU policies, because outside that mode those bits hold the LRU clock instead. Conversely `OBJECT IDLETIME` is rejected under LFU. Sampling `OBJECT FREQ` over a set of interesting keys is the practical way to check whether the tuning actually separates your hot keys from your cold ones: if everything you look at reports 255, the scale is too compressed for your traffic and the log factor should go up; if hot and cold keys both sit near 5, the counter is not climbing and either the factor is too high for your rates or decay is eating the counter. ## What to say in an interview Name the three pieces: eight bits, probabilistic logarithmic increment, an initial value of 5 to protect new keys. Then the tuning consequence: `lfu-log-factor` sets how many real accesses map onto the 0-255 scale, so it decides at what traffic level the counter stops discriminating.
- Why are new keys created with a counter of 5 instead of 0?Because a brand-new key has no access history and would otherwise be the lowest-scoring candidate in the very next eviction round, so it could be evicted moments after being written. Starting at 5 gives it a grace band in which it can accumulate real accesses. It also anchors the increment probability formula, which subtracts that initial value before computing the odds.
- What does OBJECT FREQ actually return, and when is it valid?It returns the raw 8-bit logarithmic counter, a value between 0 and 255 that ranks popularity rather than counting hits. It is only valid while maxmemory-policy is an LFU policy, because under LRU those same header bits hold the recency clock and the number would be meaningless.
- How does LFU eviction differ mechanically from LRU eviction in Redis?Only in the scoring function. Both sample maxmemory-samples random keys from the policy's candidate dictionary, feed them into the same bounded candidate pool, and evict the best entry inline in the command path. LRU scores by estimated idle time, LFU by the inverse of the frequency counter.
Like a volume knob marked 0-10 for a range from a whisper to a jet engine: each further notch takes far more sound than the last, so you keep a usable ranking without a scale that runs off the panel.
saying these in an interview costs you the question
- Saying OBJECT FREQ returns the number of accesses to the key
- Assuming the counter increments on every access and therefore caps at 255 hits
- Forgetting that new keys start above zero and claiming fresh keys are evicted first by design
- Thinking LFU uses a different eviction loop rather than the same sampling and pool with a different score
- Believing a higher lfu-log-factor makes the counter climb faster