A Redis Stream grows forever unless you cap it. How do XTRIM and the MAXLEN option on XADD work, and what does the tilde in MAXLEN ~ 1000 actually change?

## Why explicit trimming exists A Redis Stream is a persistent, append-only data structure in a key. Reads (`XRANGE`, `XREAD`, and group reads) are non-destructive — that is the point, since it allows many independent readers and replay. The corollary is that **the stream's memory only ever grows** until you intervene. An unbounded stream in an in-memory database is how teams meet their `maxmemory` policy the hard way, and unlike a cache key, a stream is usually one large key that eviction handles badly (evicting it drops *everything*, and it may be excluded from eviction if it has no TTL and your policy is `volatile-*`). So capping is a design decision you make when you create the stream, not a cleanup task for later. ## The two retention policies **By count — `MAXLEN n`.** Keep about the newest `n` entries. Simple, bounds memory directly, but the time window it covers varies with traffic: at 10 msg/s a 100k cap is ~3 hours; at 1000 msg/s it is ~100 seconds. If your requirement is "a consumer may be down for 30 minutes", a count cap silently violates it during a traffic spike. **By ID — `MINID id`** (Redis 6.2+). Remove entries with IDs lower than `id`. Because auto IDs are `<ms>-<seq>`, passing a millisecond timestamp gives real **time-based retention**: `XTRIM s MINID ~ <now_ms - 3600000>` keeps the last hour. This directly expresses "how far behind may a reader be", at the cost of an unbounded memory footprint if volume spikes. Production systems often apply both: `MAXLEN ~` inline on `XADD` as a hard memory ceiling, plus a periodic `MINID ~` job for the time policy. ## What `~` really does Internally a stream is a **radix tree whose leaves are macro-nodes**, each packing many entries in a compact listpack-style layout with shared field names. This is why streams are memory-efficient and why range scans are fast. Trimming removes entries from the head. With **exact** trimming (`=`, the default when no modifier is given), Redis must land on precisely `n` entries, which can require rewriting a macro-node that is only partly beyond the cutoff — real CPU work proportional to the entries removed. With **approximate** trimming (`~`), Redis stops as soon as removing the next whole macro-node would go past the limit. So you can end up with somewhat more than `n` entries — bounded by the node size (in practice on the order of a hundred entries, governed by `stream-node-max-entries` / `stream-node-max-bytes`) — but the operation is cheap and does not scan entry by entry. It **never** leaves fewer entries than requested; the error is always in the safe direction. Practical rule: **`~` on the hot path, `=` only when exactness is a requirement** (a test asserting length, or a strict fixed-size ring). Attaching an exact `MAXLEN` to every `XADD` on a high-throughput stream is a known way to turn a fast append into a latency spike, because that work happens on Redis's single execution thread. ## `LIMIT` `XADD`/`XTRIM ... MAXLEN ~ 1000 LIMIT 500` (Redis 6.2+) caps how many entries a single call may delete. It requires the `~` modifier — asking for exact trimming with a work limit is contradictory and Redis rejects it. `LIMIT 0` means no limit. This matters when you first apply a cap to a stream that has grown to millions of entries: without `LIMIT`, that one command deletes millions of entries in one shot while everything else waits. With `LIMIT`, you converge over several calls. Redis also applies an internal default limit (100 times the node size) when you use `~` without an explicit `LIMIT`, precisely to avoid such stalls. ## XDEL is not trimming `XDEL key id [id ...]` removes specific entries. It is for redaction or correcting a bad append, not for retention: it does not compact the underlying nodes the way trimming does (the entry is marked deleted within its node until the node is dropped), so it does not reliably reclaim memory, and it leaves gaps. `XLEN` decreases; `last-generated-id` does not move, so IDs are never reused and cursors stay valid. ## Trimming is blind to readers This is the one that bites in production: **trimming does not consider consumer progress.** There is no "retain until acknowledged" mode. A consumer that is down or lagging past the retention boundary simply resumes at the oldest surviving entry, having lost everything in between, with no error raised. Detection is on you: compare consumer cursors against the stream's first entry ID (`XINFO STREAM key` reports `first-entry` and `last-generated-id`), and alert when the gap shrinks toward zero. Sizing follows from that: retention must exceed worst-case consumer downtime — a deploy, a long GC pause, a poison-message stall, an on-call response time — with headroom for a traffic spike. Estimating memory is straightforward once you measure: `MEMORY USAGE key` divided by `XLEN` gives bytes per entry (streams share field names within a node, so wide entries with repeated field names are cheaper than they look), then multiply by your cap. ## Putting it together A typical event stream: `XADD s MAXLEN ~ 1000000 * ...` on every append for a hard ceiling, a cron issuing `XTRIM s MINID ~ <now-24h>` for the time policy, `LIMIT` on the first run against an already-huge stream, and a monitor comparing the oldest retained ID against each reader's cursor.