A team proposes replacing a durable partitioned message log with Redis Streams. What does Redis Streams give up compared with a disk-backed replicated log, and where is it genuinely the better choice?

## Frame the comparison honestly Redis Streams borrowed the shape of an append-only log: immutable ordered entries, reader cursors instead of destructive reads, consumer groups with acknowledgement. That similarity is exactly why the substitution gets proposed. The differences are not in the API shape but in the **storage and replication substrate underneath it**. ## What Streams give up **1. Durability semantics.** A Redis primary replies to `XADD` as soon as the entry is in memory. Persistence happens on its own schedule - RDB snapshots periodically, AOF with `appendfsync everysec` by default, which means a crash can lose roughly the last second of writes; `appendfsync always` closes most of that at a large throughput cost. Replication is asynchronous: replicas receive the entry after the client has already been told it succeeded. If the primary fails and a replica is promoted, entries the producer believes are committed can be missing. `WAIT numreplicas timeout` lets a producer block until N replicas have acknowledged, which narrows the window, but it is a best-effort barrier rather than a quorum commit and it costs a round trip per write. A durable log designed for the job writes to disk on multiple brokers before acknowledging, and its ack policy is a first-class producer setting. **2. Retention.** Stream entries occupy RAM on the shard that owns the key. You bound that with `XADD ... MAXLEN ~ 1000000` or `MINID`, or periodic `XTRIM`. Two consequences: your usable history is measured in what you are willing to hold in memory, and **trimming does not consider consumer progress** - if a group is down longer than your retention window, its unread entries are simply gone. A disk-first log routinely retains days or weeks because disk is cheap, and can be configured to retain by time. **3. Partitioning of a single topic.** A stream is one Redis key. In Redis Cluster a key lives entirely on one shard, so one stream cannot be spread across nodes: its write throughput, its memory and its history are bounded by one node's event loop and RAM. You can shard manually - `events:0` … `events:15`, chosen by a hash of the entity id, each read by its own consumers - but then *you* own partition assignment, rebalancing when consumers come and go, and per-partition cursors. A partitioned log makes that the built-in model, including consumer-group rebalancing and per-key ordering by construction. **4. Ordering guarantees you may assume you have.** Within one stream key, entry IDs are strictly increasing and the single-threaded server serializes appends, so total order per key is solid. But inside a consumer group, entries are handed to whichever consumer asks next: two events for the same entity can be processed concurrently by different consumers. Per-key ordering is not provided; you must partition to get it. **5. Ecosystem features.** No compaction (keeping only the latest entry per key), no schema registry, no exactly-once transactional produce-consume, no ready-made connectors or stream-processing runtime. Redis gives you the primitive and stops. ## Where Streams genuinely win **Latency.** Everything is in memory and the protocol is one round trip. `XADD` to `XREADGROUP BLOCK` delivery is sub-millisecond on a local network - materially faster than a system designed around disk batching, and often the actual product requirement for task dispatch or live coordination. **Operational cost.** If Redis is already deployed, Streams are a data type, not a new distributed system to run, monitor, upgrade and staff. That is frequently the dominant factor for a small team, and the honest form of the argument is not technical purity but total cost. **Simplicity of the consumer model.** No rebalance protocol to understand: a consumer is a name, appearing and disappearing freely, with pending entries claimable by any peer. For a job queue this is easier to reason about than partition assignment. **Right-sized problems.** Work queues, task fan-out to workers, short-lived event buffers, per-user activity feeds with capped length, real-time coordination where minutes of history is all that matters, and anything where the volume comfortably fits one shard's RAM and one core's throughput. ## How to answer the team Ask what happens when a message is lost, and how far back a consumer must be able to replay. If the answer is "lost is a bug we must never have" and "days", Redis Streams are the wrong substrate and the honest fix is either a durable log or an outbox in a database. If the answer is "a lost task is retried by an upstream timeout" and "minutes", Streams are likely the cheaper, faster, simpler choice - and the one already in your stack. Then set retention deliberately against your slowest consumer's worst outage, monitor group lag with `XINFO GROUPS` and pending backlogs with `XPENDING`, and decide up front whether you need manual sharding for per-key ordering.