Pub/Sub & Streams

Fire-and-forget, at-most-once. PUBLISH hands the message only to clients subscribed at that instant and stores nothing. A subscriber that is disconnected, still reconnecting, or killed for a full output buffer misses those messages permanently: no replay, no offsets, no acknowledgement.

Redis generates an ID of the form milliseconds-sequence, e.g. 1699999999999-0. The first part is the server's Unix time in milliseconds, the second a counter that increments for additional entries within the same millisecond. IDs are strictly increasing, so XADD rejects any ID not greater than the stream's last one.

Create the group once: XGROUP CREATE key group $ (use 0 to replay history, MKSTREAM if the stream may not exist). Each worker reads with XREADGROUP GROUP group consumer-name STREAMS key >. The > means entries never yet delivered to anyone in this group, so Redis gives each new entry to one consumer only.

The entry goes into the group's Pending Entries List (PEL): entry ID, owning consumer, last delivery time and delivery count. XACK key group id removes it from the PEL. Inspect with XPENDING (summary) or XPENDING key group IDLE ms start end count (per-entry detail).

Pub/Sub keeps nothing: it broadcasts once and forgets, so a worker that is restarting or crashes mid-job loses that job. A Redis Streams consumer group retains each entry, gives one consumer ownership of it, and keeps it pending and recoverable until XACK.

BLOCK ms parks the connection until an entry newer than the given ID arrives or the timeout expires (BLOCK 0 waits forever); on timeout it returns nil. $ resolves to the stream's current last ID at call time, so you get only entries added afterwards — safe for the first call only, then you must pass the last ID you actually received.

Nothing happens automatically: they stay in the group's pending list owned by the dead consumer's name forever. Another consumer must take ownership — XAUTOCLAIM key group new-consumer min-idle-time 0 (or XCLAIM on specific IDs found via XPENDING IDLE). The min-idle-time guard prevents stealing from a merely slow worker.

Pub/Sub broadcasts to every subscriber, so all twelve replicas process every event - it cannot load-balance within a service. Redis Streams give both shapes: create one consumer group per service (each group sees all entries) and run the four replicas as consumers inside that group so entries are split among them.

PSUBSCRIBE registers a glob pattern (* ? [ab]) instead of an exact channel name. When a publish matches, the client gets a pmessage push with four parts: the literal 'pmessage', the pattern that matched, the actual channel published to, and the payload.

Under RESP2 the connection enters subscriber mode: it may only run SUBSCRIBE/UNSUBSCRIBE (and their pattern and sharded variants), plus PING, RESET and QUIT. Other commands error. Leave by unsubscribing from everything, or with RESET. Under RESP3 the restriction is gone.

Two: one message push from the exact subscription and one pmessage push from the pattern. Redis delivers once per matching subscription and never de-duplicates per client, so overlapping subscriptions mean duplicate payloads the application must handle.

Ask for COUNT+ entries from a start ID, then make the next page start just after the last ID you got. Since XRANGE boundaries are inclusive, either use the exclusive prefix — XRANGE key (lastId + COUNT n — or increment the sequence part of that ID yourself.

PUBLISH is roughly O(N+M): N subscribers on the exact channel (a hash lookup plus a write each) and M registered patterns, every one glob-matched against the channel name on every publish. That matching runs on the same single command loop everyone else waits on.

Classic PUBLISH is broadcast to every node over the cluster bus so any subscriber anywhere receives it, which makes pub/sub traffic grow with cluster size and never shard. Sharded pub/sub (Redis 7.0) hashes the channel name to a slot: SPUBLISH goes only to that shard's primary and replicas, and SSUBSCRIBE must target that node.

Redis buffers undelivered messages per client, and when the pub/sub client-output-buffer-limit is crossed (default hard 32mb, soft 8mb for 60s) it closes that connection. The subscriber sees a dropped connection, reconnects, resubscribes, and silently loses everything queued. Publishers see nothing.

Nothing is removed by reading, so you trim explicitly: XTRIM key MAXLEN n (or MINID id), or the same options inline on XADD. The tilde makes trimming approximate — Redis removes whole internal macro-nodes and may leave a few extra entries, which is O(1)-ish instead of proportional to the entries removed.

Redis Streams keep history in RAM on one shard, replicate asynchronously, and trim by length or ID - so retention is short, an acknowledged write can be lost on failover, and one stream key cannot be partitioned across nodes. They win on latency, operational simplicity, and when Redis is already in the stack.

At-least-once: an entry is acknowledged only after the work runs, so a crash between the two causes redelivery. Design idempotent handlers keyed on the entry ID, claim stale entries with a min-idle-time above p99 job time, cap retries via the delivery counter, and route poison entries to your own dead-letter stream.

Ask who is on the other end of the contract and how long it lives. Team-internal, short-lived, team-operated flows stay in Redis; cross-team, long-lived, unenumerable consumers move to a durable log. Always require idempotent consumers, sized retention with lag alerting, a written durability expectation, and a dead-letter path.

Use PUNSUBSCRIBE with the exact pattern string, byte-for-byte. UNSUBSCRIBE only removes exact-channel subscriptions and never touches patterns, so the pmessage deliveries keep arriving. PUNSUBSCRIBE with no arguments removes every pattern on the connection.