skip to content

A Redis subscriber consumes published messages more slowly than they are produced. What does the Redis server eventually do about it, which configuration governs that, and what does the application see?

level: seniorimportance: should knowfreq 36%

answer

  1. Buffer lives on the server, per client
  2. pubsub 32mb 8mb 60 defaults
  3. Hard = instant kill, soft = 60s sustained
  4. omem in CLIENT LIST, client_recent_max_output_buffer
  5. Silent: publisher fine, subscriber just reconnects

basics

~20 s

Redis buffers undelivered messages per client, and when the pub/sub client-output-buffer-limit is crossed (default hard 32mb, soft 8mb for 60s) it closes that connection. The subscriber sees a dropped connection, reconnects, resubscribes, and silently loses everything queued. Publishers see nothing.

solid answer

~50 s

Every connected client has a server-side output buffer. A subscriber that reads slowly makes its buffer grow, which costs the server memory and eventually triggers `client-output-buffer-limit pubsub 32mb 8mb 60`: exceed 32 MB at any instant, or stay above 8 MB for 60 seconds, and Redis closes that client. The failure is asymmetric and quiet. The publisher's PUBLISH succeeded. The subscriber just sees a closed socket; its library reconnects and resubscribes, so the process looks healthy while a block of messages has vanished. Meanwhile those buffers counted toward `used_memory`, so a big fan-out to slow consumers can push the instance toward its maxmemory and start evicting keys. Diagnose with `CLIENT LIST` (`omem`, `tot-mem`, `qbuf`), `INFO clients` (`client_recent_max_output_buffer`), and the server log line about a client scheduled to be closed for overcoming output buffer limits. Fix the consumer first: never do slow work in the message callback, shrink or batch payloads, spread subscribers across shards, and only then consider raising the limit.

code

text · 13 lines
text
> CONFIG GET client-output-buffer-limit
1) "client-output-buffer-limit"
2) "normal 0 0 0 slave 268435456 67108864 60 pubsub 33554432 8388608 60"

> CLIENT LIST TYPE pubsub
id=42 addr=10.0.0.7:51234 ... omem=9437184 tot-mem=9500000 cmd=subscribe

> INFO clients
connected_clients:214
client_recent_max_output_buffer:9437184

# raise only with a memory-headroom calculation behind it
> CONFIG SET client-output-buffer-limit "pubsub 67108864 16777216 60"

go deeper

for a junior

Know that Redis buffers messages for each subscriber and disconnects one that falls too far behind, losing those messages.

for a middle

Name the setting and its defaults, explain hard versus soft limit with the time window, and note that the publisher sees no failure.

for a senior

Drive the diagnosis: CLIENT LIST omem, INFO clients high-water mark, server log line, correlation with fan-out, then fix the consumer and payload before touching limits.

for a principal

Frame it as where the queue is allowed to accumulate: Redis deliberately caps in-server buffering and sheds the slow consumer, so decide whether your workload can tolerate that loss or needs a different, replayable transport.

## Where the backlog lives Redis does not apply backpressure to a publisher. `PUBLISH` writes the message into each subscriber's client output buffer and returns immediately; the socket is drained later by the event loop. If a subscriber reads slower than the publish rate, the difference accumulates in that buffer, in the server's memory, on the publisher's behalf. The producer is never slowed down and never told. ## The limit that ends it Because an unbounded buffer would take the whole instance down, Redis enforces per-class limits via `client-output-buffer-limit <class> <hard> <soft> <soft-seconds>`. The pub/sub class defaults to `32mb 8mb 60`: a client is closed immediately if its buffer exceeds the hard limit of 32 MB, or if it stays above the soft limit of 8 MB continuously for 60 seconds. Setting all three values to 0 disables the limit, which trades a lost subscriber for a possible out-of-memory on the server, rarely the better deal. The server logs the kill ("scheduled to be closed ASAP for overcoming of output buffer limits") and `INFO stats` counts it in `client_output_buffer_limit_disconnections`. ## What each side observes Publisher: nothing. PUBLISH returned a positive count for a client that was later killed, since the count reflects the write into the buffer, not delivery to the application. Subscriber: a connection reset. Virtually every client library reconnects and replays its SUBSCRIBE commands automatically, so unless you log reconnects, the incident is invisible and the only symptom is missing events downstream. This compounds the at-most-once nature of Pub/Sub: the messages queued in that buffer are gone, and there is no offset to resume from. Operator: memory. Output buffers are part of `used_memory`, so a large fan-out to lagging consumers inflates the instance's footprint and can trigger eviction of real keys, or make `maxmemory` behaviour look mysterious. ## Diagnosing `CLIENT LIST TYPE pubsub` shows per-client `omem` (output buffer memory) and `tot-mem`; a client with a growing `omem` is the culprit. `INFO clients` exposes `client_recent_max_output_buffer` as a cheap high-water signal to alert on, plus `pubsub_clients` (7.x) for the subscriber count. `INFO memory` will show the aggregate. Correlate a spike with your publish rate multiplied by subscriber count: fan-out is the amplifier, since one 100 KB publish to 200 subscribers is 20 MB of buffer. ## Fixes, in order Start with the consumer: the message callback must not do I/O, parse huge payloads, or take locks. Read the message, push it onto an in-process queue, return to the socket. Then shrink the payload: publish an identifier and let the consumer fetch the body, which also fits the "message as hint" rule that Pub/Sub's at-most-once semantics already forces on you. Then reduce fan-out per node, for example by splitting one busy channel into several so subscribers spread across shards. Raising `client-output-buffer-limit` is a legitimate last step when bursts are known and bounded, but it must be paired with a memory headroom calculation, because you are choosing to store more of the backlog in the server's RAM. If messages genuinely must not be dropped when a consumer lags, the honest conclusion is that Pub/Sub's fire-and-forget delivery does not match the requirement, and the design needs a stored, replayable structure instead.

  • Would raising the pub/sub output-buffer limit to unlimited (0 0 0) solve the problem?
    No, it moves the failure. With no limit the buffer grows until the instance hits maxmemory and starts evicting keys, or until the OS OOM-killer takes Redis down, which affects every client rather than one slow subscriber. The limit exists to contain a single bad consumer; the real fix is making the consumer read fast enough or publishing less data.
  • How would you tell this from an ordinary network problem when subscribers report missing messages?
    Look for the server log line about a client being closed for overcoming output buffer limits and the disconnection counter in INFO stats, and check whether omem or client_recent_max_output_buffer spiked just before the drop. A network fault usually hits many clients at once and leaves buffers small, while buffer eviction targets specific lagging subscribers and correlates with a publish-rate or payload-size increase.

saying these in an interview costs you the question

  • Assuming Redis applies backpressure to the publisher when a subscriber is slow
  • Thinking the subscriber gets an error explaining what was lost
  • Believing the messages are redelivered after the client reconnects
  • Treating the fix as "raise the limit" without considering server memory
  • Doing blocking work inside the subscriber callback and blaming the network

context