skip to content

A Laravel Reverb server for a collaborative whiteboard stops accepting connections at about a thousand users; what limits cause this, and how do you raise them?

level: seniorimportance: should knowfreq 22%

answer

  1. every connection is a file descriptor
  2. stream_select caps near 1,024
  3. ext-uv via pecl install uv
  4. ulimit -n and limits.conf
  5. Supervisor minfds, Nginx nofile

basics

~20 s

Each WebSocket is an open file. Reverb's default ReactPHP loop uses stream_select, capped around 1,024 files, so install ext-uv, which Reverb uses automatically. Then raise the open-file limits everywhere: ulimit/limits.conf, Supervisor's minfds and Nginx's worker_rlimit_nofile and worker_connections.

solid answer

~40 s

A held WebSocket connection is a file descriptor in the Reverb process, so the ceiling is whichever file limit is lowest. The first one near 1,000 is the **event loop**: Reverb runs on ReactPHP, whose default loop uses PHP's `stream_select`, typically limited to 1,024 open files. Installing the **`ext-uv`** extension (`pecl install uv`) makes Reverb switch to a libuv loop automatically. Next come the **OS limits**: check `ulimit -n` for the user Reverb runs as and raise `nofile` in `/etc/security/limits.conf`. If **Supervisor** runs Reverb, raise its `minfds`. If **Nginx** proxies the sockets, it holds a descriptor per client too, so raise `worker_rlimit_nofile` and `worker_connections`. Past one machine's limits, scale Reverb horizontally.

code

ini · 3 lines
ini
# /etc/security/limits.conf
reverb        soft  nofile  10000
reverb        hard  nofile  10000

go deeper

for a junior

Recall that every WebSocket connection is an open file and that the operating system limits open files per process.

for a middle

Explain why the default stream_select loop caps Reverb near 1,024 connections and how ext-uv removes that cap.

for a senior

Raise every layer's limit in the right order, verify it applies to the daemon rather than a shell, and know when to scale out instead.

for a principal

Plan connection capacity per host against CPU and memory, choose when horizontal scaling becomes cheaper than larger machines, and budget the operational load.

## Why a connection count hits a wall A WebSocket server holds every connection open for as long as the user keeps the page. On Unix-like systems each open connection is a **file descriptor**, and several layers cap how many descriptors a process may hold. The observed ceiling is the lowest of them, and several of the defaults sit around 1,024, which is why a Reverb deployment often stalls at "about a thousand users" of a whiteboard. ## The layers, from the inside out 1. **The event loop.** Reverb is built on a ReactPHP event loop. Without extra extensions that loop uses PHP's `stream_select`, which is typically limited to **1,024** open files. Reverb's documentation draws the line at roughly 1,000 concurrent connections. 2. **The process's open-file limit.** The OS gives each process a soft and hard `nofile` limit, shown by `ulimit -n` for the user. 3. **The process manager.** Supervisor, if it runs Reverb, has its own `minfds` setting for the files it must be able to open. 4. **The reverse proxy.** Nginx keeps a client-side and an upstream descriptor per proxied connection and caps them with `worker_rlimit_nofile` and `worker_connections`. | Layer | Symptom when it is the limit | Fix | |---|---|---| | `stream_select` loop | failures near 1,024 even with high ulimits | install `ext-uv` | | OS `nofile` | "too many open files" errors in Reverb's output | raise limits for the Reverb user | | Supervisor | Reverb inherits a low limit when started by Supervisor | raise `minfds` | | Nginx | the proxy refuses or drops new upgrades | raise `worker_rlimit_nofile` and `worker_connections` | ## Switching the event loop Install the libuv binding for PHP: ```bash pecl install uv ``` Once the `uv` extension is loaded in the CLI PHP that runs `reverb:start`, Reverb uses an `ext-uv` powered loop on its own; there is no configuration flag to set. Remember that the CLI and web PHP often have separate `php.ini` files, so check with `php -m` from the same binary that Supervisor starts. ## Raising the OS and manager limits - **OS**: add `nofile` lines for the user in `/etc/security/limits.conf`, for example a soft and hard limit of 10,000, and confirm with `ulimit -n` in a fresh session. - **Supervisor**: set `minfds=10000` in the `[supervisord]` section, then restart Supervisor itself, not just the program. - **Nginx**: set `worker_rlimit_nofile 10000;` at the top level and `worker_connections 10000;` in the `events` block. Each change is necessary but none is sufficient alone: raising `ulimit` to 100,000 does nothing while the loop is still `stream_select`, and installing `ext-uv` does nothing while Supervisor starts Reverb with a limit of 1,024. ## Other ceilings to know - A single Reverb process runs on one CPU core; very busy whiteboards with many messages per connection can saturate it before the connection limits are reached. - When a proxy on the same host connects to Reverb, each proxied connection also uses a local port, so the ephemeral port range can become a limit at very high counts. - Per-app `max_connections` in `config/reverb.php` (unset by default) is a deliberate cap you may want as a safety valve. ## When one machine is not enough Past what one host can hold, Reverb scales **horizontally**: several servers behind a load balancer, sharing messages through Redis when `REVERB_SCALING_ENABLED=true`. Raising the per-machine limits first is still worth it, because it decides how many servers you need. ## Monitoring the fix Reverb can report connection and message counts to Pulse through its recorders, which makes it easy to confirm that a change actually moved the ceiling rather than guessing from user complaints.

  • You installed ext-uv and raised ulimit, but connections still fail near 1,024; what do you check next?
    Whether the PHP binary that Supervisor starts actually loads `uv` (`php -m` from that binary), and whether Supervisor's own `minfds` or the Reverb user's limits apply to the process, since limits set for an interactive shell do not reach a daemon. Then check Nginx's `worker_rlimit_nofile` and `worker_connections`.
  • Why does Nginx need a higher file limit than the number of users?
    For every proxied WebSocket, Nginx holds the client connection and a separate upstream connection to Reverb, so each user costs it at least two descriptors, plus whatever normal HTTP traffic the same workers serve.

It is like a restaurant that seats guests by the number of chairs, tables and staff: adding chairs (ulimit) does not help while the host's seating chart (stream_select) only has 1,024 slots, and a bigger chart does not help if the fire marshal (Supervisor or Nginx) still caps the room.

saying these in an interview costs you the question

  • Raising ulimit alone lets a default Reverb install hold tens of thousands of connections.
  • Reverb needs a config flag to start using ext-uv after installing it.
  • Supervisor always passes the shell's ulimit to the processes it starts.
  • The 1,024 ceiling comes from a limit inside Laravel's broadcasting config.
  • Each WebSocket costs Nginx exactly one file descriptor.