skip to content

When a Node.js service already runs as several container replicas behind a load balancer, why is adding the cluster module inside each container often redundant?

level: seniorimportance: should knowfreq 50%

answer

  1. two supervisors, one job
  2. what cluster actually adds
  3. crashed workers hidden from health checks
  4. availableParallelism inside a CPU limit
  5. one heap per forked worker

basics

~20 s

The orchestrator already runs, restarts and load-balances replicas. Cluster repeats that inside each container, hiding worker crashes from health checks, sizing from a CPU count the container may not get, and packing several V8 heaps into one memory limit.

solid answer

~40 s

`cluster` forks processes that share a port, with the primary accepting connections and handing them out round-robin by default outside Windows; restarting dead workers is code you write. Replicas behind a load balancer already give you multiple processes, distribution, restarts and rolling deploys. Running `cluster` inside each container duplicates that: a crashed worker is restarted, or silently missing, while the replica still passes health checks; worker counts taken from `os.availableParallelism()` come from a value libuv documents as the affinity mask, with no mention of CPU quotas, so they can exceed what a CPU-limited container may use; and each worker brings its own V8 heap into one memory limit. The usual default is one Node.js process per container and more replicas, keeping `cluster` for hosts with no other supervisor.

go deeper

for a junior

Recall that the cluster module runs several Node.js processes sharing one port, and that container platforms can run several copies of a service on their own.

for a middle

Explain how cluster distributes connections by default, that the primary accepts them, and that restarting a crashed worker is code the application must write.

for a senior

Name the concrete collisions with an orchestrator: hidden worker crashes, CPU-count sizing inside a quota, memory multiplied per worker, and the primary as a single point of failure.

for a principal

Set the team default for supervision, one layer that owns restarts and scaling, and document the narrow cases where cluster inside one host is still the right call.

## What the cluster module actually provides `node:cluster` starts several Node.js processes with `child_process.fork()` and lets them share one server port. On every platform except Windows the default policy is round-robin (`cluster.SCHED_RR`): the **primary** process listens on the port, accepts each new connection and hands it to a worker. The alternative, `cluster.SCHED_NONE`, lets workers accept directly, which the docs say tends to be badly unbalanced in practice. It was designed for a machine where nothing else runs the service: one script starts a process per core, and port sharing means no separate load balancer is needed. Inside a container that may use only a fraction of a machine, most of that value is already supplied by the platform. That is the whole feature set. The docs are explicit that Node does not automatically manage the number of workers: restarting a crashed worker, rolling restarts and graceful shutdown are code you write in the primary. ## What the orchestrator already does A service running as container replicas behind a load balancer already has: - **Multiple processes**: one per replica, each on its own share of CPU. - **Connection distribution**: the load balancer in front of the replicas. - **Supervision**: a crashed container is restarted, and failing health checks take a replica out of rotation. - **Rolling deploys**: replicas are replaced a few at a time. Adding `cluster` inside each container rebuilds all four one level down, and the two levels can disagree. ## Where the two layers collide 1. **Hidden failures.** When a cluster worker crashes, the primary restarts it (if you wrote that code) and the container stays up. The orchestrator sees a healthy replica running at reduced capacity, and a health check may land on a surviving worker and pass. If you did not write the restart, capacity simply shrinks without any alarm. 2. **CPU sizing.** The usual pattern forks one worker per `os.availableParallelism()`. libuv documents that value as reading the CPU affinity mask on Linux and the online CPU count on most other platforms; its documentation does not say a CPU quota is taken into account. Treat it as an upper bound, not as the container's allowance: if it reports more CPUs than the container may use, the extra busy processes only take turns and get throttled. Set the worker count from configuration. 3. **Memory.** Each worker is a full Node.js process with its own V8 heap. Eight workers in one container mean eight heaps and eight sets of buffers in one memory budget, and every heap limit must be divided to fit. 4. **A second single point of failure.** Under round-robin the primary accepts every new connection. Workers call `process.exit(0)` when their IPC channel to the primary disconnects unexpectedly, so if the primary dies the replica goes down anyway, which the orchestrator would have handled on its own. 5. **Shutdown.** The orchestrator's termination signal reaches the primary, which must forward it and wait for every worker to drain; the lesson on graceful shutdown covers that sequence. 6. **State.** Per-process caches and sessions were already split across replicas; `cluster` splits them further, per worker. ## When cluster inside one host still makes sense - **No orchestrator**: a single VM or bare-metal host where nothing else restarts processes or spreads connections. - **Few, large replicas by policy**: a platform that limits the replica count, or a process whose startup and warm-up are expensive enough to amortise across cores. Even then, the cluster docs point to `worker_threads` when process isolation is not needed: for CPU-heavy work inside one server, a bounded worker pool fits better than more copies of the server. ## The usual defaults | Situation | Typical choice | |---|---| | Orchestrated containers | One Node.js process per container, about one CPU each; scale replicas | | Single host with no supervisor | `cluster` with a primary that restarts workers, or an external process manager | | CPU-heavy work inside one server | A bounded `worker_threads` pool, under either model above | Interviewers asking this want to hear that **someone must supervise every process**, and that two supervisors stacked on top of each other give you duplicated work and failures that neither layer sees clearly.

  • If you do keep cluster inside a container, how should you choose the number of workers?
    From configuration tied to the container's CPU allocation, not from `os.availableParallelism()` alone, which libuv documents as the affinity mask or online CPU count. Leave room for the primary and for libuv's threadpool in each worker, and make each worker's heap limit fit the container's memory divided across all workers.
  • What happens to a Node.js cluster's workers if the primary process dies?
    Their IPC channel to the primary disconnects, and cluster workers call `process.exit(0)` on an unexpected disconnect. Under round-robin the primary was also accepting every connection, so the replica stops serving and must be restarted from outside, by the orchestrator or a process manager.

  • The cluster module restarts crashed workers automatically.
  • os.availableParallelism() is guaranteed to equal the container's CPU limit.
  • Cluster workers share one heap, so memory per container stays flat.
  • In cluster's default mode each worker accepts connections directly through SO_REUSEPORT.
  • Health checks reliably reveal a dead cluster worker inside a running container.