skip to content

A Node.js API renders PDF exports in a worker_threads pool, yet one large export sometimes kills the whole pod and its unrelated requests. Why, and what would you change?

level: seniorimportance: should knowfreq 45%

answer

  1. threads share one process's fate
  2. heap limits ignore external memory
  3. one container limit for every heap
  4. move the boundary to a process

basics

~20 s

Worker threads share one process and one container memory limit. resourceLimits caps only each worker's JavaScript heap, not Buffers or native memory, so concurrent large exports can exhaust the process. Cap concurrency and budget memory, or render in a separate process.

solid answer

~40 s

A worker thread has its own heap but shares the process and the container's memory limit, so when the process aborts or the container is killed for memory, every request in it dies. `resourceLimits` only ends a worker whose JavaScript heap overflows, with `ERR_WORKER_OUT_OF_MEMORY`; the docs say it does not cover external data such as `ArrayBuffer`s, and the process can still abort on a global out-of-memory condition. PDF bytes in `Buffer`s and a native renderer's allocations escape it. I would set `resourceLimits`, cap the pool, stream output, and budget the container for main heap plus N workers with their buffers. If exports must never share fate with the API, I would move rendering into a forked child pool with its own heap cap, or into a separate deployment fed by a queue.

code

javascript · 21 lines
javascript
import { Worker } from 'node:worker_threads';

export function renderPdf(job) {
  return new Promise((resolve, reject) => {
    const worker = new Worker(new URL('./render-pdf.js', import.meta.url), {
      workerData: job,
      resourceLimits: { maxOldGenerationSizeMb: 256 },
    });
    worker.once('message', resolve);
    worker.once('error', (err) => {
      if (err.code === 'ERR_WORKER_OUT_OF_MEMORY') {
        reject(new Error('export exceeded its heap budget', { cause: err }));
      } else {
        reject(err);
      }
    });
    worker.once('exit', (code) => {
      if (code !== 0) reject(new Error(`render worker exited with code ${code}`));
    });
  });
}

go deeper

for a junior

Remember that worker threads live inside one process, so the process's memory and crashes are shared by every worker in it.

for a middle

Know what resourceLimits covers, the JavaScript heap only, and what the parent sees when a worker reaches it: an error with ERR_WORKER_OUT_OF_MEMORY, then exit.

for a senior

Diagnose which of three failures happened, worker limit, process abort or container kill, and bound memory with pool caps, streaming and an honest per-replica budget.

for a principal

Decide where the isolation boundary belongs: whether export failures may share fate with API traffic, and what a process pool or separate deployment costs the team.

## What the symptom tells you When PDF exports run in a `worker_threads` pool and a large export takes the whole pod down, health checks, I/O endpoints and in-flight requests included, the failure is happening at the level of the **process** or the **container**, not the worker. A worker thread is a thread inside the same process. It has its own JavaScript heap and event loop, but it shares the process's address space, its memory accounting and its fate. Two different events look alike from outside: 1. **The process aborts.** V8 prints a fatal "JavaScript heap out of memory" error and the process exits. This happens when the main thread's heap runs out, or when the process as a whole runs out of memory despite per-worker limits. 2. **The container is killed.** The kernel or the orchestrator kills the process for exceeding the container's memory limit. No JavaScript runs: no `'exit'` handler, no log line from Node, only the orchestrator's record of a memory-limit kill. Either way, every request that process was serving is lost. ## Why per-worker limits did not save you `new Worker()` accepts `resourceLimits`, and reaching them terminates just that worker: the parent receives `'error'` with code `ERR_WORKER_OUT_OF_MEMORY`, then `'exit'`. That is the behaviour you want, but the docs scope it narrowly: - The limits "only affect the JS engine, and no external data, including no `ArrayBuffer`s". PDF bytes, images and anything held in a `Buffer` live outside the heap and are not capped. - Memory a native add-on allocates, such as a C++ rendering library's, is outside the JavaScript heap too. - "Even if these limits are set, the process may still abort if it encounters a global out-of-memory situation." - A worker's `execArgv` does not accept V8 options such as `--max-old-space-size`; per-worker caps go through `resourceLimits`. And if `--max-old-space-size` is set on the command line, it overrides each worker's `maxOldGenerationSizeMb`. Now add concurrency. Four workers each assembling a few hundred megabytes of document in a `Buffer`, plus their own heaps, plus the main thread's heap, all count against one container limit. Nothing in Node adds them up for you. A native crash is worse still: a fault in an add-on's code kills the process whichever thread it ran on. Add-ons must also be Node-API or context-aware to be loaded in more than one thread. ## What to change | Change | What it buys | What it costs | |---|---|---| | `resourceLimits` on every render worker | A JavaScript-heap blowup ends one worker, not the process | Nothing for buffers or native memory | | Cap the pool and queue the rest | A bounded worst case of N exports at once | Exports wait under load | | Stream output instead of one big `Buffer` | Lower peak memory per export | The renderer must support streaming | | Budget main heap + N x (worker heap + buffers) | A container limit that matches reality | More memory per replica | | A forked child pool for rendering | An export crash or OOM ends that child only | A full Node.js instance per child, IPC copying | | A separate export deployment behind a queue | Export load and failures never touch API pods | A queue, job status, more to operate | The first four keep the design and bound the damage. The last two move the isolation boundary: once another process does the rendering, an export that exhausts memory or crashes in native code ends that process, and the API's process keeps serving. A forked child can get its own heap cap through `execArgv` (for example `--max-old-space-size=512`) and can be stopped with `subprocess.kill()` when a job overruns. It still shares the container's memory limit, so the concurrency cap stays. ## How to confirm the diagnosis - Read the orchestrator's restart reason: a memory-limit kill points at total process memory rather than any one heap. - Search the logs for Node's fatal heap message: that points at the main heap or a global out-of-memory abort. - Listen for `'error'` on every `Worker` and log `err.code`; `ERR_WORKER_OUT_OF_MEMORY` means the per-worker limits are firing. - Watch `process.memoryUsage()` during a large export. With worker threads, `rss` covers the whole process while `heapUsed` and `arrayBuffers` describe only the calling thread, so an `rss` far above the main thread's heap means the memory sits in workers, buffers or native code.

  • How do you tell a worker hitting its resourceLimits apart from the container being killed for memory?
    A worker at its limit is a contained event: the parent gets `'error'` with code `ERR_WORKER_OUT_OF_MEMORY` and then `'exit'`, and the process keeps running. A container kill leaves no trace inside Node: the process is gone, no handler ran, and only the orchestrator records a memory-limit kill. A fatal 'JavaScript heap out of memory' line in between means the process itself aborted.
  • Why is moving rendering into child_process.fork() children not sufficient on its own?
    A child process isolates crashes and gets its own heap cap through `execArgv`, but it still runs inside the same container and counts against the same memory and CPU limits. Without a concurrency cap and headroom in the budget, several large exports can still push the container over its limit. Full independence needs a separate deployment with its own limits.

  • resourceLimits caps everything a worker allocates, Buffers included.
  • Worker threads are isolated like processes, so one cannot take down the rest.
  • Raising the container memory limit permanently fixes it without capping concurrent exports.
  • Any worker failure, even a handled exception, brings the pod down.
  • Passing --max-old-space-size in a Worker's execArgv caps that worker's heap.