skip to content

As a lead, what rule would you set for CPU-heavy work in a Node.js service's request path, and how would you enforce it across the team?

level: principalimportance: should knowfreq 30%

answer

  1. a budget, not a slogan
  2. bound every input-dependent cost
  3. tiers: slice, worker, service
  4. lint, trace flag, loop-delay SLO

basics

~20 s

Set a main-thread CPU budget per callback, bound every input-dependent cost, and tier heavier work: slice cheap loops, use a worker-thread pool for per-request CPU, move bulk jobs to a separate service. Enforce with lint, --trace-sync-io and loop-delay alerts.

solid answer

~50 s

I would state the rule as a budget derived from the latency objective: no main-thread callback may exceed a few milliseconds of CPU for any accepted input, because every request in the process shares that thread. That forces bounds on every input-dependent cost: body limits, array lengths, length caps before regexes. Work then goes by tier: within budget stays inline; long loops over cheap items are sliced with `setImmediate`; real per-request CPU work goes to a `worker_threads` pool sized to cores, returning small results; large or bursty jobs go to a separate service behind a queue. Enforcement: a lint rule against `*Sync` calls outside startup code, CI runs with `--trace-sync-io`, load tests with a canary request, and production alerts on per-instance `monitorEventLoopDelay()` p99 and max plus event loop utilization. Exceptions need an owner and a tested input cap.

go deeper

for a junior

Recall that every request shares one main thread, so CPU-heavy code in one handler slows all of them, and that async wrappers do not move work elsewhere.

for a middle

Explain the main options for heavy work: slicing with setImmediate, a worker thread, or another process, and what each costs.

for a senior

Show how you would detect a violation in production with loop-delay and utilization metrics and trace it to a route and payload.

for a principal

Turn it into a team rule with a budget tied to the latency objective, tiers with clear criteria, automated enforcement and owned exceptions, and weigh worker pools against separate services.

## What the rule protects Every request a Node.js process serves shares one main thread. A team's rule for CPU-heavy work therefore protects a **shared resource**, not one endpoint: a slow handler written by one developer raises the latency of every route in the process. A useful rule is stated as a **budget**, because "never block the loop" is unenforceable (every callback blocks for some time), while "no main-thread callback may cost more than a few milliseconds for any accepted input" can be designed for, tested and measured. The budget should come from the latency objective. If the service promises a p99 of 100 ms and a handler spends 30 ms of uninterrupted CPU, then any request that arrives behind it inherits those 30 ms. Two or three such handlers in flight together consume the whole budget. ## Tiers of CPU work | Cost class | Where it runs | Mechanism | Tradeoff | |---|---|---|---| | Within budget, bounded input | main thread | plain code | none, as long as the bound holds | | Long loops over cheap items | main thread, in slices | yield with `setImmediate` between batches | more latency for that request, more code | | Real CPU work per request | worker threads | a pool of `worker_threads.Worker` | memory per isolate, copying data in and out | | Large or bursty batch jobs | separate process or service | a queue and dedicated workers | another deployable, asynchronous results | A few rules make the tiers concrete: - **Every input-dependent cost gets a bound.** Body size limits, maximum array lengths, length caps before any regex, page sizes. A bound turns "might block" into a known worst case you can measure. - **No `*Sync` I/O or crypto in request paths.** `fs.readFileSync`, `crypto.pbkdf2Sync`, `zlib.gunzipSync` and `child_process.execSync` are allowed at startup only. The async twins exist and cost nothing extra to call. - **Partition only cheap, divisible work.** Slicing a loop with `setImmediate` works when each item costs microseconds; it cannot split one `JSON.parse`, and each slice waits for a later loop iteration, so the job as a whole finishes later. - **Size worker pools to cores**, not to traffic. More CPU-bound threads than logical cores add memory and context switches without adding throughput. ## Enforcing it 1. **Static checks.** A lint rule that flags `*Sync` calls outside startup modules, reviewed exceptions only. 2. **Runtime checks in CI or staging.** Run integration tests with `node --trace-sync-io`, which prints a stack trace for synchronous I/O after the first turn of the event loop. 3. **Load tests with a canary probe.** Send the heaviest legitimate payloads while a cheap request measures latency; the canary's p99 is the budget check. 4. **Production SLOs.** Export `monitorEventLoopDelay()` p99 and `max` and `performance.eventLoopUtilization()` per instance, and alert on them. A new spike after a deploy names the release that broke the rule. ## Worker pool or separate service? - **Worker threads** suit work that is **per request and latency-sensitive**: hashing, image thumbnails, parsing a moderately large document. The caller awaits the result in the same request, and deployment stays one unit. - **A separate service** suits work that is **large, bursty or long**: report generation, bulk imports, video. It scales on its own, can crash without taking the API with it, and can run on a runtime better suited to computation. The official Node.js guide on not blocking the event loop is explicit that Node excels at I/O-bound work and may not be the best option for heavy computation. - **Child processes or `cluster`** add isolation or use more cores for serving, but neither makes a single blocking request cheaper for the process that runs it. ## Pitfalls to rule on explicitly 1. Wrapping CPU work in `async` or a promise changes nothing about where it runs; the rule must say so, because it is the most common false fix. 2. Returning a large object graph from a worker costs a structured clone on the main thread; workers should return small results. 3. Third-party libraries may block. The rule should require measuring a dependency's cost on realistic input, since an asynchronous API can still do large slices of work on the main thread. 4. Exceptions need an owner and a bound. "This endpoint may take 200 ms of CPU" is acceptable only if the input cap that guarantees it is written down and tested.

  • When would you move CPU work to a separate service rather than a worker-thread pool?
    When the work is large, bursty or long: bulk imports, reports, media processing. A separate service scales independently, can fail without taking the API down, and can use a runtime better suited to computation. Worker threads fit per-request, latency-sensitive work where the caller awaits the result and one deployable is simpler.
  • How do you catch a violation that slipped past code review?
    In production, per-instance `monitorEventLoopDelay()` p99 and max alerts flag the stall, and a spike that starts with a deploy names the release. In staging, `--trace-sync-io` and a load test with a canary request catch sync I/O and slow handlers before they ship.

  • Marking a CPU-heavy function async is enough to keep the loop free.
  • Running more cluster processes fixes a request that blocks its own process.
  • Worker pools should grow with traffic rather than with available cores.
  • Setting a rule of never blocking the loop, with no measurable budget.
  • Trusting that an asynchronous library API never does heavy work on the main thread.