skip to content

Spawning Subprocesses

Running another program or Node process via four APIs that differ in shell use and output buffering. Interviewers treat exec with interpolated input as command injection and expect execFile.

on this pageshow

explore

questions

6

In Node.js's node:child_process module, how do spawn, exec, execFile and fork differ in shell use, output handling and communication?

level: middleimportance: must knowfreq 70%

answer

  1. one primitive, three conveniences
  2. which one starts /bin/sh
  3. streams versus a callback with strings
  4. fork adds a message channel

basics

~20 s

spawn starts a program without a shell and streams its stdio; exec runs a command string through a shell and buffers the output; execFile buffers too but skips the shell; fork starts a Node.js module with an IPC channel.

solid answer

~40 s

All four are asynchronous, return a `ChildProcess` and are built on `spawn`. `spawn(command, args)` runs the program directly (`shell: false` by default) and exposes `stdout` and `stderr` as streams, so output of any size arrives incrementally. `exec(command)` hands one string to `/bin/sh -c` (or `process.env.ComSpec` on Windows), so pipes, globs and redirection work, and collects stdout and stderr into strings for its callback, capped by `maxBuffer` at `1024 * 1024` bytes each by default. `execFile(file, args)` buffers the same way but launches the file directly with an argument array, which is slightly cheaper and keeps arguments away from shell parsing. `fork(modulePath)` is a `spawn` of `process.execPath` with an IPC channel added for `send()` and `'message'`. The `*Sync` variants block the event loop until the child exits.

code

javascript · 12 lines
javascript
import { execFile, spawn } from 'node:child_process';
import { promisify } from 'node:util';

const execFileAsync = promisify(execFile);

// Buffered: small, bounded output; no shell involved.
const { stdout } = await execFileAsync(process.execPath, ['--version']);
console.log('node', stdout.trim());

// Streamed: output of any size, consumed as it arrives.
const child = spawn(process.execPath, ['-e', 'for (let i = 0; i < 3; i++) console.log(i)']);
for await (const chunk of child.stdout) process.stdout.write(chunk);

go deeper

for a junior

Know the four names and the one-line difference: spawn streams, exec uses a shell and buffers, execFile buffers without a shell, fork runs Node code with messaging.

for a middle

Explain the three axes (shell, buffering, IPC), the maxBuffer default of 1024 * 1024 bytes, and why fork is not a clone of the parent.

for a senior

Justify the choice per call site: execFile for request-driven arguments, spawn for unbounded output, never the Sync variants on a request path.

for a principal

Set a team default that external programs run through execFile or spawn with argument arrays, so exec with a shell needs a reviewed reason.

## One primitive and three conveniences The `node:child_process` module has one real primitive, **`spawn()`**, and three asynchronous conveniences built on top of it: **`exec()`**, **`execFile()`** and **`fork()`**. All four return a **`ChildProcess`** (an `EventEmitter` with `stdin`, `stdout` and `stderr` streams), and none of them blocks the event loop. They differ along three axes an interviewer is probing: - **Shell:** does Node start a shell and let it parse a command line, or does it start the program directly with an argument list? - **Output:** do you get streams that deliver chunks as they come, or one buffered string per stream at the end? - **Communication:** is there a message channel between parent and child beyond the three stdio pipes? ## Side by side | Function | Starts a shell? | Output | Extra | |---|---|---|---| | `spawn(command, args, options)` | No (`shell: false` default) | streams on the `ChildProcess` | full `stdio` control, `detached`, `timeout`, `signal` | | `exec(command, options, callback)` | Yes: `/bin/sh -c` on Unix, `process.env.ComSpec` on Windows | buffered strings in `callback(error, stdout, stderr)` | `maxBuffer` cap, `timeout`, `killSignal` | | `execFile(file, args, options, callback)` | No (`shell: false` default) | buffered strings, same as `exec` | same options as `exec`, argument array | | `fork(modulePath, args, options)` | No (the `shell` option is ignored) | inherits the parent's stdio unless `silent: true` | IPC channel: `send()`, `'message'`, `serialization` | Some details worth stating precisely: - `exec()` takes **one string** and the shell splits it into words, so quoting, `$VAR` expansion, globbing (`*.pdf`), pipes (`a | b`) and redirection (`> out.txt`) all work, and all of it applies to anything you interpolate into that string. - `execFile()` takes the executable and an **array**. Each element reaches the program as exactly one argument, and because no shell runs, globbing and redirection do not happen. Node's docs call it slightly more efficient than `exec()` for the same reason. - Both buffering functions enforce **`maxBuffer`**, the largest number of bytes allowed on stdout or stderr, default `1024 * 1024`. Crossing it terminates the child. - `fork()` is a special case of `spawn()` for **Node.js programs**: it runs `process.execPath` (the current `node` binary) with the parent's `execArgv` and adds an `'ipc'` entry to `stdio`. Despite the name it does **not** clone the current process the way the POSIX `fork(2)` call does; the child loads the module from scratch with its own memory and V8 instance. ## Choosing between them 1. You need a shell feature (a pipeline, a glob) on a **fixed, trusted** command string, and the output is small: `exec()`. 2. You run one program with arguments, some of them coming from requests, and the output is small and bounded: `execFile()`. 3. Output is large or unbounded, you want to stream it somewhere, feed `stdin`, or control file descriptors and process groups: `spawn()`. 4. The child is your own Node.js code and the two sides must exchange messages: `fork()`. Moving CPU work off the main thread inside one process is the worker-threads material, and spreading one HTTP server over several cores is the cluster lesson. ## Promisified forms `exec` and `execFile` define custom promisified versions, so `util.promisify(execFile)` returns a promise for `{ stdout, stderr }` with the `ChildProcess` attached as its `child` property. The promise **rejects for any non-zero exit**, and the rejection carries `stdout` and `stderr` too, so you still see what the program printed before failing. `spawn` and `fork` have no such form: you wait for `'close'` or `'exit'`, for example with `events.once()`. ## The synchronous variants `spawnSync()`, `execSync()` and `execFileSync()` do the same jobs but **block the event loop** until the child exits. They suit a command-line script or a one-time read of configuration at start-up. Inside a request handler they stall every other connection the process serves for as long as the child runs. There is no synchronous form of `fork()`. ## Mistakes interviewers listen for - "`fork()` copies the parent like `fork(2)`": it starts a brand-new Node.js process. - "`spawn` needs a shell to find programs": it looks the command up on `PATH` itself (`options.env.PATH` when you pass `env`). - "`exec` is fine because the output is streamed": its callback receives the whole output at once, capped by `maxBuffer`. - "`execFile` expands wildcards": nothing expands them without a shell, so `'*.pdf'` reaches the program literally. - Reaching for `execSync()` in a server because it reads more simply.

  • Can you stream output from exec or execFile, or are you forced to wait for the callback?
    Both return a `ChildProcess`, so you can attach `'data'` listeners to `child.stdout` while the program runs. Node still accumulates the same output for the callback and still enforces `maxBuffer`, so streaming from them does not lift the size limit. When output is large, use `spawn`, which never buffers on your behalf.
  • Why would you run a Node.js script with spawn(process.execPath, [script]) instead of fork(script)?
    When you do not want the IPC channel. A forked child keeps the parent's event loop alive through that channel until `disconnect()`, inherits the parent's `execArgv` (flags such as `--max-old-space-size`), and expects Node on the other side. `spawn` of `process.execPath` gives a plain process with stdio pipes, which is enough for a script you only start, read and wait for.

  • child_process.fork() clones the running process like the POSIX fork call.
  • execFile runs through a shell, so wildcards in its arguments expand.
  • exec's callback can take output of any size.
  • spawn uses a shell by default, so 'a | b' works as a command.
  • execSync is fine in a request handler because the child does the work.
open as a page

A Node.js conversion endpoint builds a child_process.exec command string from the uploaded file's name; what is wrong, and how should the call be rewritten?

level: seniorimportance: must knowfreq 60%

basics

~20 s

exec hands the whole string to /bin/sh, so a file name containing shell syntax runs commands. Use execFile or spawn with a fixed binary path and an argument array, keep shell off, and never pass the client's name through.

open as a page

In Node.js, after starting a program with child_process.spawn, how do you read its output, get its exit code, and detect that it never started?

level: juniorimportance: should knowfreq 50%

basics

~10 s

child_process.spawn returns a ChildProcess: read subprocess.stdout and subprocess.stderr as streams, take the exit code from the 'close' event, and listen for 'error', which fires with ENOENT when the program cannot be started.

open as a page

In Node.js, what does child_process.fork add on top of spawn, and how do messages sent over its IPC channel behave?

level: middleimportance: should knowfreq 45%

basics

~20 s

child_process.fork starts a new Node.js process running a module, with its own memory and V8 instance, plus an IPC channel: send() and 'message' carry copies serialized as JSON by default, or by the v8 serializer when serialization is 'advanced'.

open as a page

A Node.js service converting documents with child_process.execFile now fails with 'stdout maxBuffer length exceeded'; what did Node do, and how do you fix it?

level: middleimportance: should knowfreq 40%

basics

~20 s

execFile buffers stdout and stderr up to maxBuffer bytes each (1024 * 1024 by default); past that Node truncates the output, signals the child with killSignal and fails with ERR_CHILD_PROCESS_STDIO_MAXBUFFER. Stream large output with spawn instead.

open as a page

A Node.js service calls subprocess.kill() on converter jobs that time out, yet stray converter processes keep piling up on the host; why, and how do you fix it?

level: seniorimportance: should knowfreq 35%

basics

~20 s

subprocess.kill() sends SIGTERM to the one process Node started; a shell wrapper's or converter's own children survive, and a program may ignore SIGTERM. Spawn with detached: true, signal the group with process.kill(-pid), and escalate to SIGKILL.

open as a page