A Node.js service calls subprocess.kill() on converter jobs that time out, yet stray converter processes keep piling up on the host; why, and how do you fix it?
answer
- kill() signals one PID
- shells and helpers survive
- SIGTERM can be caught
- detached starts a process group
- a negative PID means the group
basics
~20 ssubprocess.kill() sends SIGTERM to the one process Node started; a shell wrapper's or converter's own children survive, and a program may ignore SIGTERM. Spawn with detached: true, signal the group with process.kill(-pid), and escalate to SIGKILL.
solid answer
~50 s`subprocess.kill()` sends `'SIGTERM'` (or the signal you name) to the child's PID only, and Node's docs say that on Linux the child's own children are not terminated with it. That is exactly the situation when the job runs through a shell (`exec`, `shell: true`, `sh -c`) or the converter starts helpers. `kill()` returning `true` and `subprocess.killed` only mean a signal was delivered, and a program can catch or ignore `SIGTERM`. A survivor holding the stdout pipe also stops `'close'` from firing. On POSIX, spawn with `detached: true`, which makes the child the leader of a new process group, signal the group with `process.kill(-child.pid, 'SIGTERM')`, and send `'SIGKILL'` to whatever remains after a grace period. Children also outlive the parent's own exit, so shutdown must kill the jobs it still tracks. Windows rejects group kills and needs its own handling.
code
javascript · 34 linesimport { spawn } from 'node:child_process';
import process from 'node:process';
import { setTimeout as sleep } from 'node:timers/promises';
async function killGroup(child, graceMs = 5_000) {
const group = -child.pid; // a negative PID addresses the whole process group
const send = (signal) => {
try {
process.kill(group, signal);
return true;
} catch (err) {
if (err.code === 'ESRCH') return false; // nothing left in the group
throw err;
}
};
if (!send('SIGTERM')) return;
const deadline = Date.now() + graceMs;
while (Date.now() < deadline) {
await sleep(100);
if (!send(0)) return; // signal 0 only tests that the group still exists
}
send('SIGKILL'); // whatever ignored SIGTERM
}
// A shell whose own child would survive a plain subprocess.kill().
const job = spawn('sh', ['-c', 'sleep 300 & wait'], {
detached: true, // POSIX: the child leads a new process group
stdio: 'ignore',
});
job.on('exit', (code, signal) => console.log('job exit', { code, signal }));
await sleep(200);
await killGroup(job, 2_000);
// job exit { code: null, signal: 'SIGTERM' }go deeper
Remember that subprocess.kill() sends SIGTERM by default and that a signalled child reports code null with the signal name.
Explain why kill() reaches only one PID, why killed does not mean exited, and why 'exit' can fire while 'close' waits.
Show the fix: detached: true for a process group, process.kill(-pid), a grace period, SIGKILL, and cleanup of tracked children on the service's own shutdown.
Decide where subprocess lifetimes are owned, by the service or by a supervisor or container boundary, so orphaned jobs cannot exhaust a shared host.
## What kill() really does `subprocess.kill([signal])` is a signal sender, not a terminator. It calls the operating system's `kill(2)` for **one PID**, the process Node started, with `'SIGTERM'` unless you name another signal, and returns `true` if delivery succeeded. Three consequences follow, all documented: - **`subprocess.killed`** becomes `true` when the signal was delivered. It does *not* say the child has terminated. - The signal **may not terminate** the process at all: a program can install a handler for `SIGTERM`, or ignore it. A Node.js child, for instance, handles `SIGTERM` by running its cleanup and then re-raising the signal. - **Descendants are untouched.** Node's docs state that on Linux, child processes of child processes are not terminated when you kill their parent, and name the common cause: running the program through a shell or with the `shell` option. The built-in limits behave the same way. The `timeout` option and an aborted `signal` on `spawn`, `exec` and `execFile` both end in `child.kill(killSignal)`, so they also reach only the direct child. ## Why processes pile up Put the service's situation together: 1. The job runs as `/bin/sh -c "convert ..."` (any `exec` call, or `shell: true`), or the converter itself starts worker processes. 2. The timeout fires and `kill()` signals the shell or the converter's top process. 3. The grandchildren keep running, now reparented, still using CPU, memory and temporary files. 4. If a survivor still holds the stdout or stderr pipe, `'exit'` fires but **`'close'` does not**, because `'close'` waits for every process sharing those streams. Code waiting on `'close'` waits on a process it believes it killed. The service also leaks children when **it** goes away. Node's docs say children may keep running after the parent exits whether or not they are detached. A crash or `process.exit()` in the parent leaves every in-flight conversion behind unless shutdown code ends them first. ## Reading how a child ended | Observation | Exited on its own | Ended by a signal | |---|---|---| | `'exit'` / `'close'` arguments | `(code, null)` | `(null, 'SIGTERM')` | | `subprocess.exitCode` | the code | `null` | | `subprocess.signalCode` | `null` | the signal name | Shells and container runtimes report a signal death as `128 + signal number` (143 for `SIGTERM`, 137 for `SIGKILL`). Node passes the signal name instead, and since **v24.14.0** `util.convertProcessSignalToExitCode(signal)` converts it when you need that number for logs or for your own exit code. ## Killing the whole job On Linux and macOS the fix is to give the job its own **process group** and signal the group: 1. Spawn with **`detached: true`**. On non-Windows platforms this makes the child the leader of a **new process group and session**, so the group ID equals `child.pid` and everything it starts joins that group by default. 2. Signal the group with **`process.kill(-child.pid, 'SIGTERM')`**. `process.kill(pid, signal)` follows `kill(2)`, which treats a negative PID as a process-group ID. It throws if no such target exists, so treat `ESRCH` as "already gone". 3. Wait a **grace period**, checking with signal `0`, which tests existence without delivering anything. 4. Send **`'SIGKILL'`** to the group for whatever ignored `SIGTERM`. 5. Track running jobs and run the same routine from the service's own shutdown path. ```js process.kill(-child.pid, 'SIGTERM'); // the whole group, not one PID ``` ## detached is not the same as background `detached: true` is also used to start a daemon that outlives the parent. That needs more than the flag: the parent still waits for a detached child unless you call **`subprocess.unref()`**, and the child stays tied to the parent's terminal unless its `stdio` is not connected to the parent (for example `'ignore'`). For a converter job you want the opposite: the group for killing, while the parent keeps tracking the child. ## Pitfalls - **PID reuse:** signalling a child that already exited is not an error, but if its PID was reassigned the signal hits an unrelated process. Stop signalling once `'exit'` has fired and the group check fails. - **Windows:** there are no POSIX signals. `process.kill()` throws when given a process-group PID, and `subprocess.kill()` targets a single process, so Windows needs its own way of ending a process tree. - **A Node.js child** delays its exit briefly on `SIGTERM` while it cleans up, so give it a real grace period before `SIGKILL`.
- Why does a 'close' handler never run after you kill a shell-wrapped job, even though 'exit' fired?`'close'` is emitted only after the process has ended and all its stdio streams have closed. A grandchild that survived the kill inherited those pipes and still holds them open, so the streams never reach end-of-file. Killing the whole process group, or spawning with stdio not connected to the parent, lets `'close'` arrive.
- When should the parent send SIGKILL straight away instead of SIGTERM first?Almost never as the first step. `SIGTERM` lets the converter delete temporary files and flush output, and a Node.js child runs its cleanup before re-raising the signal. Send `SIGKILL` only after the grace period, or when you already know the program ignores `SIGTERM`, because nothing can catch it and cleanup is skipped.
- subprocess.kill() terminates the child and everything it started.
- subprocess.killed === true means the child has exited.
- Child processes are killed automatically when the Node.js parent exits.
- The timeout option kills the whole process tree when it fires.
- process.kill(-pid) works the same way on Windows.
Assumes Node 24.14 or later for util.convertProcessSignalToExitCode, added in v24.14.0; kill(), detached process groups and the descendant behaviour are unchanged across supported releases.