A Node.js production process is misbehaving and you cannot restart it. How do you inspect it live, and what does each option cost?
answer
- a signal reserved for the debugger
- loopback plus a tunnel
- a breakpoint freezes every request
- a JSON snapshot instead
- environment variables in the report
basics
~20 sSend SIGUSR1 to open the inspector, by default on 127.0.0.1:9229, reach it through a tunnel, and profile rather than pause. Or write a diagnostic report, if enabled with --report-on-signal or via process.report.writeReport(), which never stops the loop for long.
solid answer
~40 sOn Linux and macOS, `kill -USR1 <pid>` makes Node start its inspector without a restart, on `127.0.0.1:9229` or the address given by `--inspect-port`, unless the process runs with `--disable-sigusr1`. Forward the port over SSH and attach DevTools; never bind it publicly, because the inspector is unauthenticated code execution. Inside, prefer a CPU profile or evaluating an expression over breakpoints: a paused breakpoint stops the event loop, so every request on that process stalls and health checks start failing. The lighter option is a **diagnostic report**: a JSON file with the JavaScript and native stacks, heap statistics, libuv handles, resource usage and environment. It needs `--report-on-signal` at start-up (then `kill -USR2`) or a code path calling `process.report.writeReport()`. Reports include environment variables, so pass `--report-exclude-env` if secrets live there.
code
bash · 9 lines# on the host or in the container, next to the process
kill -USR1 4123 # inspector opens on 127.0.0.1:9229
# from your workstation: tunnel, then attach DevTools to localhost:9229
ssh -L 9229:127.0.0.1:9229 app-host
# reports on demand, decided at start-up
node --report-on-signal --report-exclude-env --report-directory=/var/diag server.js
kill -USR2 4123 # writes a report; only safe with --report-on-signalgo deeper
Know that Node can open its debugger on a running process when sent SIGUSR1, and that a diagnostic report is a JSON snapshot of the process.
Explain the default inspector address, why it stays on loopback, and the flags and API that produce a diagnostic report.
Show the operational judgment: tunnel instead of exposing, profile instead of pausing, check the start-up flags before sending a signal, and keep secrets out of reports.
Set the fleet default: whether SIGUSR1 stays enabled, which report-on-crash flags every service starts with, and where reports go so they survive the container and stay access-controlled.
## Two built-in ways in A running Node.js process can be examined without restarting it in two ways, both shipped in the runtime: 1. **The V8 inspector**, the debugging server behind `--inspect`, which can be switched on after start-up. 2. **The diagnostic report**, a JSON summary of the process's state written to a file on demand. They answer different questions. The inspector lets you look around interactively; the report is a snapshot you can take, copy off the machine and read later, with far less risk to live traffic. ## Opening the inspector after start-up Node reserves `SIGUSR1`: when a Node.js process receives it, Node starts the inspector. It listens on the address set by `--inspect-port` when the process was started, which defaults to `127.0.0.1:9229`. `--disable-sigusr1` turns this off, which you can set so that anyone able to signal the process cannot open a debugger in it. Code can do the same with `inspector.open()` from `node:inspector`, but then the trigger is your own code path and needs its own access control. The address matters more here than anywhere else. The inspector has **no authentication**, and Node's documentation warns that exposing it on a public interface allows **remote code execution**. Keep it on loopback and reach it from your workstation through an SSH tunnel or a port-forward to that host or container. ## What the inspector costs in production - **Breakpoints stop the world.** A paused breakpoint halts the main JavaScript thread, so the event loop stops too: every in-flight request on that process waits, timeouts fire upstream, and a health check may get the process killed while you are looking at it. - **Prefer non-pausing tools.** A CPU profile (the DevTools `Profiler` domain, which `inspector.Session` can also drive from code) records where time goes while traffic keeps flowing. Evaluating an expression in the console reads state without stopping. - **Profiling and heap work still add load.** Heap snapshots in particular are heavy, and they belong to memory-leak investigation rather than routine inspection. - **Close it afterwards.** `inspector.close()` deactivates the inspector and drops any connections, so the port does not stay open for the life of the process. ## The diagnostic report A diagnostic report is a JSON document that includes, among other sections: | Section | What it tells you | |---|---| | `header` | trigger, time, process id, Node and dependency versions, command line | | `javascriptStack`, `nativeStack` | what the main thread was executing | | `javascriptHeap` | V8 heap spaces and their usage | | `libuv` | open handles: sockets, timers, file watchers, with endpoints | | `resourceUsage` | CPU time, memory, page faults | | `environmentVariables` | the whole environment, secrets included | Ways to produce one: - `--report-on-signal` at start-up, then `kill -USR2 <pid>` (`--report-signal` changes the signal; signal-based reports are not supported on Windows). The setting can also be switched at run time with `process.report.reportOnSignal = true`, if your code exposes a way to do that. - `--report-uncaught-exception` and `--report-on-fatalerror` write one automatically when the process dies, the latter covering internal fatal errors such as running out of memory. - `process.report.writeReport()` from code, for example from an authenticated admin endpoint; `process.report.getReport()` returns the same data as an object. Without `--report-on-signal`, and unless the application installs its own listener, nothing in Node handles `SIGUSR2`, so the signal gets the operating system's default action, which on Linux terminates the process. Check how the process was started before sending it. Reports are written to the process's current working directory unless `--report-directory` says otherwise; in a container that directory may be read-only or disappear with the container. `--report-exclude-env` drops the environment section and `--report-exclude-network` drops network interfaces and the reverse DNS lookups that can slow generation. ## Choosing between them - Hung or slow process, need to know what it is doing **right now**: report first, it is quick and safe; inspector with a CPU profile if the report is not enough. - Need to step through logic: reproduce on a non-serving instance instead of pausing a live one. - Process keeps dying: enable `--report-on-fatalerror` and `--report-uncaught-exception` so the next death leaves evidence. - Several reports a minute apart: compare the `libuv` handle lists and heap statistics to see what is growing or stuck, since one snapshot cannot show a trend.
- Why might a team start every service with `--disable-sigusr1`?Without it, anyone who can send a signal to the process, any code running as the same OS user, can make it open an unauthenticated inspector. Node's permission-model notes say exactly that: cross-process inspector activation is not gated by `--permission`. Disabling it trades away a no-restart debugging path for a smaller attack surface, and leaves diagnostic reports as the no-restart option.
- When is a diagnostic report more useful than attaching the inspector?When the process must keep serving, when you need evidence from a crash, or when you are not allowed to attach a debugger. The report is written in one short synchronous step, can be triggered by signal, crash or code, and captures stacks, heap statistics and libuv handles. It is one snapshot, not an interactive session, so take several to see a trend.
- Believing the inspector can only be enabled by restarting with --inspect
- Binding the inspector to 0.0.0.0 to reach it from a laptop
- Setting breakpoints in a live server without expecting requests to stall
- Sending SIGUSR2 for a report without checking --report-on-signal was set
- Shipping diagnostic reports to shared storage without excluding the environment