What does running supervisord inside a Docker container cost you when the container stops or a process crashes?

### The shape of the problem `supervisord`, `s6-overlay` and similar supervision suites solve a real problem *outside* containers: on a host you need something to start services, restart them when they die, and keep them running. Inside a container that job already has an owner -- the Docker daemon (and, above it, whatever schedules your containers). Putting a supervisor inside a container inserts a second, private control plane between the engine and your application, and the engine cannot see through it. Concretely, PID 1 in the container becomes the supervisor. Everything the engine reports and everything the engine acts on is therefore about the supervisor, not about your application. ### Stop: two nested deadlines `docker stop` sends SIGTERM to PID 1 and then waits a grace period -- 10 seconds by default, settable with `docker stop -t` or `docker run --stop-timeout` -- before sending SIGKILL to everything left in the container. With a supervisor as PID 1, that SIGTERM is delivered to the supervisor. A well-configured supervisor does forward a stop signal to the programs it manages and then waits its own configured time for them to leave. You now have two timeouts stacked: the engine's grace period, and the supervisor's own wait for each program. If the inner wait is as long as or longer than the outer one -- and the common defaults are uncomfortably close -- the engine's SIGKILL arrives while the application is still draining, and the graceful shutdown you carefully wrote never completes. Worse, a process the supervisor did not start, or a child the application forked into its own process group, may never be signalled at all and is simply killed at the end of the grace period. The failure is quiet. Nothing errors; the container just always takes the full grace period and dies hard, and in-flight requests are lost every deploy. ### Crash: the container stays "up" A supervisor's whole purpose is to survive the death of the things it supervises. Default configurations restart a failed program a few times and then park it in a failed state -- while the supervisor itself keeps running happily. From the engine's point of view nothing happened: PID 1 is alive, so the container is `Up`, so `--restart on-failure` is never evaluated, so no orchestrator above notices either. You get a container that is running and doing nothing. Take a Rust recommendation re-ranker packaged with a cache-warmer under one supervisor. The re-ranker panics 41 hours into the run and the supervisor gives up on it after a few retries. `docker ps` still prints `Up 6 days`; `docker inspect` still shows `"Status": "running"` with no exit code; the only signal anything is wrong is a p99 latency graph on a service that no longer answers. Had the re-ranker been PID 1 of its own container, it would have exited, the exit code would have been recorded, the restart policy would have fired, and a crash loop would have been visible. ### The exit code stops telling you what died `State.ExitCode` is PID 1's status. With a supervisor in front, the container's exit code is the supervisor's -- typically a single generic value however the application died. The distinction between "the application chose to exit", "the application was killed", and "the supervisor could not start anything at all" collapses into one number, and post-mortem tooling that keys on exit status loses its input. ### Logs By default a supervisor writes each program's output to log files *inside* the container rather than to its own stdout, so `docker logs` shows supervisor chatter and none of the application's output. This is fixable -- point each program's stdout and stderr at the container's own stdout/stderr in the supervisor config -- but it is extra configuration you must remember, and interleaving two programs' output on one stream then makes it harder to tell them apart again. ### Everything else that gets murkier Resource limits (`--memory`, `--cpus`, `--pids-limit`) apply to the whole container, so the two programs share one budget and the kernel picks which to kill under pressure. `docker stats` gives you one row for both. A `HEALTHCHECK` can be written to check both, but it is now the only mechanism that notices a dead child -- and an unhealthy container is not restarted by the engine on its own. The image grows to carry the supervisor and its runtime, and its configuration file becomes a second, undocumented deployment descriptor. ### What to do instead Make the application PID 1 of its own container, in exec form, and let the engine do the supervising: one concern per container, a restart policy per container, one stream of logs per container, one exit code per container. If two programs really must ship together, the burden is on that design to restore what the engine lost: forward signals correctly, exit non-zero as soon as any supervised program fails permanently, and route all output to stdout.