Why should a long-lived Python service run in the foreground instead of double-fork daemonising itself?
answer
- Who is the parent matters
- Supervision runs on the parent-child link
- Detaching throws away status and output
- fork, setsid, fork again is legacy
- Foreground plus the standard streams
basics
~20 sA supervisor manages a process by being its parent. Staying in the foreground keeps the supervisor as the parent, so it knows the pid, reads the exit status, and captures whatever the process writes to sys.stdout and sys.stderr.
solid answer
~40 sThe double-fork ritual — `os.fork()`, let the parent exit, `os.setsid()`, fork again, `os.chdir('/')`, close the inherited descriptors and reopen the standard streams onto a file — exists to make a process survive with nothing watching it. Under a supervisor that machinery is actively harmful: the process the supervisor started exits within milliseconds, so the supervisor thinks the service already died; the surviving grandchild is reparented away, so its exit status goes nowhere useful; and the pipes the supervisor wired to the child were closed and reopened onto a log file it cannot see. The modern contract is the opposite: stay in the process that was exec'd, log records to `sys.stdout`/`sys.stderr` and let the supervisor route them, exit non-zero on failure, and own no restart policy at all.
code
python · 14 linesimport logging
import sys
log = logging.getLogger("inventory.sync")
def main() -> int:
logging.basicConfig(level=logging.INFO, stream=sys.stderr)
log.info("running in the foreground; the supervisor owns the pid")
return 0
if __name__ == "__main__":
sys.exit(main())go deeper
Be able to say that a supervised program stays in the foreground and prints its logs to the standard streams. Recognise sys.exit(main()) under if __name__ == '__main__': as the normal shape of a service entry point.
Explain the mechanics: the supervisor is the parent, so it owns the pid, the exit status and the pipes attached to sys.stdout and sys.stderr. Walk through what each step of the double-fork recipe was for and which supervisor property it breaks.
Show that you have debugged the failure mode: a service that daemonised itself, a supervisor managing a pid that had already been recycled, and logs going to a file nobody was shipping. Be ready to argue for deleting the daemonise flag rather than configuring the supervisor around it.
Own the platform position: one supervision model for every service, logs to the standard streams as an organisational contract, and no per-service pidfile or forking mode. Be able to explain the migration cost for legacy services that still detach, and why a per-team exception is expensive.
## What double-forking was actually for The classic Unix daemon recipe is a fixed sequence: 1. call `os.fork()` and let the parent exit so the shell that launched you gets its prompt back; 2. call `os.setsid()` in the child so it becomes the leader of a brand-new session with no controlling terminal and cannot be signalled by a terminal hangup; 3. fork a second time so the surviving grandchild is not a session leader and can therefore never reacquire a terminal; 4. `os.chdir('/')` so the process does not pin a mount point; 5. reset the file-creation mask with `os.umask`; 6. close every inherited file descriptor; 7. reopen the standard streams with `os.dup2` onto `os.devnull` or a log file; 8. and finally record the new pid in a file so someone can find the process later. Every one of those steps solves the same problem: **nothing is watching this process**, so it has to detach from the terminal that started it and leave its own breadcrumbs behind. ## Why a supervisor inverts every step A process supervisor manages a service by **being its parent**. That single relationship is where all of its power comes from. - It knows the pid without a pidfile, because it is the pid it got back from the fork it did. - It reaps the child and therefore reads the exit status directly. - It holds the read ends of the pipes wired to the child's `sys.stdout` and `sys.stderr`, so everything the process prints is captured, timestamped and shipped without the process ever opening a file. - It decides whether, when and how often to start a replacement. Double-forking destroys all four properties at once. The process the supervisor actually started exits almost immediately, which the supervisor reads as the service having finished — either an instant clean exit or an instant crash, and neither is true. The surviving grandchild is reparented, so when it eventually dies its exit status is delivered to a process that has no idea what the code means and no restart policy to apply. The descriptors the supervisor handed over were closed and reopened onto a log file, so the log pipeline it set up carries nothing. Supervisors that offer a `forking` mode do so by trying to guess the real pid out of a pidfile written by a process it does not control, which is a race at start-up, a race at restart, and a source of the classic failure where a supervisor cheerfully manages a pid that now belongs to something else. ## The contract, stated positively A supervised Python service owes four things. 1. **Run in the foreground**: whatever the entry point does, it must do it in the process that was exec'd, never in a detached descendant. 2. **Log to the standard streams**: configure `logging.basicConfig(stream=sys.stderr)` or add a `logging.StreamHandler`, and delete the `logging.handlers.RotatingFileHandler` you were going to configure — rotation, retention and destination are the supervisor's job, and a service that writes its own log file is invisible to it. 3. **Exit non-zero on failure**, so the supervisor's restart policy has something to act on. 4. And **own no restart backoff** of your own. In practice this collapses to one idiom at the bottom of the module: a `main()` that returns an `int`, called as `sys.exit(main())` under `if __name__ == '__main__':`. Nothing forks, nothing writes a pidfile, nothing redirects a stream. ## Two details that make the old recipe obsolete rather than merely unnecessary - Since **PEP 446** landed in 3.4, file descriptors that Python itself creates are non-inheritable by default, so the `close every descriptor from 3 to the limit` loop in the daemonise ritual no longer protects you from anything your own code opened; `os.set_inheritable` exists for the rare case where you deliberately want a descriptor to survive an exec. - And since 3.12, `os.fork()` emits a `DeprecationWarning` when it is called from a process that already has more than one thread — which is exactly what a hand-rolled daemoniser does when it forks after a metrics or logging thread has started. CPython never shipped a daemonising helper of its own: PEP 3143 proposed a standard daemon library and it never landed, and the reason it stopped mattering is that supervision moved out of the process. ## Where forking still belongs None of this says a service may never fork. A pre-forking server that loads the application and then forks workers is forking *downward*, and the parent it forks from is still the process the supervisor started and still the process whose exit status the supervisor reads. Daemonising is different: it forks *away* from the supervisor and then abandons the relationship. That is the thing that is obsolete.
- What does the supervisor lose if your entry point calls os._exit() instead of returning from main?The status still reaches the supervisor, because `os._exit` passes it straight to the kernel, but everything else is skipped: `atexit` callbacks never run, `finally` blocks never run, logging handlers are never flushed and buffered output can be lost. Reserve `os._exit` for a forked child that must not run the parent's cleanup; in the main process, return a code from `main()` and let `sys.exit` raise `SystemExit` so the stack unwinds normally.
- A team wants the service to keep running after the terminal that started it is closed. What is the right answer today?Start it under the supervisor instead of teaching the program to detach. Terminal-detachment tricks are what you did when nothing was supervising the process; a supervisor starts the service outside any terminal session already, so the application never needs `os.setsid` and never needs to survive a hangup on its own.
- Does a supervised service still need to close its inherited file descriptors at startup?Not as a blanket loop. Since PEP 446 in 3.4, descriptors that Python itself creates are non-inheritable by default, so they do not leak across an exec unless you explicitly mark them with `os.set_inheritable`. What you may still care about is a descriptor deliberately passed in by the supervisor — an inherited listening socket, for example — which you want to keep, not close.
Daemonising is like an employee who clocks in at reception and then slips out the back door to do the work anonymously: the manager sees them leave, has no way to tell whether the job got done, and cannot send anyone to replace them.
saying these in an interview costs you the question
- Says a real daemon must fork twice and detach
- Redirects sys.stdout to a log file the supervisor cannot read
- Writes a pidfile so the supervisor can find the process
- Assumes a supervisor can track a process it did not parent
- Calls os.setsid inside the application to survive the terminal
- Configures a rotating file handler in a supervised service