How would you set preload and worker-recycling policy across a fleet of long-lived Python services?

## The question is not which setting is right; it is which decisions belong where There are three separable choices here, and treating them as one is how platforms end up with a single mandated job limit that is wrong for every service. ## First: preload or not Importing the application once in the parent and forking workers from it buys three things - - pages shared through copy-on-write, - workers that start without re-importing, - and a single place where an import error surfaces before any worker takes traffic. It costs **discipline**. Every connection, thread, pool or seeded generator built during the import is inherited by every worker and must be recreated, and the class of bug that produces is subtle and production-only. It also means the preload is a **single point of failure**: a bad import fails all workers simultaneously rather than one at a time. Not preloading inverts the trade - full isolation, per-worker reload, no inherited-descriptor discipline - at the cost of N import times, N copies of the imported heap, and slower scale-up. A service whose memory is dominated by per-request data rather than by a large static heap gains little from sharing, so the discipline buys nothing. This is a per-service judgement, and the platform's job is to make the tradeoff legible, not to pick for everyone. ## Second: how to recycle The *form* should be standard, because every service that gets it wrong gets it wrong the same way: - retire on a job count, - add a memory ceiling as a second trigger for uneven workloads, - jitter the threshold so the pool never restarts in lockstep, - and always leave at a job boundary after the in-flight work is finished and acknowledged. The *numbers* cannot be standard. A job limit is derived from measured growth per job and the memory the service is allowed, and those differ by an order of magnitude across a fleet. A platform that mandates one limit is really mandating that most services recycle far too often or far too rarely. Publish defaults with the derivation attached - here is how to measure growth per job, here is how to turn it into a limit - and let each service land its own number. ## Third: what it costs the fleet Recycling always trades throughput for a memory bound. If a pool is sized exactly to its load, it is short-handed for the whole restart, so the platform needs an explicit answer to how many workers may be recycling at once and how much headroom that requires. That answer belongs to **capacity planning**, not to individual teams, and it is the reason a per-service knob still needs a fleet-level ceiling on churn. ## Standardise the contract, not the numbers Three things are worth writing down and enforcing. 1. What the preload phase may construct: code and read-mostly data only, never sockets, sessions, threads or seeded generators. 2. What a graceful worker exit means: stop accepting, finish and acknowledge in-flight work, flush, exit with a status the supervisor treats as normal. 3. What every worker must emit: restarts per hour, jobs completed at exit, and resident memory at exit. The first prevents the inherited-state family of bugs entirely. The second makes restarts invisible to callers. The third is the one most fleets skip, and it is what keeps recycling from becoming a way to never fix anything. ## The organisational failure to design against Recycling is genuinely good engineering and it is also the most comfortable place in a system to hide a defect. A team under pressure lowers the job limit, the graph flattens, and the growth is never investigated again. Two quarters later the limit is a tenth of what it was and nobody remembers why. The countermeasure is to treat the **recycle interval as a service-level signal** rather than a configuration value: - alert when it shortens, - require an owner and a ticket for any service absorbing growth this way, - and review the numbers on a schedule. Preloading has its own version of the same trap - a memory saving that decays as workers dirty shared pages, quietly, so a capacity model built on the day-one number is wrong by the end of the first hour. ## When the interpreter's defaults move Finally, mind the platform floor. The interpreter's own defaults move: 3.14 changed the `multiprocessing` default start method to `forkserver` on Unix other than macOS, so the inherit-the-preloaded-app model now has to be requested explicitly. A fleet-wide policy that assumed the old default is a policy that silently stopped applying on upgrade, which is exactly why these decisions want to be written down with their reasons rather than inherited as folklore.