skip to content

Serving one FastAPI API both on a Kubernetes cluster and on a single VM, how do you decide between `--workers` inside a container and one process per container?

level: principalimportance: should knowfreq 45%

answer

  1. who already replicates and restarts
  2. memory you can account for
  3. one image, two start counts
  4. steps that must run once

basics

~20 s

Let whatever already replicates and restarts processes own it: on Kubernetes run one Uvicorn process per container and scale replicas; on a lone VM or Docker host use fastapi run --workers, sized by memory and cores.

solid answer

~40 s

The question is which layer owns replication, restarts and load balancing. On Kubernetes the cluster already does all three, so FastAPI's docs recommend one Uvicorn process per container, `fastapi run` without `--workers`, and replicas instead: each container's memory is one copy of the app, its limits and health checks describe one process, and a crash restarts one container. On a single VM or Docker host nothing above the container balances across processes, so `fastapi run --workers N` lets Uvicorn's supervisor do it. A clean team default is one image whose exec-form `CMD` omits `--workers`, with `WEB_CONCURRENCY` set only where workers are wanted. Either way, run migrations once, outside the workers.

code

dockerfile · 7 lines
dockerfile
FROM python:3.14
WORKDIR /code
COPY ./requirements.txt /code/requirements.txt
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
COPY ./app /code/app
# One Uvicorn process; set WEB_CONCURRENCY where workers are wanted
CMD ["fastapi", "run", "app/main.py", "--port", "80"]

go deeper

for a junior

Know that a FastAPI container on Kubernetes usually runs one Uvicorn process, and that workers are for running several processes on one machine.

for a middle

Explain what workers duplicate, what the cluster already does for replicas, and why the Dockerfile uses the exec form of CMD with fastapi run.

for a senior

Size the deployment from measured memory per process and pick the layer that restarts and balances; keep migrations in a single step outside the workers.

for a principal

Set the team default: one image, no --workers in the CMD, WEB_CONCURRENCY where workers are justified, plus the written conditions under which a service may deviate.

## The decision in one sentence Put replication where restarts, load balancing and memory accounting already live. FastAPI's deployment docs frame it the same way: workers are one way to get **replication**, but restarts, startup, memory and the steps before startup still have to be handled by something, and on a cluster that something is the cluster. ## On the Kubernetes cluster: one process per container A Kubernetes deployment already runs several identical containers, restarts a container whose process exits, and spreads requests across them through a service in front. Adding Uvicorn workers inside each container duplicates those jobs one level down. FastAPI's docs recommend a **single Uvicorn process per container** there, with replicas doing the scaling. What that buys you: - **Predictable memory.** Each container holds one copy of the app, so its memory request and limit describe one process and the scheduler can place replicas where they fit. - **Honest health.** Liveness and readiness checks probe the one process that serves; there is no supervisor inside the container hiding a worker that keeps dying. - **Fine-grained scaling.** Capacity changes one process at a time, and an autoscaler sees the real per-process load. - **Simpler shutdown.** The stop signal goes straight to the one server process, which runs the app's lifespan shutdown, with no supervisor in between deciding how to stop workers. ## On the single VM: workers in one process tree On one VM, or one Docker host, nothing above your process balances traffic across several copies on the same port. `fastapi run --workers N` gives you that: Uvicorn's parent binds the port once, restarts workers that die, and lets you roll workers with `SIGHUP`. FastAPI's docs name exactly these cases, a simple app on a single server or a single-host container setup, as where a process manager inside the container makes sense. Size `N` from both limits: the cores you can use for parallel Python work and the memory that `N` copies of the app need. ## One image for both A setting that works on both targets: ```dockerfile FROM python:3.14 WORKDIR /code COPY ./requirements.txt /code/requirements.txt RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt COPY ./app /code/app CMD ["fastapi", "run", "app/main.py", "--port", "80"] ``` 1. Leave `--workers` out of the `CMD`. On the cluster the image runs one process. 2. On the VM, set `WEB_CONCURRENCY=4` in the container's environment. `fastapi run` passes no worker count to Uvicorn when the flag is absent, so Uvicorn reads the variable and starts four workers. 3. Keep the **exec form** of `CMD`. FastAPI's docs insist on it so that FastAPI can shut down gracefully and lifespan events are triggered; the shell form puts a shell in front of the server. 4. Copy the requirements before the code, so a code change does not re-run the dependency install. ## Steps that must run once Migrations and other one-off preparation must not run in every worker or replica, because parallel copies would race. FastAPI's docs put them in a separate single-process step: an **init container** or a job on the cluster, and a command run before the server on the VM. Never put them in the lifespan, which runs once per worker and once per replica. ## Comparing the options | Concern | One process per container | `--workers` in one container | |---|---|---| | Restarts | the orchestrator restarts the container | Uvicorn's parent restarts the worker | | Load balancing | the cluster's service | the shared socket inside the container | | Memory per unit | one app copy | `N` app copies | | Health checks | describe one process | describe the parent and the worker that answered | | Best fit | Kubernetes and other orchestrators | single VM, single Docker host | ## Where the default bends The docs are explicit that none of this is a rule written in stone. Reasons to choose differently: - a platform that charges or schedules per container, where a few larger containers with workers are cheaper than many small ones; - an app whose containers are slow to schedule or pull, where a worker restart inside a running container can recover faster than a new container; - a model so large that only a few copies fit per machine, where the count is set by memory on either path. Write the default down for the team, and the reasons a service may deviate, so the decision is not re-argued per service.

  • Why does FastAPI's documentation tell you not to use the old Gunicorn-based FastAPI base image?
    It was built when Uvicorn could not manage and restart dead workers, so Gunicorn did that job, at the cost of extra complexity. The image is deprecated. On a cluster you build your own image with one process; where you need workers, `--workers` on `fastapi run` or `uvicorn` now does the supervising.
  • How do you set the number of replicas against the number of workers when a model dominates memory?
    Count copies, not containers: each worker and each replica loads the model once. Measure one process's memory after startup, divide the memory you can give the service by it, and spend that budget either on replicas of one process each, on the cluster, or on workers, on the VM. The core count only caps parallelism below that number.
  • Where do you put migrations if the VM runs a single container with four workers?
    Run them once before the server starts, as a separate command or a one-shot container, not in the app's lifespan, which would run them in all four workers in parallel. FastAPI's docs allow running a step in each container right before the server starts only when running it several times in parallel is harmless, such as checking that a database is ready.

  • Always run as many Uvicorn workers per container as the node has cores
  • Replicas and workers are equivalent, so the choice never matters
  • Run migrations in the lifespan so every worker is up to date
  • The shell form of CMD is fine because fastapi run handles signals
  • Workers in one container share the app's memory, so they are cheaper than replicas