Besides the processes serving client connections, a relational database server runs several dedicated background processes. Name the main ones and say what each is responsible for.
answer
- Backend per connection; maintenance is shared
- Log writer flushes WAL; commit still needs its own record durable
- Background writer trickles; checkpointer sweeps
- Stats/auto-analyze keeps plans honest
- Vacuum/purge reclaims dead versions, throttled
basics
~20 sTypically: a log/WAL writer that flushes the transaction log, a background writer that trickles dirty data pages out of the buffer pool, a checkpointer that periodically writes all dirty pages to bound recovery time, statistics collection and auto-analyze, and a vacuum/purge daemon reclaiming dead row versions.
solid answer
~50 sA server process is dedicated to each client connection, but most maintenance is done by shared background processes so no single query pays for it. - **Log / WAL writer** — pushes transaction-log buffers to durable storage so commits do less work themselves. - **Background writer** — continuously trickles dirty pages from the buffer pool to disk so a query needing a free buffer usually finds a clean one. - **Checkpointer** — periodically forces all dirty pages as of a point in time to disk, bounding how much log must be replayed after a crash. - **Statistics / auto-analyze** — samples tables and refreshes the planner's statistics so plans stay sensible as data changes. - **Vacuum / purge daemon** — reclaims space held by dead row versions and keeps internal housekeeping current. Most engines also run archiving or log-shipping, a supervisor that restarts children, and replication senders/appliers. The pattern is the same everywhere: durability on the commit path, everything else amortised in the background.
go deeper
Name log writer, background writer, checkpointer, statistics and vacuum/purge, with one sentence each on what they flush or maintain.
Explain why each exists in terms of latency and batching, and note that a backend does the work itself when a background process falls behind.
Tie the roster to observable symptoms and metrics, and distinguish which work is on the commit path (log flush) from which is amortised.
Discuss the design principle: durability on the critical path, everything else amortised under a throttle, with instance-wide scheduling decisions no single session could make.
## Two kinds of work A database server does two very different things. It answers queries — parse, plan, execute, return rows — and it maintains itself: getting data onto durable storage, keeping memory usable, reclaiming space, keeping the planner informed. The first is inherently per-connection. The second is shared, periodic, and I/O-bound. So engines split them. Each client connection is handled by its own server process or thread (a *backend*), and the maintenance work lives in a small set of long-lived background processes started with the instance. Names differ per engine; the roles are near-universal. ## The roster **Log writer (WAL writer).** Every change is first described in the transaction log. Backends write log records into a shared buffer; the log writer pushes those buffers to disk and issues the durability call. A committing transaction still needs its own commit record durable before it can report success, but much of the log has usually been flushed already, so the commit finds less work to do. Group-commit behaviour lives here: many commits share one flush. **Background writer.** Data pages live in a shared buffer pool. Modifying a row dirties its page in memory; the page is not written immediately. If nothing wrote dirty pages in the background, a query that needs to load a page and finds only dirty candidates would have to write one out itself — foreground I/O in the middle of a user query. The background writer trickles dirty pages out ahead of demand so evictions usually find clean buffers. **Checkpointer.** A checkpoint establishes a point from which recovery can start: all changes before it are guaranteed on disk, so log before it need not be replayed (and can be recycled). The checkpointer performs that sweep on a schedule or after a volume of log, ideally spreading the writes over the interval rather than dumping them at once. **Statistics collection / auto-analyze.** The optimizer chooses plans from statistics: row counts, value distributions, correlations. A background component accumulates activity counters and triggers sampling of tables whose contents have changed materially, so plans track reality without anyone running a manual command. **Vacuum / purge daemon.** Under multi-version concurrency, updates and deletes leave older row versions behind. A daemon finds versions no transaction can still need and makes their space reusable, alongside related housekeeping. It is deliberately throttled so maintenance does not swamp the user workload. **Supporting cast.** A supervisor/monitor process starts the others and restarts the instance if one dies unexpectedly; an archiver copies completed log segments for backup and point-in-time recovery; replication senders stream log to replicas and appliers replay it; some engines have a dedicated error-log collector, a job scheduler, and a pool of generic worker processes that parallel queries borrow. ## Why this division exists Three reasons. 1. **Latency.** Users feel foreground I/O. Moving flushing off the query path turns a per-query cost into a background one. 2. **Amortisation.** Writing one page per transaction is far more expensive than writing a batch of pages once, and one log flush can commit many transactions at once. Background processes exist to batch. 3. **Global scope.** Choosing which pages to evict, when to checkpoint, or which table needs fresh statistics requires an instance-wide view no single connection has. ## What follows for practitioners When these processes fall behind, the work does not vanish — it reappears on the query path. Backends start writing their own dirty pages, commits wait on log flushes, plans go stale, dead versions accumulate and scans get slower. That is why background-process metrics are read as an early-warning system: the interesting number is not how busy they are, but how often a *foreground* session had to do their job instead.
- Why not have each connection flush its own dirty pages when it is done?Because the cost lands on the user's latency and it loses all batching. One transaction may dirty a page that ten others will dirty again seconds later, so writing per transaction multiplies the I/O. A background process can wait, coalesce repeated modifications into one write, order writes sensibly for the storage, and choose its moment. The only thing that genuinely must be durable at commit is the log record.
A restaurant kitchen: cooks (backends) serve orders, while dishwashers, stock rotation and cleaning staff (background processes) keep the line supplied. When the support staff falls behind, cooks start washing pans mid-service and every order slows down.
saying these in an interview costs you the question
- Saying committed data pages are written to their table files at commit time
- Confusing the log writer with the checkpointer
- Believing the statistics daemon rewrites data rather than sampling it
- Assuming background processes exist per connection