Why do backend services use a connection pool instead of opening a new database connection for every request, and what goes wrong in production when the pool is sized incorrectly?

## Why a connection is expensive A database connection is not a lightweight object - establishing one involves - a **TCP handshake**, - **TLS negotiation** if encrypted, - **authentication** against the database's user/password or certificate store, - and on the server side the allocation of **session state**. PostgreSQL, for instance, forks an entire OS process per connection, each consuming several megabytes of memory and its own share of CPU scheduling overhead; MySQL spins up a thread per connection with similar (if lighter) costs. Doing all of that fresh for every incoming HTTP request - which might complete in a few milliseconds - means the connection setup cost can dwarf the actual query time, and it does not scale: a spike of concurrent requests would mean a spike of concurrent connection setups, hammering the database with authentication and process-creation work instead of query work. ## What a pool actually does A **connection pool** solves this by decoupling 'a connection exists' from 'a request is using it right now.' At startup, the pool opens and authenticates a fixed (or bounded, elastic) number of connections and keeps them alive. When a request needs the database, it borrows an already-open connection from the pool, runs its queries, and returns the connection to the pool rather than closing it - so the expensive setup work happens once per connection's lifetime, not once per request. Pools exist at multiple layers: | Layer | Placement | Examples | |---|---|---| | Application-level pools | sits inside each service instance | `HikariCP` for JVM apps, node-postgres's `Pool`, SQLAlchemy's engine pool | | A centralized external pooler | sits between many application instances and the database, multiplexing a large number of client connections onto a much smaller number of actual database backend connections | like `PgBouncer` or `ProxySQL` | This is especially important for Postgres, where each backend process is expensive, and for serverless/Lambda-style architectures where dozens of ephemeral function instances would otherwise each try to hold their own pool. ## When the pool is too small Sizing the pool correctly is the operational crux, and both directions fail badly. If a pool is **too small**, once all connections are checked out, new requests must wait in a queue for one to free up; under sustained load this shows up as - growing request latency, - thread-pool starvation in the application (worker threads blocked waiting on a connection rather than doing work), - and eventually pool-checkout timeouts and 5xx errors, even though the database itself may have spare capacity. This is a very common self-inflicted outage: a service is technically fine on CPU/memory but appears 'down' because every worker is stuck waiting for a DB connection that never frees up (often because a few slow queries are holding connections for far longer than normal, starving the rest). ## When the pool is too large If a pool is **too large** - or, more commonly, many pools across many service instances are each individually sized generously - the aggregate connection count can exceed the database's configured ceiling (Postgres's `max_connections`, commonly in the hundreds by default) or simply overwhelm its available memory and CPU, since each open connection consumes server-side resources regardless of whether it's actively running a query. Symptoms include - the database refusing new connections outright ('too many connections' errors), - increased context-switching overhead as the database scheduler juggles more concurrent backends than CPU cores can usefully run, - and general throughput degradation even for connections that are already established. This failure mode is especially easy to trigger in autoscaling environments: adding more application replicas to handle load, each opening its own pool of, say, 20 connections, can multiply total DB connections faster than anyone intended, tipping the database over exactly when load is highest and resilience matters most. ## Sizing it as a shared budget The fix is to reason about the pool size as a **shared budget**: total connections across all application instances should stay comfortably under the database's real capacity, with headroom for administrative/maintenance connections and other consumers (batch jobs, analytics). A common pattern is a much smaller per-instance pool size than intuition suggests - HikariCP's own guidance, echoing benchmarking work often summarized as 'pool size should relate to the number of CPU cores available to the database, not the number of application threads,' is a deliberate correction against the instinct to just make the pool 'big enough.' A centralized pooler like `PgBouncer`, run in **transaction-pooling mode**, is frequently used precisely to decouple the number of application-side logical connections (which can be in the thousands across many services) from the number of actual database backend connections (kept in the tens), absorbing the mismatch without either starving requests or exhausting the database.