A reverse proxy terminates the client's connection and opens its own connection to the backend. Why does the client keeping its connection alive not give you connection reuse on the backend leg, and what has to be true at the proxy for that reuse to happen?

## The proxy is two endpoints, not a pipe It is tempting to picture a reverse proxy as a piece of tubing that a request slides through. It is not. A reverse proxy terminates the client's transport: it completes the TCP handshake, usually the TLS handshake too, parses the request, and only then acts as a **client** toward the backend, opening or borrowing a second connection of its own. Two sockets, two states, two sets of timers. Everything follows from that split. The client's connection can stay open for minutes across dozens of requests while the proxy opened and closed thirty separate backend connections. Or the reverse: the client can be a one-shot script that connects, sends, and disconnects, while the proxy served it entirely from a warm pooled connection to the backend. Neither leg governs the other. ## Why reuse on the upstream leg is not free On the client side, persistent connections are the norm and cost nothing to arrange. On the upstream side, several proxies default to a new connection per request, and there are reasons: pooling requires the proxy to track idle connections per backend, to detect ones the backend has closed underneath it, and to make sure a connection is never handed to a second request while the first is still using it. The cost of *not* pooling is paid on every single request: a TCP handshake (one round trip), and if the backend leg is encrypted, a TLS handshake (one to two more round trips plus asymmetric crypto on both ends). On a fast internal network the latency is small but the CPU and the socket churn are not, and the backend ends up with a large population of short-lived connections and sockets in post-close states. Under load that shows up as accept-queue pressure and CPU spent in handshakes rather than in your handler. ## What the proxy actually has to do Three things, and skipping any one of them silently disables reuse: 1. **Declare a pool.** Idle upstream connections have to be kept somewhere with a size limit. In nginx this is the `keepalive` directive inside an `upstream` block; other proxies express it as a connection-pool setting on the cluster or backend. 2. **Use a protocol version that supports persistence on that leg.** HTTP/1.0 upstream means close-per-response. Proxies that default to HTTP/1.0 upstream must be told to use 1.1. 3. **Strip hop-by-hop headers.** `Connection`, and the headers it names, apply to a single hop. A proxy that copies the client's `Connection` header onto the upstream request can hand the backend a `close` it never intended and tear down the pooled connection after every response. A typical nginx shape, for orientation: ```nginx upstream app { server 10.0.0.11:8080; keepalive 32; } location /api/ { proxy_pass http://app; proxy_http_version 1.1; proxy_set_header Connection ""; } ``` The `Connection ""` line is the one people forget, and its absence looks exactly like a working config — right up until you count connections on the backend. ## Sizing, and why the number is bigger than it looks The pool limit is almost always **per worker process**, not per proxy instance. Eight workers with a pool of 32 is 256 idle connections from one host, and ten such hosts is 2,560 before a single request arrives. Multiply before you set the number: the backend pays for each one in a file descriptor, a socket buffer and, for threaded servers, potentially a thread. Oversizing has a second effect. Idle connections are the ones a backend is most likely to close on its own idle timer, and a pooled connection that the backend closed a moment ago is exactly the connection the proxy is about to reuse. The more idle capacity you hold, the wider that window. The general defence is to keep the proxy's upstream idle timeout comfortably shorter than the backend's, so the proxy retires connections rather than discovering them dead mid-request. ## What to take away Count the two legs separately. Ask which one your metric is measuring, which one your timeout applies to, and whether the reuse you assume on one side actually exists on the other. Most surprises at this layer are someone reasoning about a single connection when there were always two.