Why do many HTTP servers and load balancers cap how many requests they will serve on a single connection - for example advertising `Keep-Alive: timeout=5, max=100` - and what does a client observe when that cap is reached?

## The knob Most servers expose two connection-recycling controls: an **idle timeout** (close a connection with no traffic for N seconds) and a **maximum requests per connection**. nginx calls the latter `keepalive_requests` (its default was 100 for years and was raised to 1000 in 1.19.10), Apache calls it `MaxKeepAliveRequests`, and many application servers and proxies have equivalents. Servers may advertise both as the informational `Keep-Alive: timeout=5, max=100` header, though clients are not required to honour it and mostly treat it as a hint. ## Why cap at all **Bounded resource lifetime.** A connection is a container for state: read/write buffers, TLS session material, per-connection objects in the server framework, sometimes a thread. Small leaks that would be invisible on a short-lived connection become significant on one that lives for days. Recycling turns an unbounded leak into a bounded one. **Rebalancing.** This is the operationally important reason. Under HTTP/1.1 with keep-alive, a client that opened 50 connections to a load balancer keeps using those same 50 TCP flows, and each is pinned to whichever backend it landed on. Autoscaling adds three instances and they receive **nothing**, because no new connections are being made. The fleet looks scaled out while the old nodes stay hot. Bounding the number of requests (or the absolute age) of a connection guarantees periodic re-connection, which re-runs DNS and load-balancer selection. The same mechanism helps traffic drain away from an instance being replaced. **Config and certificate rollovers.** New TLS certificates, new routing rules or new limits typically apply at connection setup. Long-lived connections keep running under old settings until they are recycled. **Defence in depth.** A cap limits how much a single connection can do, which slightly raises the cost of some slow-drip abuse patterns and keeps a per-connection accounting bug from being unbounded. ## What the client sees The server does not abort. It serves the final permitted request completely, adds `Connection: close` to that response, and closes after the body is written. A compliant client removes the connection from its pool and opens a fresh one for the next request. The only visible symptom is a periodic handshake - and the corresponding latency spike on that one request. Problems show up when the cap is misconfigured relative to traffic: - **Cap too low for a chatty client** (say 100 requests on a client doing thousands per second per connection): you pay TLS handshakes constantly, server CPU rises, and p99 gets a saw-tooth. Raising `keepalive_requests` is the standard remedy for reverse proxies fronting internal APIs. - **Cap interacting with the idle timeout race.** Recycling driven by `Connection: close` is safe because the signal rides on a response. Recycling driven purely by an idle timeout is not: the server can close while a pooled connection sits idle and a client may write into it at the same moment. That stale-connection race is a separate problem from the request cap and is why explicit `Connection: close` is the preferred signal. ## HTTP/2 and HTTP/3 Multiplexed protocols do not use `Connection: close` - it is a forbidden header. Instead the server sends a **GOAWAY** frame naming the highest stream ID it will process, letting in-flight requests finish while refusing new ones on that connection; the client opens a new connection for subsequent work. Many managed load balancers also enforce a maximum connection *duration* for the same rebalancing reasons, and GOAWAY is how they announce it. Because HTTP/2 concentrates all of a client's traffic onto one connection, pinning is worse there, which makes periodic GOAWAY-based recycling more important, not less. ## Practical guidance Set the cap high enough that handshakes are a rounding error (thousands of requests, or a duration measured in minutes), low enough that rebalancing happens on the timescale of your autoscaling. Verify clients handle `Connection: close` and GOAWAY without surfacing errors, and watch new-connection rate as a metric - a sudden spike often means someone lowered a cap.