HTTP/2 clients normally keep one connection per origin and may coalesce several hostnames onto the same connection. Explain when coalescing is allowed, and what a single long-lived connection per client means for a service sitting behind load balancers.

## The one-connection rule HTTP/2's specification says clients SHOULD NOT open more than one connection to a given origin (scheme + host + port). All the benefits — a warm congestion window, a shared HPACK table, coherent prioritization, low server memory — depend on traffic being concentrated. This reverses HTTP/1.1 practice, where six connections plus domain sharding was the tuning advice. ## Connection coalescing Browsers go further: they reuse one connection for *different* hostnames. RFC 9113 §9.1.1 allows this when both conditions hold: 1. **Same destination.** The new origin's DNS resolution yields an IP address the client already has a connection to. (Firefox is stricter than Chrome here about requiring a fresh resolution rather than trusting an existing certificate alone.) 2. **Authoritative certificate.** The certificate already presented on that connection covers the new host — usually a SAN entry or a wildcard such as `*.example.com` — and passes normal validation. So `img.example.com` and `api.example.com`, both behind the same CDN IP and the same wildcard certificate, share one connection and one TLS handshake. This is why domain sharding is now harmful: it either splits into several connections (extra handshakes, cold congestion windows) or coalesces anyway, giving you nothing for the added DNS lookups. Two escape hatches exist: - **421 Misdirected Request**: a server that receives a request whose `:authority` it is not configured to serve returns 421, and the client must retry on a new connection to that origin. This is the correct answer for multi-tenant edges where one certificate covers hosts served by different backends. - **The ORIGIN frame** (RFC 8336): a server sends the set of origins it is authoritative for on this connection, letting it *expand* coalescing (accept hosts the client would not have tried) or *contract* it (avoid a 421 round trip entirely). ## What breaks at scale **Load balancing becomes a connect-time decision.** An L4 (TCP) balancer hashes or round-robins once. With HTTP/1.1 and six short connections per client, imbalance self-corrects; with one HTTP/2 connection that lives for hours, whatever backend you landed on serves every request. Consequences: - **Scale-up does nothing.** New pods sit idle because no client reconnects. Autoscaling appears broken. - **Skew.** A few heavy clients (mobile app fleets, service-to-service gRPC callers) can pile hundreds of concurrent streams onto one node while its peers idle. - **Deploys hurt more.** Killing an instance drops many multiplexed sessions at once, so the retry storm is spikier. **Mitigations, roughly in order of preference:** 1. **Bounded connection age.** Have servers send a graceful two-phase GOAWAY after a maximum age (plus jitter) so clients re-resolve DNS and re-balance. gRPC exposes this directly as `MAX_CONNECTION_AGE` + grace; Envoy and most edge proxies have an equivalent drain. 2. **Layer-7 balancing.** Terminate HTTP/2 at a proxy that balances *per request* to upstream pools, so client connection stickiness never reaches the application tier. 3. **Client-side load balancing.** For internal RPC, let the client hold connections to several backends (subsetting) and spread streams across them; this is standard gRPC practice. 4. **Per-connection budgets.** Cap concurrent streams, request rate, and reset rate per connection so one connection cannot exhaust a node — the same controls that mitigate Rapid Reset also bound noisy-neighbour effects. **Blast radius**: coalescing means a single connection may carry several hostnames' traffic. If that connection stalls (flow-control deadlock, a slow TCP path, a lost packet triggering transport-level head-of-line blocking), everything on it stalls together. Splitting genuinely independent, latency-critical traffic onto its own origin — with a certificate or ORIGIN frame that prevents coalescing — is a legitimate isolation decision, and one of the few remaining reasons to use a separate hostname. ## Judgement calls to raise in an interview - Isolation versus efficiency: coalescing saves handshakes but couples failure domains. - Connection age versus handshake cost: shorter ages rebalance better but pay TLS more often; jitter avoids synchronized reconnect storms. - Where to terminate: L4 keeps state cheap but inherits stickiness; L7 rebalances per request but costs CPU and adds a hop. - Who owns retries: a drain is only invisible if clients retry the streams above the GOAWAY cutoff and have a retry budget that stops the storm from amplifying.