Go's http.Server has no max-connection setting: how do you cap an ingest tier that OOMs holding thousands of keep-alive connections?

## The shape of the failure An ingest endpoint sits behind a load balancer that keeps a pool of upstream connections open. Request rate is unremarkable. Memory climbs anyway, plateaus high, and eventually the instance is killed. This is the failure mode that keep-alives make possible: the number of connections a Go server holds is chosen by its callers, not by its own throughput. ## Step one: prove it is connections Do not start tuning. Take a heap profile of live memory and read the top entries. ``` (pprof) top flat flat% sum% 0.71GB 47.0% 47.0% bufio.NewReaderSize 0.68GB 45.1% 92.1% bufio.NewWriterSize ``` A profile dominated by `bufio` buffers allocated on the `net/http` serving path says: each accepted connection is carrying its own read and write buffering, and there are too many connections. That is a very different diagnosis from a profile dominated by your own types, which would say a handler is retaining data. The whole point of looking first is that the two failures feel identical from the memory graph and have no overlap in their fixes. ## What a connection costs Per accepted connection, roughly: * a serving goroutine with its own growable stack; * a read buffer and a write buffer on the order of a few kilobytes each; * TLS record buffers and session state, if the listener is encrypted; * whatever the handler holds while a request is in flight; * a file descriptor, and kernel socket buffers outside your heap entirely. None of it is large. Ten thousand of them is. ## Step two: cap it at the listener `http.Server` has **no field** that limits simultaneous connections. There is no `MaxConns`. The cap must live one layer down, in the `net.Listener` you hand to `Serve`: ```go ln, err := net.Listen("tcp", ":8080") … srv := &http.Server{Handler: mux} log.Fatal(srv.Serve(limited)) // limited wraps ln and stops accepting past N ``` `golang.org/x/net/netutil.LimitListener(ln, n)` is the standard wrapper: its `Accept` blocks once `n` connections are live and resumes when one closes. A hand-rolled version is a listener holding a buffered channel as a semaphore, acquiring before `Accept` returns and releasing when the wrapped connection is closed — the release-exactly-once detail is the part people get wrong. ## Understand what the cap does to clients This matters more than the mechanism. A blocking accept cap does not refuse anyone. Connections past the limit sit in the kernel's accept queue; when that fills, the kernel starts dropping or resetting them. From a client's point of view overload therefore looks like *connect latency, then an obscure network error* — not like a server saying no. Nothing in your process logs it, because your process never saw those connections. The alternative posture is to accept and shed: let connections in and answer over-capacity requests with an explicit status so the balancer can see the instance is saturated and route elsewhere. That costs the memory of accepting, but overload becomes legible. Which of the two you want is a real decision, not an implementation detail. ## Step three: shrink the per-connection worst case Lower `MaxHeaderBytes` from the 1 MB default to what your API genuinely needs. It does not change the steady-state cost of a well-behaved connection, but it bounds what a hostile one can make you buffer, which is what turns "thousands of connections" into an arithmetic you can defend rather than an open-ended risk. ## Step four: measure the idle/active split Wire the `ConnState` hook to counters and watch how many of the held connections are idle rather than serving. On an ingest tier behind a balancer, the honest answer is usually "almost all of them". That reframes the problem: you are paying memory for a pool somebody else sized. Shrinking the balancer's upstream pool, or spreading connections across more instances, may be a cheaper fix than anything inside your process — and it is a conversation you can only have with the number in hand. ## The last resort Turning keep-alives off entirely does collapse the connection count, and it is the wrong first move: it converts a memory problem into a handshake and CPU problem for every client, and hurts tail latency across the board. It is defensible only on genuinely tiny instances, which is roughly what the `net/http` documentation says about it. ## The answer in one breath "Heap profile first to prove it is per-connection buffering and not a handler leak; cap at the listener because `http.Server` has no connection field; lower `MaxHeaderBytes`; measure the idle/active split with `ConnState`; and decide deliberately whether over-capacity should look like queueing or like an explicit refusal."