skip to content

Why does http.DefaultTransport discard most connections when a Go service makes 200 concurrent requests to one host?

level: middleimportance: should knowfreq 52%

answer

  1. One zero means a default, another unlimited
  2. Nothing caps how many you open by default
  3. The per-host idle limit is tiny
  4. Two fit; the rest are closed on completion
  5. Raise the global idle cap too

basics

~20 s

Because http.Transport's MaxIdleConnsPerHost is zero by default, which means two. At peak 200 connections are open, but only two can be parked as idle for that host; the rest are closed, so the next burst dials again.

solid answer

~40 s

`http.DefaultTransport` leaves `MaxIdleConnsPerHost` at zero, and zero means `http.DefaultMaxIdleConnsPerHost`, which is 2. Nothing limits how many connections you may *open* — that is `MaxConnsPerHost`, also unset — so a burst of 200 concurrent requests opens roughly 200 connections. When each response completes, the transport tries to return the connection to the per-host idle list, finds only two slots, and closes the rest. The next burst therefore pays 198 fresh TCP and TLS handshakes, and the client host accumulates `TIME_WAIT` sockets. The fix is to clone `http.DefaultTransport` and raise `MaxIdleConnsPerHost` to roughly your steady-state concurrency against that host, raising `MaxIdleConns` (default 100, across all hosts) to match. If the server speaks HTTP/2 this rarely bites, because one connection multiplexes many streams.

code

go · 5 lines
go
t := http.DefaultTransport.(*http.Transport).Clone()
t.MaxIdleConns = 200        // global cap, default 100
t.MaxIdleConnsPerHost = 100 // was effectively 2
t.IdleConnTimeout = 30 * time.Second
search := &http.Client{Transport: t, Timeout: 2 * time.Second}

go deeper

for a junior

Know that Go's default transport keeps only two idle connections per host, and that this is a field you can raise rather than a fixed property of net/http.

for a middle

Explain the asymmetry precisely: MaxIdleConnsPerHost of zero means two, MaxConnsPerHost of zero means unlimited, and the idle cap governs reuse after a response rather than admission before a request.

for a senior

Connect the default to observed symptoms — handshake-dominated latency, TLS CPU, TIME_WAIT growth — and show how you would size the pool from measured concurrency instead of a round number.

for a principal

Frame pool sizing as shared capacity: every idle connection you hold is a socket the downstream must keep, so agree limits with the teams that own those services rather than tuning unilaterally.

## The defaults you are actually running with `http.DefaultTransport` — the transport behind `http.DefaultClient`, `http.Get`, and any `http.Client` whose `Transport` field is nil — sets a small number of fields and leaves the rest at their zero values: - `MaxIdleConns: 100` — the cap on idle connections kept **across all hosts**. - `IdleConnTimeout: 90 * time.Second` — how long an idle connection may sit in the pool before it is closed. - `TLSHandshakeTimeout`, `ExpectContinueTimeout`, `ForceAttemptHTTP2: true`, `Proxy: http.ProxyFromEnvironment`, and a `DialContext` from a `net.Dialer`. - `MaxIdleConnsPerHost` — **not set**, therefore zero. - `MaxConnsPerHost` — **not set**, therefore zero. Those last two zeros mean different things, and that asymmetry is the whole question: - `MaxIdleConnsPerHost == 0` does **not** mean unlimited. It means "use `http.DefaultMaxIdleConnsPerHost`", a package constant equal to **2**. - `MaxConnsPerHost == 0` **does** mean unlimited — there is no cap on total connections to a host. ## What happens to a 200-way burst Requests do not queue behind the idle pool. If a request needs a connection and none is idle, the transport dials a new one (subject only to `MaxConnsPerHost`, which is unlimited here). So 200 concurrent requests to one host open something close to 200 connections. That part is fine and often desirable. The damage happens on the way back. As each response is finished with, the transport offers the connection to the per-host idle list. Two fit. The other 198 are closed, and because the client initiated the close, each of those sockets enters `TIME_WAIT` on the client machine for roughly two maximum segment lifetimes. The next burst finds two warm connections and dials 198 more. The symptoms are exactly what you would predict: p99 latency dominated by handshakes rather than server time, CPU spent on TLS, a socket table full of `TIME_WAIT` entries to one address, and at high enough rates ephemeral port exhaustion. ## The three knobs, in the order they matter 1. **`MaxIdleConnsPerHost`** — how many warm connections you may keep per destination. Set it to roughly the concurrency you sustain against that host, so that a steady workload never has to redial. This is the field that fixes the churn. 2. **`MaxIdleConns`** — the global idle cap, default 100. If you raise `MaxIdleConnsPerHost` to 200 and leave this at 100, the global cap becomes the binding constraint. Raise both together, or set `MaxIdleConns` to 0 (unlimited) and let the per-host value do the limiting. 3. **`MaxConnsPerHost`** — the cap on *total* connections per host, counting dialing, active and idle. Zero means no limit. When the limit is reached, further requests **block** waiting for a connection to free up rather than failing fast, so pair any non-zero value with a per-request deadline. Use it when you must protect a fragile downstream, not as a substitute for idle-pool sizing. `IdleConnTimeout` is the fourth knob: it evicts connections that have been idle too long. Ninety seconds is a reasonable default, but it should be **shorter** than whatever idle timeout the server or an intermediary enforces, otherwise you will hand out connections the other side has already closed. ## Configuring it without breaking everyone else Start from a clone rather than mutating the global: ```go t := http.DefaultTransport.(*http.Transport).Clone() t.MaxIdleConns = 200 t.MaxIdleConnsPerHost = 100 t.IdleConnTimeout = 30 * time.Second client := &http.Client{Transport: t, Timeout: 5 * time.Second} ``` `Clone` gives you the default's proxy, dialer and HTTP/2 settings so you change only what you intend to. ## The HTTP/2 caveat On an HTTP/2 connection, many requests are multiplexed as streams over a single TCP connection, so the per-host idle limit stops being interesting: one warm connection can carry your whole burst. `ForceAttemptHTTP2` is true in `http.DefaultTransport`, so an HTTPS call to a server that negotiates h2 gets this automatically. The two-connection default therefore bites hardest on plaintext HTTP/1.1 and on HTTPS peers that do not offer HTTP/2 — which includes plenty of internal services and load balancers. ## How to size it honestly Do not pick a big round number and move on. Measure concurrency against that specific host (in-flight requests, or `httptrace`'s `GotConn` reporting `Reused`), set the idle pool near the sustained value rather than the peak, and remember that every idle connection is a file descriptor on your side and a socket on the server's. A pool sized for the peak of a rare burst is capacity the downstream must hold open for you all day.

  • What is the difference between http.Transport's MaxConnsPerHost and MaxIdleConnsPerHost?
    `MaxConnsPerHost` caps the total connections to a host, counting dialing, active and idle ones; when it is reached, new requests block until one frees up. `MaxIdleConnsPerHost` caps only how many finished connections are kept warm for reuse and never blocks anything. Zero means unlimited for the first and two for the second.
  • Why should MaxIdleConnsPerHost and MaxIdleConns be raised together?
    `MaxIdleConns` is the cap across all destinations, defaulting to 100. If you set the per-host value to 200 but leave the global at 100, the global cap evicts connections before the per-host allowance is used, so the tuning silently does less than you expect. Raise both, or set `MaxIdleConns` to zero for no global limit.
  • Does this two-connection default matter when the peer speaks HTTP/2?
    Much less. HTTP/2 multiplexes many concurrent requests as streams over one TCP connection, so a burst does not need a connection each and the idle pool barely gets exercised. The default bites on HTTP/1.1 traffic — plaintext internal services and peers that do not negotiate h2.
  • How should IdleConnTimeout relate to the server's own idle timeout?
    It should be comfortably shorter. If your client is willing to keep a connection for 90 seconds but the server or an intermediary closes idle connections at 60, the pool will keep handing out connections the peer has already torn down, producing sporadic failures on the first write. Set the client's value below the shortest timeout on the path.

The car park has two reserved bays but no barrier at the entrance. Two hundred cars can drive in; when they leave, only two may stay parked and the rest are towed, so tomorrow everyone drives in again.

saying these in an interview costs you the question

  • Thinks MaxIdleConnsPerHost of zero means unlimited
  • Believes the idle limit caps how many connections may be opened
  • Raises MaxIdleConnsPerHost but leaves MaxIdleConns at 100
  • Confuses MaxConnsPerHost with the idle pool size
  • Assumes tuning http.DefaultTransport in place is harmless
  • Sizes the pool from a rare peak rather than sustained concurrency