Why does http.Client.Do sometimes fail with a bare EOF after an idle period?

## The race Go's HTTP transport maintains a pool of idle connections per host and reuses them, because a fresh TCP dial plus TLS handshake per request is expensive. The far end also has an opinion about idle connections: an origin server, a reverse proxy, or a load balancer sitting between you will close one after its own idle window, and it does so unilaterally. Those two clocks are independent, and there is no protocol handshake for 'I am about to close this'. So there is always a window in which the client picks a connection out of its pool, writes a request into it, and the peer's FIN is already on the way. The client writes successfully - the local socket accepts the bytes - then reads and gets end-of-stream. There is nothing else to report, so the wrapped cause is `io.EOF` and the message is bare: ``` Get "https://api.example.com/v1/orders": EOF ``` No `connect:`, no address, no syscall - that bareness is the fingerprint. ## Why you only sometimes see it The transport tries to hide this. Its documented rule is that it retries a request after a network error only when the connection had already been used successfully, and the request is idempotent and either has no body or has `Request.GetBody` defined. Idempotent here means `GET`, `HEAD`, `OPTIONS` or `TRACE`, or a request carrying an `Idempotency-Key` (or `X-Idempotency-Key`) header. And it only retries when no bytes of the response have arrived yet - once a response has started, retrying could double-apply work. So a plain `GET` usually heals invisibly, and you never learn the race happened. A `POST` built from an arbitrary `io.Reader` body cannot be rebuilt, so the transport gives up and hands you the EOF. That asymmetry is why the same job sees the error on some endpoints and never on others. ## Recognising it in a postmortem The shape is distinctive, and it is worth being able to state it from memory: - It hits the **first request after an idle gap** - the opening call of a nightly reconciliation run, the first request after a lull, the first call of each burst. - It **succeeds immediately** on a retry, with no backoff needed. - The error text is **just `EOF`**, with no address or syscall wording. - It does **not correlate** with anything on the peer's side: their logs show no request, their error rate is flat, their latency is flat. Contrast that with a real outage. A dead peer gives you `connect: connection refused` or `no such host`, it persists across attempts, and the peer's own metrics agree with you. A saturated peer gives you 5xx statuses with a nil error, not an EOF. Getting this distinction wrong is how a team spends a morning accusing a partner of instability. ## What to do about it Three responses, in order of how much they belong at the call site: **Recognise it.** `errors.Is(err, io.EOF)` matches straight through the `*url.Error` wrapper, because `url.Error` implements `Unwrap`. Recording that classification separately from timeouts and refused dials is what lets the next person see the pattern rather than a wall of identical log lines. **Make the request replayable.** If the body comes from a `*bytes.Reader`, `*bytes.Buffer` or `*strings.Reader`, `http.NewRequest` sets `Request.GetBody` for you, and the transport can rebuild and re-send the request itself. A body streamed from an arbitrary reader cannot be rebuilt, so the error surfaces to you instead. **Fix it in the pool, not the call site.** The durable cure is making the client's own idle-connection lifetime shorter than whatever closes connections upstream, so the client discards a connection before the peer does. That is a transport-pooling decision rather than an error-handling one, and it is where the permanent fix lives. ## The one thing not to do Do not answer this by wrapping every call in a blind retry-everything loop. It hides the diagnosis, it turns a benign race and a real outage into the same event, and on a non-idempotent request it is a correctness problem rather than a resilience one.