In Go's net/http, what breaks when you set Accept-Encoding: gzip on the request yourself?

### Two ways a Go client can end up gzipped Compression on the wire can be arranged by the library or by you, and `net/http` deliberately refuses to do half of each. **The transparent path.** When `http.Transport` sends a request it adds `Accept-Encoding: gzip` on its own initiative, provided three things are true: `Transport.DisableCompression` is false (the default), the request carries no `Accept-Encoding` header already, and it carries no `Range` header, and the method is not `HEAD`. If the server takes it up, the Transport unwraps the gzip stream for you. From the caller's side nothing is compressed at all: `resp.Body` reads plain bytes. **The explicit path.** If your request already sets `Accept-Encoding`, the Transport concludes that compression is your business. It sends what you asked for and hands back exactly what arrived. `resp.Body` is now a gzip stream, and it is on you to look at `Content-Encoding` and wrap it. The rule underneath both is consistent and worth stating in an interview in one sentence: **whoever asked for the encoding is responsible for undoing it.** ### What the transparent path changes on the response When the Transport did the decompression, it adjusts the response so nothing in it describes the compressed form you never see: * `resp.Uncompressed` is set to `true` — this is the flag that tells you it happened. * `resp.ContentLength` is set to `-1`, meaning unknown, because the header's byte count described the compressed payload and the decompressed length is not known in advance. * `Content-Encoding` and `Content-Length` are deleted from `resp.Header`. That last point catches people out in two different ways. Code that logs `Content-Length` from a response sees it vanish for compressed endpoints. Code that reads `resp.Header.Get("Content-Encoding")` to decide whether to decompress sees an empty string and correctly does nothing — which is the intent. And code that sizes a buffer from `resp.ContentLength` has to cope with `-1`. If you want to observe the compressed bytes as the server actually sent them — to measure compression ratio, to proxy them onward untouched, to write them to disk still compressed — set `Transport.DisableCompression = true` on your own Transport, or set `Accept-Encoding` on the request. Both put you on the explicit path. ### The failure it produces The symptom is abrupt and looks nothing like a compression problem. Somebody adds `req.Header.Set("Accept-Encoding", "gzip")` — often out of a reasonable-sounding wish to "make sure we get compression", sometimes copied from a curl command — and every JSON decode against that endpoint starts failing with an invalid-character error on what looks like binary garbage, or the persisted payload turns out to be a gzip file with a `.json` name. The header that was supposed to be redundant silently changed who owns the decompression. The fix is either to delete the header and let the Transport do both halves, or to do both halves yourself: ```go if resp.Header.Get("Content-Encoding") == "gzip" { zr, err := gzip.NewReader(resp.Body) if err != nil { return err } defer zr.Close() // read from zr, not resp.Body } ``` Note the conditional. A server is not obliged to compress just because you asked, so an explicit-path client must branch on what actually came back rather than assuming gzip. ### Interaction with closing and draining On the explicit path you still close `resp.Body`, not just the `gzip.Reader` — closing the decompressor does not close the underlying network stream. And if you want the connection pooled, the thing that must reach EOF is `resp.Body`; a `gzip.Reader` that stopped early because you only wanted the first record leaves compressed bytes unread underneath it. Draining, when you do it, is `io.Copy(io.Discard, resp.Body)` on the outer stream. There is a small nicety on the transparent path too: because the Transport is reading the compressed stream and feeding you the inflated one, the amount left on the wire is not the amount left in your reader. That is another reason to express "drain and close" against `resp.Body` and let the Transport reconcile the two. ### Why the Transport does not simply always decompress It could, but it would then be lying to any caller who deliberately asked for a specific encoding — a proxy forwarding bytes onward, a tool measuring what the server actually sends, a client asking for an encoding Go does not implement. Making the behaviour hinge on who set the header keeps the library out of the way of code that knows what it is doing, at the cost of one surprise for code that sets the header without meaning to take ownership.