skip to content

Why does r.ParseForm leave r.PostForm empty, with no error, after middleware has read r.Body?

level: seniorimportance: nice to knowfreq 45%

answer

  1. who read it first
  2. no rewind on a connection
  3. zero bytes parse cleanly
  4. that is why there is no error
  5. capture it, then hand back a reader

basics

~20 s

r.Body is a one-shot stream over the connection. Once middleware drains it with io.ReadAll, ParseForm reads zero bytes, parses an empty form and returns nil. Buffer the bytes and reassign r.Body with io.NopCloser before calling the next handler.

solid answer

~40 s

`r.Body` is an `io.ReadCloser` reading straight off the connection — there is no rewind and no `Seek`. If a logging or signature-checking middleware calls `io.ReadAll(r.Body)` and passes the request on unchanged, the handler's `r.ParseForm()` reads an already-exhausted stream, parses zero bytes into `r.PostForm`, and returns a nil error, so the failure is completely silent: the values are just missing. The fix is to put the bytes back — read them once, then set `r.Body = io.NopCloser(bytes.NewReader(raw))` before calling `next.ServeHTTP`. The same rule bites inside a single handler: once you have decoded JSON from `r.Body`, a later `ParseForm` sees nothing, so an endpoint that accepts both a form post and a JSON body must branch on `Content-Type` first rather than trying one and falling back to the other.

code

go · 13 lines
go
func withRawBody(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		r.Body = http.MaxBytesReader(w, r.Body, 1<<20)
		raw, err := io.ReadAll(r.Body)
		if err != nil {
			http.Error(w, "cannot read body", http.StatusBadRequest)
			return
		}
		verifySignature(r.Header.Get("X-Signature"), raw)
		r.Body = io.NopCloser(bytes.NewReader(raw)) // downstream can read again
		next.ServeHTTP(w, r)
	})
}

go deeper

for a junior

Remember that the request body can be read only once. If something before your code has already read it, you get nothing, so avoid reading it twice in the same handler.

for a middle

Explain why the failure is silent: an exhausted stream yields zero bytes, zero bytes parse into an empty form, and an empty form is not an error. Show the io.NopCloser plus bytes.NewReader restore.

for a senior

Diagnose it end to end from a production symptom of empty fields, name the middleware classes that cause it, cap the buffered read, and design the endpoint to branch on Content-Type rather than attempting two parsers in sequence.

for a principal

Decide where raw-body buffering is allowed to live at all: which routes may pay the memory cost, whether signature verification belongs in the edge proxy instead, and what test harness the team standardises on so handler tests exercise the real middleware chain.

## r.Body is a stream, not a buffer On the server side, `r.Body` is an `io.ReadCloser` that pulls bytes off the TCP connection as you read them. `net/http` does not buffer the whole body for you, and there is no `Seek`, no `Rewind`, and no second copy hiding on the request. Whatever reads it first gets the bytes; everything after that reads end-of-stream. ## Why the failure is silent `r.ParseForm()` on a `POST` with `Content-Type: application/x-www-form-urlencoded` reads the body and parses it as a query-style string. Reading an exhausted body yields zero bytes, and parsing zero bytes yields an empty `url.Values` — which is not an error. So `ParseForm` returns `nil`, `r.PostForm` is empty, `r.FormValue("email")` returns `""`, and the handler concludes the client sent nothing. This is the shape of the bug: no panic, no error in the logs, just fields that are blank in production and fine in the unit test that never installed the middleware. The usual culprits are middleware that wants the raw bytes: - request logging or audit trails - HMAC or webhook signature verification - schema validation done ahead of the handler - a metrics or tracing wrapper measuring payload size All of them read the body, and none of them can put it back unless they are written to. ## The fix: buffer and reattach ```go raw, err := io.ReadAll(r.Body) if err != nil { http.Error(w, "cannot read body", http.StatusBadRequest) return } r.Body = io.NopCloser(bytes.NewReader(raw)) next.ServeHTTP(w, r) ``` `bytes.NewReader` gives a `*bytes.Reader` over the captured bytes, and `io.NopCloser` adapts it to `io.ReadCloser` with a no-op `Close`. Downstream code reads normally and never knows the difference. Two caveats: - **Cap it first.** `io.ReadAll` on an unbounded body is exactly how a middleware becomes a memory amplifier. Wrap with `http.MaxBytesReader(w, r.Body, n)` before reading, and answer 413 on an `*http.MaxBytesError`. - **Do it selectively.** Buffering every body on every route to serve one webhook endpoint costs allocation on all of them; scope the middleware to the routes that need it. ## The same trap inside one handler An endpoint that accepts both an HTML form post and a JSON body cannot try one and fall back to the other. If it decodes JSON first and the request was actually a form post, `ParseForm` afterwards finds an empty stream. Curiously the reverse order appears to work — calling `ParseForm` on a request with `Content-Type: application/json` does not consume the body at all, because `ParseForm` only reads bodies it recognises as urlencoded — but relying on that is relying on a side effect. Branch explicitly on the media type instead: ```go ct, _, _ := mime.ParseMediaType(r.Header.Get("Content-Type")) switch ct { case "application/json": // json.NewDecoder(r.Body).Decode(&in) case "application/x-www-form-urlencoded": // r.ParseForm() then read r.PostForm default: // 415 Unsupported Media Type } ``` That also gives you a clean 415 for anything else, instead of a mysterious empty struct. ## Testing for it A table test that drives the handler directly with `httptest.NewRequest` will never reproduce this, because it exercises the handler without the middleware and with a fresh body per case. Run the table through the composed `http.Handler` — the router with its middleware attached — using `httptest.NewRecorder`, and the empty-field case shows up immediately. Reusing one `*http.Request` across two subtests will also reproduce it for the wrong reason: the body of the first case is gone by the second, so build a new request per case. ## Related one-shot bodies The rule generalises. A client's `resp.Body` is equally one-shot, a retry that resends a request needs the body regenerated (which is what `http.Request.GetBody` exists for), and any `io.Reader` you hand to a decoder is consumed by it. "Who reads the stream first, and does anyone need it afterwards" is the question to ask of every layer in the chain. ## Interview-ready summary One-shot stream; the second reader gets zero bytes; `ParseForm` turns zero bytes into an empty form and a nil error, so nothing surfaces. Buffer with `io.ReadAll` under a `MaxBytesReader` cap, reattach with `io.NopCloser(bytes.NewReader(raw))`, and branch on `Content-Type` rather than trying two parsers in sequence.

  • Why does calling r.ParseForm before decoding a JSON body appear to work?
    Because `ParseForm` only reads the body when the `Content-Type` is `application/x-www-form-urlencoded` (or on the multipart path). With `application/json` it parses just the query string and leaves `r.Body` untouched, so a later `json.NewDecoder(r.Body)` still sees everything. It works by accident, not by contract — branch on the media type explicitly.
  • What is the risk of a middleware that calls io.ReadAll on every request body?
    It buffers the entire body in memory before the handler ever runs, so a client streaming a huge upload turns one request into a large allocation, and many concurrent ones into an out-of-memory. Wrap the body in `http.MaxBytesReader` first, answer 413 past the cap, and apply the middleware only to the routes that genuinely need the raw bytes.
  • Why does a handler unit test miss this bug entirely?
    Because it usually calls the handler function directly with a fresh `httptest.NewRequest` per case, so the body has never been read and no middleware is in the chain. Drive the composed `http.Handler` — router plus middleware — with `httptest.NewRecorder` instead, and build a new request for every table case, since one request's body cannot serve two.

saying these in an interview costs you the question

  • Expects ParseForm to error when the body is already consumed
  • Thinks net/http buffers the request body for repeated reads
  • Tries to rewind r.Body with a Seek call
  • Calls io.ReadAll on an uncapped request body in middleware
  • Decodes JSON first and falls back to ParseForm on failure
  • Reuses one *http.Request across several table-test cases