skip to content

When does Go's net/http server frame a response as chunked instead of sending a Content-Length header?

level: middleimportance: should knowfreq 38%

answer

  1. the server puts off sending headers
  2. two ways to say where a body ends
  3. the first flush forces the decision
  4. unknown length leaves only one option

basics

~20 s

Whenever the body length is still unknown when the headers go out. A handler that returns with a short body buffered gets Content-Length; one that flushes first, or overruns the buffer, gets chunked framing on HTTP/1.1.

solid answer

~50 s

The server delays sending the header block as long as it can. If the handler returns with the whole body still inside the response buffer - a few kilobytes - the length is known, so the server adds `Content-Length` and no chunking happens. If the handler flushes, or writes more than the buffer holds, the headers must go out before the body is finished, so on HTTP/1.1 the server falls back to `Transfer-Encoding: chunked` and frames each write for you. You never write chunk sizes yourself. You can also set `Content-Length` explicitly when you genuinely know the size, and then the response is not chunked - but writing more bytes than you declared is an error and the surplus is dropped. HTTP/2 has no chunked encoding at all; the same handler simply produces more data frames.

code

go · 12 lines
go
// Returns with everything buffered: server adds Content-Length.
func short(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "ok")
}

// Commits headers before the body is done: chunked framing.
func stream(w http.ResponseWriter, r *http.Request) {
	rc := http.NewResponseController(w)
	fmt.Fprintln(w, "line 1")
	rc.Flush()
	fmt.Fprintln(w, "line 2")
}

go deeper

for a junior

Know the two ways an HTTP/1.1 response can say where the body ends, and that Go picks one for you. Remember that you never write chunk sizes by hand.

for a middle

Explain when the decision is made - at header serialisation - and the three things that make the length unknown by then: flushing, overrunning the response buffer, and streaming by construction.

for a senior

Be able to diagnose a non-streaming endpoint from its response headers alone, and to say why declaring a Content-Length you cannot honour exactly is worse than letting the server chunk.

for a principal

Own the rule for the codebase: which endpoint families may declare a length, which must stream, and how that is verified in tests rather than discovered by a client that hangs waiting for a body that already ended.

## Framing is decided at header time An HTTP/1.1 response must tell the client where the body ends. There are two ways: a `Content-Length` header giving the exact byte count up front, or `Transfer-Encoding: chunked`, where the body is a sequence of length-prefixed chunks terminated by a zero-length one. Go's server picks between them at the moment the header block is serialised, and the rule is simply: use the length if it is known by then, otherwise chunk. ## Why a short handler gets Content-Length The server buffers the response body for the first few kilobytes precisely so that the common case is decidable. A handler that renders some JSON and returns has finished before anything went out; the server knows the body is 412 bytes, writes `Content-Length: 412`, and sends everything in one shot. That is cheaper on the wire than chunk framing and lets the client show a progress bar, so it is the preferred outcome. ## What pushes it into chunked Three things make the length unknown at header time: - The handler **flushes**. The first flush commits the headers, and at that moment the handler has not finished writing, so no length can be stated. - The handler **overruns the buffer**. Writing a large body without ever flushing still forces the server to start sending before the end, so the headers go out length-less. - The handler explicitly announces trailers, or the response is otherwise streamed by construction. In all of those, the server writes `Transfer-Encoding: chunked` and takes care of the chunk framing itself: each write or flush becomes one or more chunks with a hexadecimal size prefix, and when the handler returns the server writes the terminating zero-length chunk. Handler code never sees any of that - you write bytes, the server frames them. Setting `Transfer-Encoding` by hand in the header map is not how you opt in; flushing is. ## Declaring the length yourself If you genuinely know the size before you start - a file you already stat'ed, a buffer you already built - set `Content-Length` yourself and the response will not be chunked, even if you flush along the way. That is a promise, though. Write more bytes than you declared and the extra write fails with an error and the surplus is not sent; return having written fewer, and the server has already told the client to expect more, so the client sees a truncated response. Only declare a length you can honour exactly. ## What a streaming endpoint should not do A live tail cannot know its length - it may run for hours - so it must not declare one. Concretely: set the `Content-Type` you want, do not set `Content-Length`, flush after each unit, and let the chunked framing happen. Setting `Content-Type` matters more than usual here, because with sniffing the server only inspects the first bytes it gets, and a stream whose first line does not look like the eventual content type gets mislabelled. ## HTTP/2 and beyond Chunked transfer encoding is an HTTP/1.1 mechanism. Over HTTP/2 there is no `Transfer-Encoding: chunked` - the body is a sequence of DATA frames and the end is signalled by a flag - so the same handler streams without any chunk framing at all. Go's HTTP/2 response writer supports flushing the same way, and your handler code does not change. This is one of the reasons to write handlers against `http.ResponseWriter` and flushing rather than against the wire format: the framing is the server's business and it differs by protocol version. ## How to see which one you got Run the endpoint under `curl -N -v` (or dump the response headers however you like) and look at the header block. `Content-Length: N` means the server buffered the whole body and your streaming did not happen. `Transfer-Encoding: chunked` on an HTTP/1.1 response means the headers went out early, which is what a stream is supposed to look like. It is a fast, unambiguous check that costs one command, and it distinguishes a server that is not streaming from a client that is not displaying.

  • What happens if a handler sets Content-Length and then writes more bytes than it declared?
    The surplus write fails with an error and those bytes are not sent; the server logs that the handler wrote more than the declared `Content-Length`. Declaring a length is a promise about the exact byte count, so only set it from something you have already measured - a stat'ed file, a fully built buffer - never from an estimate.
  • Does a streaming handler need to set Transfer-Encoding: chunked in the header map?
    No. Framing is the server's job: it decides between a length and chunked encoding when it serialises the headers, and it writes the chunk sizes and the terminating zero-length chunk itself. Your side of the contract is to not set `Content-Length` and to flush. Hand-writing chunk framing into the body produces a doubly framed, corrupt response.
  • How does this change when the same handler is served over HTTP/2?
    Chunked transfer encoding does not exist in HTTP/2; the body is carried in data frames and the end is flagged, so there is no `Transfer-Encoding` header to see. The handler code is identical - flushing still pushes bytes out promptly. Framing is a property of the protocol version the connection negotiated, not of your handler.

saying these in an interview costs you the question

  • Thinks the handler must set Transfer-Encoding: chunked itself
  • Writes chunk size prefixes into the response body
  • Sets Content-Length from an estimate before streaming
  • Believes chunked encoding exists in HTTP/2
  • Says Go always chunks responses of unknown size even after Content-Length was set