skip to content

A Go handler flushes 200 OK, then its data source fails mid-stream - how do you report the failure?

level: seniorimportance: nice to knowfreq 22%

answer

  1. the status went out long ago
  2. a second WriteHeader changes nothing
  3. the body is the only channel left
  4. one panic value ends it without a stack trace

basics

~20 s

Not with a status code - the 200 is already on the wire, so http.Error would only append text to the body. Report it in the stream's own format, in an HTTP trailer, or abort with panic(http.ErrAbortHandler) so the client sees a broken response.

solid answer

~50 s

Once the first flush has gone out, the status line and headers are committed; calling `WriteHeader` again does nothing but log a superfluous-call warning, and `http.Error` just appends its message to the body a client is already parsing. So you have three real options. Design failure into the stream format - a final record or line the consumer checks, which is the only option that carries a reason. Declare a trailer before the first write and set it at the end, which is clean but ignored by many clients. Or abort: `panic(http.ErrAbortHandler)` tears the response down without logging a stack trace, leaving an HTTP/1.1 chunked body unterminated, so a correct client reports an unexpected EOF rather than a complete document. Whichever you pick, log the failure server-side with the request's identifiers - the client's copy of the story will always be poorer.

code

go · 10 lines
go
// The 200 went out at the first flush, minutes ago.
if _, err := next(); err != nil {
	log.Printf("tail %s failed after %d lines: %v", id, n, err)
	// 1. terminal record in the stream's own format
	_ = json.NewEncoder(w).Encode(map[string]string{"error": "source unavailable"})
	_ = rc.Flush()
	// 2. or break the response instead of ending it cleanly:
	// panic(http.ErrAbortHandler)
	return
}

go deeper

for a junior

Remember that the status code goes out with the headers and cannot be changed afterwards, so a failure discovered mid-stream cannot become a 500.

for a middle

Explain what http.Error and a second WriteHeader actually do after the first flush, and name the alternatives: a terminal record in the stream format, a trailer, or aborting the response.

for a senior

Show the judgment: choose the signal your consumers will really check, keep the reason in your logs, separate the three endings in metrics, and prefer buffering a bounded response so you keep proper status codes.

for a principal

Decide as a contract, not per handler: whether streaming endpoints must carry an explicit terminator, who is responsible for checking it, and which payload sizes are allowed to stream at all given the error reporting it costs.

## Why the usual error path is gone An HTTP response can say its status exactly once, and it says it the moment the header block leaves the server. In a streaming handler that happens at the first flush, potentially minutes before the failure. After that: - `w.WriteHeader(500)` is ignored and the server logs a message about a superfluous `WriteHeader` call. - `http.Error(w, ...)` calls `WriteHeader` and writes its message - so all it actually does is append a line of prose to a body the client is already parsing as data. - Changes to `w.Header()` have no effect; those bytes are long gone. A handler that streams has therefore given up the ability to fail with a status code, and that is a design consequence, not a bug to be worked around. The question is how the consumer learns that what it received is incomplete. ## Option 1: failure is part of the stream format The most useful option is to make the wire format able to express failure. If the endpoint emits one JSON object per line, then the last object can be `{"error":"source unavailable"}`, or every record can carry a type field with a terminal `error` type. This is the only option that carries a machine-readable reason, and it survives every intermediary and every client library. The cost is that consumers must actually check - a client that stops at EOF and assumes success will silently treat a truncated tail as a complete one. If you choose this, document it as part of the endpoint's contract and make the successful case terminal too, for example a final `{"done":true}` record, so "no terminator" is unambiguously an incomplete stream. ## Option 2: trailers HTTP/1.1 chunked responses can carry headers *after* the body. In Go you announce them before the first write by adding names to the `Trailer` header, then set those keys in the header map at the end; `http.TrailerPrefix` lets you add a trailer without announcing it up front. It is the protocol's own answer to exactly this problem. In practice, support is thin: many clients, libraries and intermediaries drop trailers or never expose them to application code, and over HTTP/1.1 they only exist for chunked responses. Use them when you control both ends, not as the primary signal for a public endpoint. ## Option 3: abort the response Sometimes the honest thing is to break the response. `panic(http.ErrAbortHandler)` is a sentinel: the server aborts the response to the client and, unlike any other panic value, suppresses the stack trace in the error log. On HTTP/1.1 the chunked body never gets its terminating zero-length chunk, so a correct client sees an unexpected EOF and fails; on HTTP/2 the stream is reset. That is precisely the outcome you want when the alternative is a client happily saving a half-written export as if it were whole. It gives no reason - that lives in your logs - and it is useless against a consumer that ignores transport errors, which is another reason a terminal record in the format is worth having as well. ## What not to do Do not swallow the error and return normally. A stream that ends cleanly is indistinguishable from a complete one, so a truncated result gets stored, indexed or paid out as if it were correct - and that is the expensive failure. Do not write a JSON error document into the middle of a body of a different shape. Do not retry the source silently for minutes while the consumer waits with no indication; if you retry, bound it, and make the stall visible in the stream if the format allows. ## Server side Whatever the client learns, your logs should carry the whole story: the error, how far the stream got (records and bytes), how long it ran, and the request identifiers. Metrics should separate the three endings - completed, client disconnected, source failed - because their causes and their fixes have nothing in common. On a live tail, the support engineer who reports "it just stopped" is describing option 3 working exactly as intended, and your logs are the only place the reason exists. ## Designing to avoid the choice Where it is possible, buffer or validate before the first flush. If the response is small and bounded, build it fully, then write it - you keep the ability to return a proper status, and you get `Content-Length` for free. Streaming is worth this loss of error reporting when the response is unbounded or latency matters; it is not worth it for a payload that would have fit in memory anyway.

  • What exactly does panic(http.ErrAbortHandler) do that a plain panic does not?
    Both abort the response to the client, but `http.ErrAbortHandler` is a sentinel the server recognises and it suppresses the stack trace in the server's error log. So it is the way to say "tear this response down deliberately" without filling the log with a fake crash. The client still sees a broken response - an unterminated chunked body on HTTP/1.1.
  • Why is silently returning after the error the worst option?
    Because a cleanly ended response is indistinguishable from a complete one. The consumer sees a well-formed, terminated body and stores, indexes or bills on a truncated result. Every other option - a terminal error record, a trailer, an aborted connection - at least gives a careful client something to notice; returning normally guarantees the failure is invisible.
  • When are HTTP trailers a reasonable way to report this?
    When you control both ends. You announce the trailer names in the `Trailer` header before the first write and set their values after the body, or use `http.TrailerPrefix`; over HTTP/1.1 they exist only on chunked responses. Many clients and intermediaries drop them or never surface them to application code, so for a public endpoint they are a supplement, not the signal.

saying these in an interview costs you the question

  • Calls http.Error after the first flush and expects a 500
  • Thinks WriteHeader can be called a second time to change the status
  • Returns normally after the source fails, ending the body cleanly
  • Writes an error document into the middle of a differently shaped body
  • Assumes every client will surface HTTP trailers