skip to content

What does httputil.ReverseProxy send the client when the upstream is unreachable, and how do you change it?

level: seniorimportance: should knowfreq 46%

answer

  1. the default is a bare 502
  2. plus one line in the error log
  3. one hook for both failure sources
  4. too late once headers are out
  5. join it against the upstream's access log

basics

~10 s

By default it logs a proxy error through ErrorLog and writes 502 Bad Gateway with an empty body. Set ErrorHandler to own that response instead; it also receives any error returned by ModifyResponse.

solid answer

~50 s

If the round trip fails — connection refused, DNS failure, TLS error, a cancelled request context — there is no upstream response to copy, so `httputil.ReverseProxy` logs the error through `ErrorLog` (the standard logger when it is nil) and writes `502 Bad Gateway` with no body. Setting `ErrorHandler func(http.ResponseWriter, *http.Request, error)` replaces that: you own the status, the body and the log line, so you can emit a JSON error, return `503` with `Retry-After` for a known drain, and log the method and path alongside the error. The same handler receives any non-nil error returned by `ModifyResponse`, which runs for every upstream response whatever its status; returning an error there closes the upstream body and routes to `ErrorHandler`. The limit worth naming: once the response headers have gone out, streaming failures cannot become a status, and the proxy aborts the connection instead.

code

go · 13 lines
go
proxy := &httputil.ReverseProxy{
	Rewrite: func(r *httputil.ProxyRequest) { r.SetURL(upstream) },
	ModifyResponse: func(res *http.Response) error {
		if res.StatusCode == http.StatusUnauthorized {
			return fmt.Errorf("upstream rejected credentials: %d", res.StatusCode)
		}
		return nil
	},
	ErrorHandler: func(w http.ResponseWriter, r *http.Request, err error) {
		log.Printf("proxy %s %s: %v", r.Method, r.URL.Path, err)
		http.Error(w, "upstream unavailable", http.StatusServiceUnavailable)
	},
}

go deeper

for a junior

Remember that when the upstream cannot be reached the proxy answers on its own with 502 Bad Gateway and logs the error, rather than passing anything through from the upstream.

for a middle

Explain the two ways an error reaches ErrorHandler — a failed round trip and a non-nil return from ModifyResponse — and what each hook is allowed to change about the response.

for a senior

Demonstrate the operational judgment: distinguishing client cancellation from upstream failure, choosing 502 versus 503 with Retry-After, and correlating your proxy log with the upstream's access log to place the fault.

for a principal

Own the error contract of the gateway itself: which upstream conditions become which status for every caller, what is logged, and what pages, so services behind the proxy do not each invent an answer.

## The default A reverse proxy has exactly one hard case: the upstream produced nothing. `ReverseProxy` handles it in the smallest way that is still correct. When the round trip returns an error, it logs a proxy-error line — to `ErrorLog` if you set one, otherwise the standard logger — and writes `502 Bad Gateway` with an empty body. That is a reasonable default and a poor production answer. The status is the same whether the upstream refused the connection, the DNS name did not resolve, TLS failed, or the client hung up mid-request; the log line has no request identity in it; and the body tells a caller nothing. ## Taking it over `ErrorHandler func(http.ResponseWriter, *http.Request, error)` replaces the whole behaviour. It is called with the response writer, the request, and the error, and from that point the response is yours: - **Distinguish causes.** A cancelled inbound context means the client went away, not that the upstream is unhealthy; that should not look like a dependency failure on your dashboards. A refused dial is a real upstream problem. - **Choose the status deliberately.** `502` says the upstream misbehaved, `503` plus `Retry-After` says come back — which is what you want during a planned drain, because it tells clients how to behave rather than making them guess. - **Log with identity.** Method, path and any request id, so the line can be joined against the upstream's own access log. - **Shape the body.** Services calling you through the proxy will parse whatever you emit; an empty body forces every caller to guess. ## ModifyResponse feeds the same handler `ModifyResponse func(*http.Response) error` runs whenever the upstream returned a response *at all*, whatever its status — this is not a 2xx-only hook. Returning `nil` forwards the response, including any headers you changed. Returning an error closes the upstream body and hands the error to `ErrorHandler`, or to the default 502 if you set none. So one function decides what a failure looks like whether it originated in the transport or in your own inspection of the response. That is what makes `ModifyResponse` more than a header rewriter: it is where you convert an upstream response you consider unacceptable into your gateway's error contract, instead of forwarding it and hoping. ## The limit: headers already sent Once the status line and headers have been written to the client, nothing can change them. If the connection to the upstream dies halfway through copying the body, the client has already been told `200 OK`, and `ErrorHandler` cannot rescue it — the proxy aborts the connection rather than pretending the truncated body was complete. Two consequences follow. First, `ErrorHandler` handles connection-establishment and response-inspection failures, not mid-stream ones. Second, if a caller must be able to tell truncation from success, that has to live in the payload — a length, a terminator, a trailer — and not in the status code. ## Reading two logs side by side This is where the `ErrorHandler` earns its keep as a diagnostic. Line the proxy's log up against the upstream's access log for the same window: - **Proxy logged an error, upstream logged nothing.** The request never arrived. Look at name resolution, the target URL, the dial path and network policy — not at the upstream's handler code. - **Upstream logged a 200, client saw an error.** The failure is on the way back: `ModifyResponse` rejected the response, or the body copy died mid-stream. - **Upstream logged a 500, client saw a 502.** The upstream answered; your gateway translated it. That is `ModifyResponse` territory, and whether the translation is right is a policy question. - **Proxy logged a cancellation, upstream logged a completed request.** The client left before the answer arrived. Slow upstream, impatient caller — not an availability incident. Without a proxy-side log carrying the request path, all four look identical from the outside, which is why leaving `ErrorHandler` nil hurts most at three in the morning rather than in review.

  • When is ModifyResponse called, and what happens if it returns an error?
    It is called whenever the upstream produced a response, whatever the status — not only for 2xx. Returning nil forwards the response with any changes you made. Returning an error closes the upstream response body and hands the error to ErrorHandler, or to the default 502 if none is set.
  • Why can ErrorHandler not rescue a failure halfway through the response body?
    Because the status line and headers have already reached the client. There is no status left to change, so the proxy stops the response and aborts the connection; the client sees a truncated body rather than a fabricated success.
  • The proxy logged an error but the upstream's access log has no matching entry. What does that tell you?
    The request never reached the upstream — resolution, dial, network policy or TLS. Investigate the target URL and the path between proxy and upstream, not the upstream handler. Had the upstream logged a 200, the failure would be on the response side instead.
  • Should a cancelled client request be logged the same way as a refused upstream?
    No. A cancelled inbound context means the caller disconnected, which is a client-side event; a refused dial is a dependency failure. Separating them in ErrorHandler keeps your error rate meaningful and stops a page firing for impatient callers.

saying these in an interview costs you the question

  • Thinks an unreachable upstream surfaces as a 500 from the handler
  • Believes ModifyResponse runs only for successful responses
  • Expects ErrorHandler to fix a failure that began mid-body
  • Treats client disconnects as upstream availability incidents
  • Leaves ErrorHandler nil and has no proxy-side log to correlate