You own a platform where dozens of services call each other over HTTP. How would you set policy for which 5xx status code each service returns under overload, dependency failure and deploy, and how should client retry behaviour and the Retry-After header follow from that policy?

## The problem: codes drift into noise In a large estate, each team picks its own 5xx conventions. One service returns 500 when its database is slow; another returns 503 for a null pointer; a third passes a downstream 502 straight through. Within a year the platform's aggregate 5xx metric means nothing: you cannot tell defects from saturation, alerting is either deafening or blind, and client retry logic has no principled basis. The fix is to standardise what each code *asserts* and to make the client library act on those assertions. ## The vocabulary **503 — I am refusing on purpose, and it is temporary.** This is the workhorse. It covers load shedding when a queue or pool crosses a threshold, an instance draining ahead of shutdown, an open circuit breaker in front of a sick dependency, and planned maintenance. The service is healthy enough to answer; it is choosing not to do work. Attach `Retry-After` whenever you have a defensible number: remaining drain time, breaker cooldown, queue depth divided by drain rate. **500 — something is broken that should not be.** Reserve it for unanticipated defects. If 500 only ever means "a bug fired", then the 500 rate is a defect rate and can page a human with a low false-positive rate. The moment capacity problems are also 500, that signal is destroyed. **502 and 504 — the intermediary's codes.** These describe a hop that failed between two components, and only the component in the middle can honestly assert them. When service A calls service B and B times out, A should not return 504 to its own caller — A is not acting as B's gateway from the caller's point of view. A decides what its own availability story is: shed with 503 if it cannot serve without B, or degrade and return a partial 200 if the contract allows. ## Choosing shed over collapse A fast 503 is strictly better than a slow 504. Once a service is saturated, every additional accepted request makes latency worse for the requests already in flight; the queue grows, timeouts fire at the edge, and the work is discarded after the cost has been paid. Explicit admission control — bounded queues, concurrency limits, shedding at a threshold with 503 — converts an unbounded latency collapse into a bounded, well-labelled error rate. This is the single most consequential item in the policy, and it is why the platform should make shedding easy: a shared middleware, not a per-team project. ## Client policy that follows The policy is only real if the shared client implements it. 1. **Retry 503**, honouring `Retry-After` as a floor, adding randomised jitter, and capping the delay so a synchronous path is not parked indefinitely. 2. **Retry 502** when the failure occurred before any response bytes arrived, which strongly implies the request never took effect. 3. **Do not blind-retry 500 or 504.** Both mean application code ran and may have committed. Retry only when the method is idempotent or the request carries an idempotency mechanism. 4. **Enforce a retry budget.** Per-request attempt caps are insufficient: during a partial outage, every client retrying three times triples load exactly when the system can least afford it. A budget expressed as a fraction of successful traffic — retries allowed only while they remain a small share of requests — bounds amplification. 5. **Propagate deadlines.** Each hop passes its remaining time budget downstream, so no service starts work that its caller has already abandoned. ## Deploy behaviour Rollouts are the biggest avoidable source of 502. The required sequence is: fail the health check or deregister first, let in-flight requests drain, then signal the process. Skipping deregistration means the proxy keeps forwarding to a socket that is closing, and the estate sees 502s on every deploy — which teams then learn to ignore, which is worse than the errors themselves. ## Observability contract Standardising codes pays off in dashboards. Split 5xx by code and by emitter (edge, mesh sidecar, application), alert on 500 rate as a defect signal and on 503 rate as a saturation signal, and treat sustained 502 as a lifecycle or connection-management defect. Require a request id propagated across hops so any 5xx can be walked end to end. Where an error budget or SLO exists, decide explicitly whether deliberate load shedding counts against availability — it usually should, or shedding becomes a way to hide capacity problems. ## The judgement being tested There is no single right answer here; the interviewer is listening for whether you treat status codes as an interface between operations and clients rather than as cosmetic detail, whether you can articulate the shed-versus-collapse tradeoff, and whether you recognise that client behaviour must be centralised for any of it to take effect.