Your service is overloaded and starts shedding traffic. How do you decide between HTTP status 429, 503 and 500 for those responses, and what do the codes tell a client to do?

## Codes as instructions to the client Every failure status implicitly answers two questions: is this the client's doing, and will retrying help? Under load those answers matter more than usual because clients multiply whatever you tell them by their retry policy. ## 429 Too Many Requests "You, specifically, sent too much." The limit may be per API key, per user, per IP, per endpoint. The response should carry: ``` HTTP/1.1 429 Too Many Requests Retry-After: 30 RateLimit: limit=1000, remaining=0, reset=30 Content-Type: application/problem+json {"code":"RATE_LIMITED","scope":"api-key","limitPerMinute":1000} ``` `Retry-After` (seconds or an HTTP date) is what makes 429 cooperative — without it clients guess, and guessing badly is how a limiter turns into a thundering herd at the top of every minute. Publishing remaining-quota headers on *successful* responses too lets well-behaved clients slow down before hitting the wall. Be explicit in the body about *which* limit fired (per-key, per-endpoint, burst versus sustained), otherwise the integrator cannot fix anything. ## 503 Service Unavailable "Not you — us, right now." Legitimate triggers: - a critical dependency is down and the request cannot be served meaningfully, - deliberate load shedding when a queue or thread pool is saturated, - rolling deploy / graceful shutdown draining connections, - planned maintenance. 503 also carries `Retry-After`. The important design point is that 503 must be **cheap**. Load shedding only helps if rejecting costs far less than serving — reject at the edge, before the expensive work, ideally before acquiring a database connection. A 503 produced after a 30-second dependency timeout has already consumed the resource you were trying to protect; that is what turns overload into collapse. Related: `504 Gateway Timeout` (an upstream did not answer in time) and `502 Bad Gateway` (an upstream sent something invalid) are the proxy-flavoured cousins. Both are effectively "try again, it is not your fault", and both should be distinguishable in your dashboards from application 500s. ## 500 Internal Server Error The catch-all for defects: an unhandled exception, a null dereference, a serialization failure. Properties that matter: - **Not the client's fault** and not fixable by changing the request. If the client *can* fix it, the status was wrong — that request should have been a 4xx. - **Not usefully retryable.** The same input generally reproduces it. - **Alertable.** A 500 rate above baseline is a bug, and that should be the loudest signal in the dashboard. So never use 500 for load shedding, quota, validation, or a known-down dependency. Each of those has a code that tells the client something true. The body must stay opaque: a stable error code and a correlation/trace id, never a stack trace, SQL fragment, or internal hostname — 5xx bodies are attacker-visible and stack traces are a reconnaissance gift. ## Retry behaviour and the feedback loop The client contract should read roughly: | Status | Retry? | How | |---|---|---| | 429 | yes | honour `Retry-After`, then exponential backoff + jitter | | 503 / 504 | yes | exponential backoff + jitter, circuit breaker after N failures | | 500 | no (or once, cautiously) | surface to the caller, log with the correlation id | | 4xx (other) | no | fix the request | Two failure modes to name in an interview: **retry amplification** — three layers each retrying three times is 27 requests to a service already failing — and **synchronized retries**, which is why jitter is not optional. Budgeted retries (a cap on the fraction of traffic that may be retries) are the modern answer, along with circuit breakers that stop sending entirely for a while. ## Idempotency is the precondition for any of this Telling clients to retry is only safe if retries are safe. GET/PUT/DELETE are idempotent by definition; POST is not, so retry advice on POST requires an idempotency-key mechanism. An API that returns 503 on a payment POST and expects clients to retry, without deduplication, is asking for double charges. ## What good operations look like Separate dashboards for 4xx (client behaviour), 429 (limits firing), and 5xx (your health). Alert on 5xx rate and on 503 volume; treat sustained 429 as a capacity or customer-integration conversation rather than an outage. And make sure health checks and load balancers understand your 503s — a node returning 503 during graceful shutdown should be pulled from rotation, not restarted in a loop.