You own a user-facing API with a 500 ms latency target that fans out to several downstream services, some of which call further services. How would you assign timeout and deadline budgets across that call graph?

## Budgeting is subtraction, not repetition The common failure is picking one plausible number and configuring it everywhere. Budgets are allocated top-down from the only number that exists for a real reason: what the user (or the calling contract) will tolerate. Everything else is derived. ```text 500 ms user target - 40 ms edge overhead (TLS, auth, parsing, serialization) - 40 ms own work + response assembly - 40 ms reserve (cleanup after timeout, jitter, safety margin) = 380 ms available for downstream work on the critical path ``` The reserve matters more than people expect: post-timeout cancellation and cleanup happen *after* the budget is spent, and without slack the API blows its own target while tidying up. ## Allocate by graph structure Walk the call graph and budget the **critical path**. - **Sequential** calls divide the remaining budget in series: auth 40 ms, then profile 120 ms, then recommendations 180 ms, with 40 ms slack. - **Parallel** calls each receive the same remaining budget, because they overlap; the hop is bounded by the slowest, and a fan-out of ten with a 200 ms slice costs 200 ms, not 2 s. - **Nested** hops receive what is left when they are called, not a fixed constant — which is exactly why a deadline is propagated rather than a duration configured. Size each slice from measured behaviour: a dependency's p99 (including its own internal retries) plus margin. A cap below its p99 manufactures failures for requests that would have succeeded; a cap far above it wastes budget that a sibling could have used and delays failure detection. If the sum of realistic p99s along the critical path exceeds the target, no configuration fixes it — the answer is caching, precomputation, parallelizing sequential hops, dropping a dependency, or renegotiating the target. ## Enforce the deadline on the server side too A deadline propagated but not enforced is documentation. Each service should read the incoming deadline, check it **when the request is dequeued** (so time spent queued is charged), and reject expired requests instantly with a distinct "deadline exceeded" outcome. This is among the cheapest and most effective load-shedding mechanisms available: under overload the queue drains itself of work nobody is waiting for, instead of the server spending its capacity producing answers that will be discarded. Services should also cap the deadline they will accept. A caller handing down 60 s to a service whose work is always sub-second is either buggy or abusive, and honouring it ties up capacity. ## Classes of traffic, not one global number Interactive requests, asynchronous jobs and bulk backfills have genuinely different budgets. Model them as named classes (for example interactive 500 ms, background 30 s, batch minutes) carried with the request, so a shared downstream service can prioritize and shed by class. Without classes, protecting interactive latency forces you to give batch work the same tiny budget, and it fails constantly. ## Retries, hedging, and degradation live inside the budget Retries consume the same allocation: a hop with 180 ms might attempt twice at ~80 ms with jittered backoff, or once at 180 ms — a decision the retry loop makes from the remaining time rather than from a hard-coded count. Hedging likewise shares the deadline, never extends it. Decide per dependency what expiry *means*: - **Critical** — its failure fails the request. Give it the generous slice. - **Optional** — recommendations, badges, personalization. Give it a short slice and a default value on expiry, so the response degrades rather than fails. - **Fire-forward** — analytics, audit. Move it off the request path entirely into a queue with its own lifetime, so it never competes for the user's budget. This is usually where the largest wins are: much of a fan-out is optional, and treating it as such buys the critical path a bigger slice. ## Guardrails and review Guard against the classic inconsistencies: a downstream timeout larger than the caller's remaining budget (guaranteed wasted work), a connect timeout longer than the whole request budget, retry counts multiplied across layers, and any unbounded call at all — clients without a default timeout should fail a build or a lint rule, not a review. ## Measure and iterate Export, per hop: p50/p99 latency, timeout rate, deadline-expired-on-arrival rate, and headroom (slice minus observed p99). A hop whose headroom is negative is a scheduled incident; a hop whose headroom is enormous is holding budget hostage from the critical path. Re-derive the allocation whenever the topology changes, and treat the budget as an interface contract between teams: each service publishes the deadline it needs and the deadline it promises, so the graph can be checked end to end rather than negotiated incident by incident.