How would you keep failure precedence uniform across an edge gateway and dozens of independently built backend services?

On one service, failure precedence is an ordering question. Across a fleet it becomes an organisational one, because the property you are trying to hold is not "this endpoint is correct" but "no endpoint disagrees". ## Uniformity is the security property A policy that hides whether records exist is only as good as its least conforming endpoint. If most surfaces answer not-found for a record the caller may not see, and one answers forbidden, that one is an oracle: an attacker uses it to confirm identifiers and then uses them everywhere else. The same is true of ordering. If one service checks existence before rights, its status codes enumerate its data regardless of what the other services do. This is why the fleet-level question is not "what is the best order" but "how do we make one order inevitable". ## What the contract has to pin down 1. **The order of the stages** - credential, coarse rights, binding, lookup, fine-grained rights - stated as behaviour, not as a code structure. 2. **Which resource classes hide existence**, by class rather than by endpoint, so the decision is reviewable and consistent. 3. **The exact shape of a refusal**: status, body fields, which headers are present, and that the hidden case is byte-identical to the absent one. 4. **Which paths are anonymous** - probes, metrics, credential issuance, public assets - as an explicit, short list. 5. **Where the true reason lives**: log fields, a correlation id returned to the caller, and metrics that separate masked refusals from genuine absences. 6. **Who may deviate, and how** a deviation is recorded, because some surfaces legitimately need a different answer. ## Where each decision belongs | Decision | Best home | Why there | |---|---|---| | Credential validity and the `401` shape | Shared edge, re-checked in the service | One answer for every caller; the service must not trust reachability | | Coarse rights per route | The service | Only the service knows its own operations | | Existence masking per resource class | The service, after the lookup | Only the stage holding the record can make the two cases identical | | Failure body format | Shared library or template | Identical shape is what makes masking believable | | Anonymous path list | Shared configuration, reviewed | It is the hole in the credential boundary | Note the first row. An edge that enforces the credential is not sufficient on its own: traffic can reach a service by another route - internal calls, a mesh path, a misconfigured ingress, a test harness - so each service still refuses on its own. The edge makes the common case uniform; the service makes it true. ## Make the shape inheritable, not memorable - Ship a service template that already has the chain wired in the agreed order, so a new service inherits it without a decision. - Put the failure-response construction in one shared component, so the body cannot drift per team. - Add a conformance check to the delivery pipeline that probes the agreed negative paths on every service, rather than relying on each team to remember. - Make the anonymous path list a reviewed artefact with named owners, since it is the one place the boundary is deliberately open. ## What you trade away - **Diagnosability.** A legitimate user who cannot tell "no access" from "wrong URL" opens a support ticket instead of fixing their own request. - **Client affordance.** An integration that could have prompted "request access" now sees only an absent resource. - **Support and incident cost.** Answering "why can this user not see this" needs an internal tool, because the API deliberately no longer answers it. - **Onboarding friction.** Every new engineer meets a surprising `401` on a typo URL, and the reasoning must be written down somewhere they will find it. ## Failure modes to watch for - The anonymous path list grows by prefix until it covers more than anyone intended. - The edge short-circuits with a different body than the services use, so the edge's refusals are distinguishable from theirs. - A new service is written from scratch instead of the template and quietly checks existence first. - Masking is applied to the credential failure as well, and clients lose the challenge that tells them to authenticate. - The true reason is dropped from the logs alongside the wire response, leaving nobody able to reconstruct what happened. The judgement to demonstrate here is that this is a standardisation problem with a security payoff, and that a policy which depends on every team remembering it is a policy that has already failed somewhere you have not looked yet.