How do you resolve conflicts when several guardrails judge one LLM response?

## Composition is where guardrail systems actually break Individually each rail is easy to reason about. The incidents come from their interaction: a rail that redacted text another rail was about to judge, a rewrite that resurrected a forbidden phrase, a timing-out classifier that silently let everything through for two hours, or a reask loop that spent six model calls before returning nothing. A principal-level answer is about the resolution policy, not the individual checks. ## Give rails distinct verbs The first structural move is to stop treating every rail as a boolean. Three verbs cover almost everything: - **Block** — terminate the turn and return a refusal. Reserved for hard lines. - **Transform** — modify the content and continue: redact a span, mask an identifier, strip a link. - **Annotate** — attach a label or score, change nothing, feed dashboards and review queues. Once the verbs are distinct, precedence follows naturally: any block wins over any transform, and transforms are applied in a declared order. A rail author now has to state what kind of rail they are building, which is itself a useful forcing function. ## Order matters because rails mutate If a redaction rail rewrites a customer name to a placeholder and a groundedness rail then scores the redacted text, the second rail is judging something the model never produced. Sometimes that is what you want; often it is not. Two disciplines help: run non-mutating judgement rails against the *original* text and mutating rails in a fixed, documented sequence; and re-run the cheap deterministic checks after the last mutation, because rewriting is exactly how a cleared violation returns. ## Short-circuit or evaluate everything Short-circuiting on the first block is cheapest and lowest latency. Evaluating all rails even after a block costs more but tells you *how many* rails a request tripped, which is the signal that distinguishes an ordinary policy miss from an adversarial probe. A reasonable compromise: short-circuit in the user path, evaluate fully on a sampled fraction of traffic and asynchronously on blocked turns. ## Fail-open versus fail-closed, per rail Rails are software and will time out, throw, and depend on services that go down. The system needs a declared answer for each rail, not one global default. A rail enforcing a legal or safety hard line fails closed — no answer beats an unchecked one. A stylistic or advisory rail fails open with an alert, because a dependency blip should not take the product down. Write the choice next to the rail, alarm loudly on every fail-open event, and test the behaviour deliberately, because it is only exercised during incidents. ## Repair loops between rails When a rail's on-fail action is to reask the model with the violation described, two rails can push in opposite directions: the rewrite that satisfies the competitor rail trips the tone rail, and the fix for that reintroduces the competitor. Cap total attempts across the whole pipeline rather than per rail, and define the terminal state explicitly — normally a canned refusal on the permitted path. Also notice the cost: each reask is another generation plus another full rail pass, so a nominally cheap policy can multiply per-turn cost several times over on the tail. ## False positives compound If five rails each pass legitimate content 99% of the time and their errors are roughly independent, about 5% of good responses are blocked or mangled. Nobody feels this when adding the fifth rail, because they measure only their own. Track the *pipeline's* aggregate false-positive rate as a first-class number, attribute blocks per rail, and require anyone adding a rail to state its expected firing rate and to justify the marginal harm prevented against the marginal legitimate traffic lost. Rails should also be retired; a rail that has not fired on a true positive in a quarter is a latency and false-positive tax. ## Ownership and governance The organisational half matters as much as the mechanism. Rails accumulate from many teams, each locally reasonable. Without one owner for the composed policy — its precedence order, its verbs, its aggregate error budget — you get contradictory rules and no one who can say what the system will do with a given input. Version the rail set as a unit, review changes as policy changes, and keep a single test corpus of should-pass and should-block cases that runs on every change. ## The honest caveat All of this makes the pipeline predictable and reviewable. It does not make it a guarantee. Composed probabilistic rails are still probabilistic, and an adversary adapts to the ensemble much as they adapt to a single check. Precedence and fail-closed defaults buy you determinism about *your own* behaviour, which is exactly what an incident review needs, and they layer over the structural containment that provides the actual boundary.