Where do you place input, output and tool-call guardrails around an LLM agent?

## Why placement is the whole question A guardrail is a check plus a placement, and teams that get the check right and the placement wrong ship a system that blocks the wrong things at the wrong time. Interviewers ask this because the answer reveals whether you have drawn the loop for yourself or only used a library. ## The three seams **Input rails** run on the incoming turn before the model has committed to anything. Their advantage is that a blocked request produces no generation at all — nothing to leak, nothing to pay for, no partial answer to retract. Their limit is that they judge intent from the user's words, and intent is often invisible until the answer exists. For an in-car voice assistant scoped to navigation and vehicle questions, an input rail is the natural home for topic scoping: legal, medical and competitor-comparison requests are deflected on the way in. **Output rails** run on the finished response. They exist because the system prompt is a request, not a constraint — a model that was told to stay on vehicle topics will still occasionally give a confident legal opinion. The output rail is the last thing between the model and the user, so it is where a hard policy line belongs. Its costs are real: it adds tail latency to every turn, and it interacts badly with streaming, because by the time the rail has a full response to judge, the user may have read half of it. Teams resolve this either by not streaming rail-gated surfaces, by streaming into a buffer that is only revealed at the end, or by running incremental checks over chunks and accepting weaker guarantees. **Tool rails** wrap the function call in both directions. Outbound, the rail inspects the *arguments the model chose* — the seam where an intention becomes an effect, and the last possible moment to stop a message going to the wrong recipient or a write hitting the wrong record. Inbound, the returned result is content from the outside world that is about to be pasted into the model's context, so it deserves the same suspicion as user input. This inbound direction is the one candidates most often forget, and it is where indirect-injection content arrives. ## Concurrency and tripwires Running an input rail strictly before generation is simple and costs you the rail's latency on every turn. Agent SDKs offer an alternative: start the model and the input guardrail at the same time, and if the guardrail trips, cancel the in-flight run. This hides the rail's latency behind generation on the overwhelming majority of turns that pass. The price is that a tripped guardrail may fire after tokens have already been produced and possibly streamed, so the client must be able to discard partial output and swap in the refusal. If your UI cannot roll back what the user has already seen, concurrent evaluation buys you nothing and costs you a visible flicker of forbidden content. ## Layering, not choosing The seams are not alternatives. A topic policy is usually expressed at all three: scoping on input so most requests never reach the model, instructions in the system prompt so the model cooperates, an output rail as the enforcement backstop, and a tool rail so the model cannot route around the conversation by calling something. Each layer catches what the previous one lets through, and each has a different false-positive profile — input rails see the least context and over-block most. ## Latency, cost and the honest limits Rails that are themselves model calls multiply your per-turn cost and latency, so the usual design is cheap-and-deterministic first (allowlists, pattern checks, schema conformance) and model-based judgement only for what genuinely needs it. Say plainly in an interview that rails are probabilistic filters, not a security boundary: they raise the cost of a violation and catch the ordinary case, but a determined adversary adapts to them, and the guarantees that hold are the deterministic ones — a tool that does not exist cannot be called, a credential the agent does not hold cannot be used. Rails complement that containment; they do not substitute for it. ## The failure the rail cannot see One more placement subtlety: a rail only judges what passes through its seam. A model that never emits forbidden text but arranges a tool call whose *result* is the violation has produced clean output at the output rail. That is precisely why the tool seam exists as a distinct placement rather than as a special case of output checking.