A legal assistant cited a case that doesn't exist. How do you ground and verify its answers?

## Why the citation is the dangerous part A fabricated citation is the worst-shaped failure a language model produces. The format is highly regular, so the model reproduces it perfectly; the content is arbitrary, so it has nothing to reproduce it *from*; and the artefact carries social authority, so a reader treats it as the evidence rather than as another generated claim. The result is a string that looks exactly like proof and is not. Any design that treats "it gave a citation" as reassurance has inverted the risk. Start from that: a citation emitted by the model is a claim requiring verification, at least as much as the sentence it supports. ## Layer one — grounding Grounding means the answer is produced conditioned on source text placed in the context, with the instruction that claims must come from that text. This converts an open recall problem into a reading problem, which is a much easier one, and it gives you something to check against afterwards. Two design details do most of the work. First, the instruction must be specific about scope: answer only from the passages provided, and if they do not address the question, say so. Second, the product must have somewhere for that refusal to go. If "the retrieved authorities do not address this point" has no place in the interface, the system will produce prose instead — you have designed the gap that the model then fills. Make the not-covered response a first-class result with the next action attached, such as which sources were searched and what to broaden. Grounding is necessary and not sufficient. The right passage can sit in the context while the model still merges in a half-remembered holding from pretraining, or generalizes a narrow ruling. That is what the next two layers are for. ## Layer two — attribution that is validated, not trusted Ask for structured output where each claim carries the identifier of the passage it rests on, plus the quoted span. Then check it mechanically: - **Resolve the identifier** against an authoritative index — a real case database, your document store, the corpus you actually retrieved from. If the identifier does not resolve, the answer fails closed. This single check kills the fabricated-case class outright and it is ordinary software, not model behaviour. - **Confirm the span exists** in the resolved document, by exact or near-exact match. A resolvable case with an invented quotation is the second failure mode, and string matching catches it. - **Confirm the cited passage was actually retrieved** for this request. A citation pointing at a real document that never entered the context is evidence the model produced it from memory. A useful discipline: never let the model author an identifier freely. Have retrieval hand it opaque passage ids and require the answer to reference only those, so an invented id is a syntactic error rather than a plausible one. ## Layer three — the verification pass The final layer is a separate call whose only job is checking. Split the answer into atomic claims, hand each one the source paragraphs it cites, and ask for a verdict per claim: **supported** (the passage entails it), **unsupported** (the passage contradicts it or does not entail it), or **not addressed** (the passage is about something else). Making the check a fresh call over the source text — rather than asking the writer to re-read its own work — matters: the checker sees the claim and the passage, not the reasoning that produced the claim, so it is not defending a conclusion it already committed to. What you do with the verdicts is the design decision. On a low-stakes drafting surface, strip unsupported claims and return the rest with a note. On a filing-bound surface, any unsupported claim blocks release and the item goes to a human with the claim and the passage side by side, which makes review a matter of checking attribution rather than re-reading the whole brief. Be honest about the cost. Verification adds a call per answer and latency proportional to how much source text must be re-read, and it is imperfect — a checker can wave through a subtly over-general claim. It is a large reduction, not a proof. ## Sequencing and what to say If asked what you would do first with a limited budget, the order is: hard-resolve citations (cheapest, kills the headline failure), then make abstention real, then ground with attribution, then add the verification pass on the surfaces where an error reaches a court or a customer. Fabricated-citation incidents are usually failures of the first item, which is the least model-dependent of the four. The interview answer that lands names the three layers, insists that the citation is itself generated and must be resolved externally, and closes on where the human sits. The answer that does not land proposes a better prompt.