Grounded generation and citations

## What the feature is Most providers make retrieval-augmented generation your problem: you retrieve, you concatenate the results into a prompt, and you invent your own convention for making the model cite. Cohere's chat endpoint moves part of that inside the API. You hand it your retrieved material in a dedicated request field, and it hands back the answer *plus* a machine-readable list of which supplied document supported which stretch of that answer. Nothing about how you found the documents changes — that is still your retrieval pipeline — but the prompt assembly and the citation format become the API's job rather than yours. ## The request shape On the v2 chat call the parameters that matter here are: - `model` — a Command-family model id. - `messages` — the conversation, in the usual role/content form. - `documents` — a list of the sources the answer must be grounded in. `documents` is top-level. It is not a message, not a role, not a string you glue onto the last user turn. Two element forms are accepted: 1. **A plain string.** Simplest, and fine for a quick experiment, but you get no identifier of your own back in the citations, so you are left matching on position or text. 2. **An object with `id` and `data`.** `id` is any stable key you choose — a chunk id, a database primary key, a URL. `data` is a flat map of field names to string values, for example `{"title": ..., "text": ..., "url": ...}`. The model sees those fields as a small labelled record, so field names carry meaning; calling a field `text` or `snippet` beats calling it `f1`. In production you almost always want form 2, because the id is what makes a citation actionable in your UI. ## What comes back The response's assistant message carries the generated text in its content blocks and, separately, a `citations` array. Each citation identifies a span of the generated text and the sources that support it. The sources refer back to the documents you supplied, keyed by the ids you gave them. That is the whole contract: text in one place, attribution in another, joined by offsets and ids. Crucially, citations are *derived from what you supplied*. The model cannot cite a page it did not receive. If your retrieval returned the wrong three chunks, you get a confidently cited wrong answer — the citations prove the answer is consistent with the supplied documents, not that the supplied documents were the right ones. ## Grounded versus ungrounded With no `documents` (and no tool results), the same endpoint behaves like any other chat completion: the model answers from parametric knowledge and the citations list is empty or absent. So "grounded" here is not a mode flag you switch on; it is a consequence of having handed the call something to ground against. This is worth saying explicitly in an interview, because candidates often assume there is a `grounded=true` switch. ## Common mistakes - **Duplicating the documents into the user message.** Then you pay for the tokens twice and the model sees the same material in two shapes. Put it in `documents` only. - **Nesting structure inside `data`.** Keep it flat and stringly-typed; deep JSON is not what the field is for, and the model reads it worse. - **Random or per-request ids.** If the id is a UUID you generate at request time, the citation tells you nothing you can look up afterwards. Use the id of the thing in your store. - **Sending fifty documents because the context window allows it.** Every document is input tokens you are billed for on every turn, and dilution hurts answer quality. Retrieve, rerank, then send a short list. - **Assuming citations are mandatory.** The model will leave connective and general-knowledge text uncited, by design. ## Operational notes Document tokens dominate the cost of a grounded call, so cache or trim aggressively on multi-turn conversations rather than re-sending the whole document set on every turn. And treat retrieved documents as untrusted input: text you pulled from a wiki or a customer ticket can contain instructions aimed at the model, and putting it in `documents` rather than in the user turn does not sanitise it.