Cohere

Send texts plus a model and an input_type to POST /v2/embed. The response groups vectors by type: a float request returns them under embeddings.float, one vector per input text, in the order you sent them.

Send a top-level documents array alongside messages in the v2 chat call. Each entry is a plain string or an object with an id and a data map. The reply's message.citations then link answer spans to those documents.

A Cohere v2/rerank call sends a model name, one query string, and a documents list of plain strings, plus an optional top_n. It returns a results array ordered by relevance, each entry carrying the document's original index and a relevance_score.

Command A is Cohere's current flagship chat model with a 256K-token context, above the earlier Command R+ and Command R at 128K and the small Command R7B. IDs carry a date suffix; undated aliases follow the newest snapshot, so production should pin the dated ID.

Cohere's Embed models are asymmetric: input_type tells the model what the text is for, and it conditions the vector accordingly. Corpus chunks go in as search_document, user queries as search_query, so a short question lands near the long passage that answers it.

Each citation carries start and end character offsets into the generated text plus the sources that support that span. Slice the raw answer with text[start:end] and join each source to the document id you supplied. Offsets index the untouched text, so apply them before any escaping or trimming.

Rerank returns a reordered list, so each result must say which input it came from. The index is the document's position in the request's documents array, which is how you join back to your own records without paying to send the text back.

Cohere spells nucleus and top-k sampling as p and k, not top_p and top_k. Both narrow the candidate pool the next token is drawn from; k defaults to 0, which disables top-k. Temperature is separate and defaults to 0.3.

Cohere streams typed events rather than uniform chunks: message-start opens the turn, content-start/content-delta/content-end wrap each content block, and message-end closes it with finish_reason and usage. You accumulate the text from content-delta events.

int8 returns one signed byte per dimension, about four times smaller than float32. binary packs one bit per dimension into bytes, so a 1024-dimension vector becomes 128 values — roughly thirty-two times smaller. You can request several types in one call.

citation_options.mode selects how much work the model spends on attribution. ACCURATE produces higher-quality citations at extra latency, FAST produces them more cheaply and quickly with somewhat coarser attribution, and OFF suppresses citations entirely so you get plain grounded text.

It truncates the document to that token budget (4096 by default) and scores only the kept portion; the overflow is invisible to the model. The v1 API instead split long documents into chunks and kept the best chunk's score, which v2 dropped.

Use a multilingual rerank model such as rerank-v3.5, which covers 100-plus languages and scores a query in one language against documents in another. You send one mixed candidate list to one call — no per-language routing and no translation step.

Cohere Command models ship beyond Cohere's own API: through hyperscaler catalogues such as Amazon Bedrock, Amazon SageMaker and Azure AI Foundry, and as container images deployed inside your own VPC or on-prem under a commercial licence. Auth, model IDs and version currency all change with the channel.

Chunk the corpus, send batches within the documented per-call text limit of 96, keep input_type constant, set truncate deliberately, and drive concurrency with exponential backoff on 429. For very large corpora, use the asynchronous Embed Jobs API over an uploaded dataset instead.

Uncited spans are text the model did not attribute to any document you supplied — connective phrasing, framing, or claims drawn from parametric knowledge. Citations are never guaranteed to cover the whole answer, so treat an uncited claim as unsupported rather than as a bug.

Treat Rerank as an optional enhancement, not a dependency: give it a timeout smaller than the remaining request budget and, on timeout, 429 or 5xx, serve the vector-recall order instead of failing. Log every degradation so the silent quality loss stays visible.

Rerank is billed in search units, where one unit covers a single query against up to 100 documents. A smaller top_n changes nothing: every document you send is still scored, so the cost lever is the candidate count, not the number of results returned.

Do not choose one. Request both types in a single embed call — the token bill is the same — then serve a compressed index for first-pass recall and rescore the top candidates with the higher-fidelity vectors. Validate the recall loss on your own eval set.

Images are passed to Cohere's embed endpoint as base64-encoded data URIs rather than as remote URLs or file uploads. Because one multimodal model produces both text and image vectors in a single shared space, a text query can retrieve images directly.

You run the search yourself. Cohere's v1 chat accepted a connectors list naming a managed web-search connector; v2 drops connectors, so web grounding becomes an ordinary tool-calling loop where your code performs the search and returns the results, which the model then cites.

Treat a vendor's language list as a starting hypothesis, not a guarantee. Command A is marketed at 23 languages and the earlier Command R line at ten key business languages, but only a task-specific eval with native reviewers tells you whether quality, safety behaviour and token cost hold in that market.

When attribution has to outlive the vendor. Cohere's documents-and-citations contract is specific to its chat API, so a service that must run the same grounding across several model providers, or needs citation rules the API does not expose, is better served by attribution it owns.