Prompt caching

## What "prefix caching" means here When you call Anthropic's Messages API, the request is flattened into one long token sequence in a fixed order: the `tools` array first, then the `system` parameter, then the `messages` array in order. Prompt caching lets the service keep the internal state it computed for a leading run of that sequence and reuse it on a later call, so those tokens do not have to be processed again. The crucial consequence is that the unit of caching is a *prefix* — a contiguous run starting at the very beginning of the request — not an arbitrary block. `cache_control` does not mean "cache this block". It means "the cacheable prefix ends here". ## The mechanics of a breakpoint You mark a breakpoint by attaching `"cache_control": {"type": "ephemeral"}` to a content block: an entry in the `system` array, a tool definition in `tools`, or a content block inside a message. `ephemeral` is the only type; an optional `"ttl": "1h"` extends the default lifetime. On a request, the API hashes the prefix up to each breakpoint and looks it up. A hit means those tokens are billed at a small fraction of the normal input price and skip most of the prefill work; a miss means the prefix is processed normally and then written to the cache for next time. The response reports both outcomes in `usage` via `cache_read_input_tokens` and `cache_creation_input_tokens`. Because matching is exact, *any* difference before the breakpoint is a miss — a changed word in the system prompt, a reordered or edited tool schema, an injected timestamp, a different model. There is no fuzzy or partial matching on content; the only flexibility is that the API will also check for hits on somewhat shorter prefixes near your breakpoint (roughly the preceding twenty content blocks), which is what makes a moving breakpoint in a growing conversation still land on the older cached prefix. ## The layout rule Sort your request from most stable to most variable, then cut: 1. **Tools** — JSON Schema definitions change only when you ship code. They sit at the front of the prefix whether you like it or not, so an unstable tool list poisons everything behind it. 2. **System** — persona, policies, and long static context such as a manual, a codebase excerpt, or a pasted contract. 3. **Early messages** — few-shot examples, an uploaded document turn, or completed conversation turns. 4. **Everything volatile** — the current user question, freshly retrieved RAG chunks, per-user metadata, the current time. Put the breakpoint at the boundary between 3 and 4. The classic beginner failure is interpolating something variable into the system prompt — "Today is {date}", "User: {id}" — which silently makes every request a fresh cache write. That material belongs in the first user message, after the breakpoint. ## Multiple breakpoints A request may carry up to four breakpoints. Use them for genuinely different stability tiers, for example one after the tool definitions, one after the static system context, and one or two rolling markers in a growing conversation. With two breakpoints, a later call can read the matched earlier portion at cache-read price and write only the new segment, so you pay the write premium on the delta rather than the whole prompt. ## Scope and lifetime Cache entries are private to your organization and keyed on the exact prefix plus the model, so different models never share an entry. Entries live five minutes by default, and every hit refreshes that clock, so steady traffic keeps a prefix warm indefinitely. Prefixes below a minimum size — on the order of a thousand tokens, higher for the smaller model tiers — are simply not cached, and no error is raised. ## Verifying it Don't assume; measure. Send the same prefix twice and read `usage`. The first call should show a non-zero `cache_creation_input_tokens`; the second should show a comparable `cache_read_input_tokens` with `input_tokens` reduced to just the tail. If the second call still shows creation rather than read, something before your breakpoint changed between the two requests.