How many cache breakpoints do you set, and at which layers of a prompt?

## What a breakpoint is A cache breakpoint is a marker saying "everything from the start of the request up to this point is a prefix worth storing and reusing". Providers differ in how you express it. Some require you to place explicit markers and cap how many you may use — Anthropic's API allows up to four per request, checking for a hit at each one, as of mid-2026. Others cache automatically once a prefix exceeds a minimum length; OpenAI's automatic caching kicks in above roughly a thousand tokens with no marker at all. The placement reasoning below is provider-independent: even where markers are automatic, you are still choosing *where the stability boundaries fall*, and the same layout decisions apply. ## Why more than one With a single breakpoint you get one all-or-nothing outcome: either the request matches up to that point or it does not. With several, you get graceful degradation. Suppose your prompt is system prompt → tools → policy corpus → history. If the corpus is updated but the system prompt and tools are untouched, a breakpoint after the tools still hits and only the corpus and everything after it is recomputed. Without that earlier breakpoint, the corpus change costs you the whole prefix. That is the entire argument for multiple breakpoints: each one is an insurance policy against changes in the layers below it. ## Which layers Work down the stack and put a breakpoint wherever the answer to "when does this change?" changes: - **After the system prompt and tool definitions.** These move on deploy. If your toolset is dynamic within a session, split them: static system prompt, breakpoint, tools, breakpoint. - **After the large static content** — the policy corpus, the reference document, the few-shot example block. These move on a content update, typically much less often than a session. - **After the last completed conversation turn.** This is the one that moves every turn, and it is the workhorse breakpoint in a chat application. What you do *not* do is place one after every document in a 40-document corpus. Documents in a corpus all change on the same event, so the extra breakpoints protect against nothing, and you will run out of your allowance before reaching the boundaries that matter. ## Order layers by rate of change, not by convention Breakpoint placement only works if the underlying ordering is right, and the ordering rule is: the topmost block must be the most stable thing in the prompt. Convention often violates this. Many agent frameworks assemble the request as tools first, then system prompt, because that is the order the objects were constructed in. That is fine when the toolset is fixed at deploy. It is a trap in an agent that loads tools dynamically — progressive disclosure, per-task toolsets, a plugin activating mid-session. Add one tool at turn six and every token after the tool block changes: the system prompt, the corpus, all six turns of history. The prefix collapses to whatever preceded the tools, which is nothing. Reorder so the immutable system prompt leads and the mutable tool block sits behind it with a breakpoint between them, and the same mid-session tool addition costs you only the tokens below the tools. ## A placement recipe 1. List every block with the event that changes it. 2. Group blocks that change on the same event into a tier. 3. Sort tiers slowest-changing first, and check that this sort did not get overridden by how your code happens to build the request. 4. Put one breakpoint at the end of each tier, from the top, until you run out of breakpoints or tiers. 5. Spend any remaining breakpoint on the boundary you expect to be crossed most often — usually the end of the conversation history. ## When one is enough A single-shot classification or extraction prompt with a fixed system prompt, a fixed instruction block and one variable input has exactly one boundary. Add a second breakpoint and you have added nothing except an extra thing to maintain. Complexity here should be proportional to how many genuinely independent change schedules your prompt contains — most prompts have two or three, not four. ## Common mistakes Treating breakpoints as decorations sprinkled at message boundaries; placing them *before* the block they are meant to protect rather than after it; putting one in the middle of the fastest-changing region, where it can essentially never be hit twice; and forgetting that the block ordering, not the markers, is what actually determines what is reusable. A breakpoint cannot rescue a prompt whose first line contains today's date.