Why does interleaved thinking require passing earlier thinking blocks back to the model?

## What interleaved thinking is A non-interleaved thinking model deliberates once, up front, then acts. Interleaved thinking means the model can think **again after every observation**: it reads a tool result, reasons about what that result implies, and only then decides the next action. The difference shows up on exactly the tasks where an agent earns its keep. Take an airline irregular-operations agent handling a cancelled leg. It reads the disruption feed and thinks about which passengers are connection-critical. It queries seat inventory on the alternates. Now it must think *again* — the inventory came back thinner than expected, so the plan of rebooking everyone on one flight is dead and a split across two carriers is on the table. Without a thinking step after that observation, the model is choosing its next tool call reflexively, at the exact moment the situation changed. ## Why the blocks have to travel back The chat completion interface is stateless. The provider is not holding your agent's train of thought between calls; the conversation *is* the state. Everything the model will know on turn N+1 is precisely what you serialize into the request. So when your loop does the standard thing — take the assistant's turn, execute the tools, append the results, call again — whatever you dropped from that assistant turn is permanently gone from the model's view. If the thinking is what you dropped, the model now sees tool results attached to calls whose *rationale* has vanished. The observable failure modes are consistent: - **Redundant work.** It re-calls a tool because nothing in context records that it already reasoned about that data. - **Constraint amnesia.** A constraint it derived in step two ("this passenger has a minimum connection time we can't violate") existed only in the thinking, so step five violates it. - **Reversals.** It reaches a conclusion, loses the reasoning, and reasons its way to the opposite conclusion two steps later. - **Cost inflation.** Every re-derivation is thinking tokens you pay for twice. These look like model flakiness in a bug report. They are usually a harness that is throwing away state. ## Unmodified, not merely present A subtlety worth knowing, because it defeats the obvious optimization. Providers commonly require the thinking blocks to be returned **byte-for-byte as issued**, and may attach an integrity signature that is checked on the next call. Some providers return the thinking as an opaque or encrypted block you cannot read at all — you are expected to store it and hand it back. That rules out the tempting move of summarizing long thinking to save context. Rewriting a block either fails validation or degrades the continuation. Treat these blocks as **opaque state you carry**, not as text you own. ## The prompt-cache interaction Providers cache on a **prefix** basis: reuse is granted only for the longest identical leading stretch of the request. An agent loop is the ideal cache workload — each iteration is the previous request plus an appended result — and that is why long tool loops are affordable at all. Edit anything earlier in the transcript and you invalidate the cache from that point. Trimming thinking blocks from the middle of a live loop therefore does something worse than saving nothing: it destroys the reasoning chain *and* forces a full re-read of the prefix at uncached rates. Two regressions from one well-intentioned change. ## Where the context actually goes Interleaved thinking obviously grows the transcript, and on a twenty-step loop that growth is real. Two things keep it manageable. First, the retention requirement is scoped to the **live loop**. Within one assistant turn's chain of tool calls, thinking must be preserved because it is the working state. Once a turn completes and the user speaks again, providers commonly stop counting earlier turns' thinking against context — the deliberation that got you to a finished answer is no longer load-bearing. Second, when you genuinely must shrink a long-running session, the right move is a **deliberate compaction step** that rebuilds the transcript into a fresh, coherent summarized state and continues from there — not a silent trim of blocks out from under an in-flight loop. ## The design principle underneath A thinking model's reasoning is not a log. It is working memory, and it lives in a place your code controls. Anything your harness deletes, the model forgets. That reframing — the transcript is the agent's memory, and you are its memory manager — is the answer a senior interviewer is listening for, with the thinking blocks as the specific case.