How do you re-anchor a drifting system-prompt constraint mid-conversation?

## The problem re-anchoring solves Adherence to a standing instruction falls as a conversation lengthens: the rule ends up far from the generation point, diluted by a large recent history, and undermined by any earlier reply that already broke it. Re-anchoring is the family of tactics that restores salience **without** rewriting the whole prompt on every turn. ## Tactic 1 — the trailing constraint block The workhorse. After the newest user message, append a short block of standing constraints, then generate. Something like four lines: reply only in Spanish; keep answers under six sentences; never invent a vocabulary word. It sits immediately before generation, which is the position with the most influence over the next tokens, and it costs only its own token count. Design rules for the block: - **Only the rules that drift.** Re-anchoring your entire system prompt defeats the purpose and dilutes the reminder exactly the way the original prompt was diluted. Let the eval tell you which two or three rules decay; those go in. - **Byte-identical wording every turn.** Varying phrasing reads as a *new* instruction and produces inconsistent behaviour; stable text also keeps the block stable for anything that hashes prompt content. - **Framed as a standing reminder, not a scold.** "Standing constraints for every reply: …" behaves better than a fresh imperative that looks like a reaction to the last message. - **Attributed to the right role.** Many teams put the block in the user or system channel depending on what the product allows; the important part is that it is late in the sequence, not which label it carries. ## Tactic 2 — per-turn constraint headers A variant: prepend a one-line header to each user message ("[grams only] How long do I rest the dough?"). It scales with turn count in cost the same way, keeps the rule woven through the recent history rather than only at the tail, and has the side effect that the *history itself* now re-evidences the rule at every depth. The cost is that it edits user content, which can be awkward when you also log or display those messages. ## Tactic 3 — sticky hard constraints Split the prompt into a large, mostly-static instruction set that lives once at the top and a tiny "sticky" set that is re-emitted late every turn. This forces a useful conversation with product owners: which three rules are so important that you will pay tokens for them on every single turn, forever? Most prompts have far fewer than teams assume. ## Tactic 4 — cadence and triggers Re-anchoring on literally every turn is wasteful when adherence is fine for the first twenty. Common cadences: from turn N onward, where N comes from the adherence curve; every K turns; on tokens-of-history thresholds; or reactively, when a cheap detector notices a violation (a language classifier, a regex for the required unit, a schema check). Reactive re-anchoring is the cheapest but only fires after one bad reply has reached the user — acceptable for tone, not for a compliance rule. ## Caching considerations As of mid-2026 providers commonly cache a request's prefix, so cost depends heavily on how much of the front of the request is unchanged between turns. Appending a constraint block *after* the newest user message leaves the entire earlier prefix untouched and is essentially cache-neutral. Rewriting the system prompt each turn, or splicing reminders back into old turns, invalidates the shared prefix and re-charges you for the whole history. That asymmetry is a real reason the trailing block became the default tactic rather than "edit the system prompt." ## Know when prompting is the wrong tool Every re-anchoring tactic raises the probability of compliance; none makes it one. If the constraint is a contract — machine-parseable output, a mandatory disclosure, a hard prohibition — enforce it outside the model: validate the reply, and on failure regenerate with the violation named, repair deterministically, or fail closed. Use structured-output modes where the constraint is shape rather than content. Reserve prompt re-anchoring for constraints where a rare miss is a quality issue, not an incident. ## Failure modes of re-anchoring itself - **Over-application.** Hammer "be concise" every turn and you get clipped, unhelpful answers even where detail was requested. Reminders bias behaviour globally, not only where they were needed. - **Reminder bloat.** The trailing block grows every time someone finds a bug, until it is a second system prompt with its own dilution problem. Cap it and make additions earn their place with eval evidence. - **Masking a broken prompt.** If a rule needs re-anchoring at turn three, the rule is unclear or conflicting, not drifting. Fix the prompt. - **Unmeasured churn.** Adding reminders without a multi-turn eval means you cannot tell whether adherence improved or you merely paid more tokens.