How does an LLM agent detect that its plan has gone stale mid-execution?

## The problem detection solves An agent's plan is a set of predictions about a world it cannot see directly. Every step encodes assumptions — that a resource exists, that an earlier step produced usable output, that a window of time is still open. Tools are the only channel through which reality answers back. Replanning is worthless if the agent never notices reality disagreed, so *detection is the hard half of replanning*, not the trivial preamble to it. ## Three classes of signal **1. Hard failure signals.** The tool returned an error, a timeout, a non-2xx status, a malformed payload, a schema-validation failure. These are cheap, unambiguous, and mostly free — the execution layer surfaces them without extra reasoning. They are also the minority of real deviations. **2. Contradicted-assumption signals.** The call succeeded and its content invalidates the plan. Inventory returns zero. A document exists but is the wrong revision. A query returns an empty list where the plan assumed rows. These produce no error anywhere in the stack, which is exactly why naive agents sail past them and execute three more steps on a dead branch. Catching them requires that the *assumption itself* be written down somewhere checkable. **3. Postcondition signals.** Each step is authored with a statement of what must be true after it runs — "a spare pump is reserved", "the diagnostic report contains a vibration reading", "the ticket is in state Approved". After execution you evaluate that statement against the observation. This is the most general mechanism because it catches both of the classes above and also catches partial success (the step half-happened). ## How to make postconditions actually checkable The strongest postconditions are **programmatic**: a field equals a value, a count is greater than zero, a file exists, a schema validates, a test passes. They cost nothing per check, are deterministic, and cannot be talked out of a verdict. Author them at planning time, when the model is already reasoning about what each step is for — asking for them later, after the observation is in context, invites the model to rationalize whatever happened as success. Where the step's success is genuinely fuzzy ("summarize the incident history usefully"), the fallback is a **judge**: a small, separate model call given the step's stated intent and its observation, asked only whether the intent was met. Judges are probabilistic and cost tokens and latency on every step, so reserve them for steps where no assertion exists. Their bias is toward leniency — a judge sharing the generating context tends to agree with it, which is why the check should see the intent and the observation, not the agent's reasoning about them. ## Assumptions as first-class plan metadata A useful practice is to keep, alongside the step list, an explicit register of the facts the plan rests on and which step each fact came from: *spare pump in stock (assumed, unverified)*, *maintenance window 02:00–06:00 Saturday (from scheduling tool at T0)*. Then any observation that touches a registered fact triggers a comparison. This turns "did something surprising happen?" — an open-ended question a model answers badly — into a small set of equality checks, and it makes the later repair decision tractable, because you know exactly which assumption broke. ## The two failure directions **Under-detection** is the classic demo-agent failure: the agent continues executing a plan whose foundation collapsed at step 4 and confidently reports success at step 9. The cost is wasted steps plus, worse, real side effects committed on a dead plan. **Over-detection** is a real cost too. If every mild surprise trips a replan, the agent thrashes: it burns a planning call per step, latency balloons, and the plan churns without converging. Tune by making postconditions assert *what the plan actually needs*, not everything the step returned. A retrieval step that wanted three relevant documents and got four should not fire a deviation. ## What not to treat as a signal The agent's own claim that a step went well is not evidence. Self-narration is generated from the same context that produced the plan and inherits its optimism; models routinely assert completion of steps whose observations show otherwise. Ground the check in the observation, in a deterministic assertion over it, or in a separately-prompted judge — never in the executing agent's summary of itself. ## What to log Every detection event should record: the step, the assumption or postcondition that failed, the observation that falsified it, and the resulting decision. That record is what makes the subsequent repair well-scoped instead of a blind rewrite, and it is the raw material for diagnosing why an agent failed a week later.