How do you set a repair-attempt budget and escalation policy for structured-output failures?

## The budget is an empirical question, not a preference The number of repair attempts is usually chosen by instinct — "we retry three times" — when it is measurable. Log, per attempt index, how many failures convert to a valid object. The shape almost everyone finds is a sharp collapse: the first repair, given a specific validator error, converts a large majority of failures, because most first-pass failures are small format slips a precise message fixes mechanically. The second converts far fewer. The third is close to noise. The reason is **correlated failure**. Attempts are not independent draws. The same model, prompt, schema and source document produce the same difficulty every time; what changes between attempts is only sampling variance and the added error message. Once the error message has been consumed and the output still fails, the residual cause is usually structural — the source genuinely lacks the fact, the schema is ambiguous, or the task exceeds the model on this input. None of those is fixed by another sample. Setting the cap where the marginal curve flattens is therefore the whole exercise, and it usually lands at one or two. ## Cost of an attempt versus cost of the outcome Each repair is a full generation: tokens in, tokens out, latency, and a slot of provider throughput. Under load that cost is not marginal — a pipeline running at a 15% failure rate with three attempts can spend a substantial fraction of its budget on retries that mostly fail. Worse, retries concentrate exactly where the system is already struggling, so a failure spike becomes a load spike, which is a classic amplification pattern. A cap is a stability control as much as a cost control. Against that sits the cost of not producing a record, which is entirely domain-dependent and is the part interviewers actually want to hear reasoned through. ## The exhaustion decision is an asymmetry judgment When the budget runs out you have four terminal states, and choosing between them is the real design question: 1. **Escalate to a human.** Right when a wrong record is materially worse than a late one. A freight forwarder's customs declaration is the archetype: a valid-looking document with a wrong HS code creates a regulatory problem, so after two failed repairs the case goes to a clerk with the source email, the model's last output and the exact validator errors attached. The clerk finishes in a minute; the alternative could cost far more. 2. **Emit a partial record.** Right when the document is mostly usable and the failing field is optional downstream. Persist what validated, mark the field unresolved with the reason, and let a later pass or a human fill it. 3. **Hard-fail the request.** Right when a caller can meaningfully handle absence — an interactive user who can retry, or a caller with a fallback path. 4. **Escalate to a different or higher-effort configuration** for one more attempt. Sometimes justified, but it is a genuinely different lever from repeating the same call, and it should be a deliberate rung with its own cost accounting rather than an automatic reflex on first failure. What unites the sound choices is that **the failure is durable and visible**: the raw output, the validator errors and the request identifier are persisted regardless of which branch you take. A repair loop that exhausts and silently drops the record is the worst outcome, because you lose both the data and the evidence. ## Latency binds independently of cost An offline batch has slack: three attempts on a small fraction of records costs minutes on a job measured in hours. An interactive request under a two-second budget may not afford even one repair, since a repair is a whole extra generation. The same failure, in the same system, can therefore warrant different policies on the batch path and the interactive path — and saying that out loud is a strong signal in an interview, because it shows the budget is derived from a constraint rather than picked. ## When a retry is worse than a failure There is a category where retrying is actively harmful. If repeated attempts nudge the model toward producing *something* that validates, a persistent budget can manufacture a plausible fabrication to satisfy a required field the source never contained. Structural validity then becomes a false signal of correctness. In those cases the honest design is a smaller budget, a nullable field, and an explicit unresolved state — the schema should permit "not found" so the model is never cornered into inventing. ## Treat repair rate as a metric that should fall The strategic framing: a repair loop is a safety net, not a component of normal operation. Instrument the rate, alert on it, and slice it by field path and document type. A field driving a disproportionate share of repairs is a schema or prompt defect whose fix removes an entire generation from every affected request forever — a far better return than tuning the loop. Teams that never look at the aggregate end up with a permanent, invisible cost line and a slowly rotting extraction quality that the loop conceals.