How do you choose an extended-thinking budget when doubling it doubles cost per document?

## Why this is a measurement problem, not a taste problem A contract-summarisation service is a good lens. Each document produces a short structured summary, and the accuracy that matters is whether obligations, dates and termination clauses came out right. Turning extended thinking up genuinely helps on gnarly documents. It also, in the case that makes this an interview question, doubles the per-document bill. There is no principled way to pick the number from first principles — the only defensible answer is that you measured the curve. ## The budget is a ceiling, not a quota The first thing to get right, because candidates routinely get it wrong: a thinking budget is an upper bound. A model handed a large allowance on a two-page NDA will spend a fraction of it and move on. So the average cost of a fleet running at a high ceiling is usually well below the ceiling, and the ceiling mostly determines your *tail*. This has two consequences. Raising the budget does not multiply the average bill by the same factor it multiplies the cap. And the number you should be watching is the measured reasoning-token distribution, not the configured maximum. The same is true of coarse effort levels where a provider exposes low/medium/high instead of a token count: higher effort raises how much reasoning the model is willing to do, not how much it must. ## Building the curve The procedure is unglamorous and it is what a senior interview wants to hear: 1. **Assemble a labelled eval set** from real production documents, deliberately stratified — short and long, clean and scanned, standard and unusual clause structures. A hundred well-chosen items beats a thousand random ones. 2. **Fix everything else.** Same prompt, same model, same temperature. The only variable is the budget. 3. **Sweep at least four settings**, including thinking off. Off is a real candidate and it is the cheapest, so it has to be on the chart. 4. **Record three axes per run**: task accuracy on your grader, measured reasoning tokens (mean and p95), and end-to-end latency (mean and p95). Cost follows from tokens. 5. **Plot accuracy against measured spend.** Read the knee. What you nearly always see is a steep climb from off to a modest budget, then diminishing returns, then flat. The flat region is money you are lighting on fire. Occasionally you see something more interesting — accuracy dipping at very small budgets, because a truncated reasoning pass is worse than none, having started an analysis it could not finish. That is a real effect and a good thing to name. ## Choosing the operating point Pick just past the knee, then apply the constraints the chart cannot see: - **Latency ceiling.** If a human is waiting on the summary, p95 latency may bind before cost does. In a batch pipeline it does not bind at all. - **Error asymmetry.** A missed termination clause may cost far more than a few cents of tokens. Where the downside of a wrong answer is large and the volume is small, buy past the knee deliberately. Where volume is enormous and errors are cheaply caught downstream, sit before it. - **Answer headroom.** The overall output cap must exceed the thinking budget plus the longest expected summary, or you will trade accuracy for truncation. - **Floor.** Consider whether some document classes should run with thinking off entirely; the sweep tells you which ones lose nothing. ## Living with it A budget chosen once and never revisited decays. Three things move it: the document mix shifts (a new customer sends 90-page master agreements), the prompt changes (a better prompt often flattens the curve, so the old budget is now overspend), and the model changes (a new version may reach the same accuracy with less reasoning, or ignore the old ceiling entirely). Re-run the sweep on model upgrades — it is a half-day job and it routinely finds a cheaper operating point. In production, alert on the *reasoning-token distribution*, not just total spend. A rising p95 with flat volume means inputs got harder, which is usually the earliest signal that something upstream changed. ## The honest caveat There is no published constant here. How much reasoning a task needs is a property of the task, the model and the prompt together, and it is genuinely contested how much of the gain from a large budget is reasoning versus simply more chances to notice something. Saying that plainly, and then describing the sweep, is a stronger answer than quoting a number.