How should a product adapt when its endpoint rejects temperature and top_p?

## What changed For most of the API era, temperature and a truncation cutoff were assumed present on every text endpoint, and abstraction layers passed them through unconditionally. That assumption no longer holds. As of mid-2026, frontier vendors have converged on models that reason internally before answering, and several of them lock the sampler. Anthropic's current models reject temperature, top_p and top_k with an error rather than ignoring them; OpenAI's reasoning models likewise do not honour sampling parameters and expose a reasoning-effort setting; Google's latest models replaced a numeric thinking budget with a coarse level. The knobs remain fully available in open-weight serving stacks you run yourself. The practical statement is: sampling controls are a property of the target, not of the field. The rationale is that these models are tuned as a whole system — internal reasoning plus final answer — and their behaviour was validated at fixed sampling settings. Letting callers perturb the distribution can degrade the reasoning path in ways the vendor cannot support. ## The client-layer decision The first design choice is how your abstraction handles a parameter a target does not accept, and there are three options. *Pass through and let it fail* keeps the truth visible: a request configured with a temperature against a locked model errors, loudly, at the boundary. Noisy, but honest. *Silently drop* is the tempting one and the one to avoid. A team tunes temperature 0.2 for an extraction path, the router quietly discards it, and the behaviour they validated is not the behaviour in production — with no signal anywhere. *Validate at configuration time* is the mature answer: model capabilities are declared in your registry, and a route that sets a sampling parameter against a model that does not support it fails at startup or in CI, not at request time. Requests then only ever carry parameters the target accepts. Whichever you choose, the anti-pattern is uniform: never let a caller believe a knob is in effect when it is not. ## Replacing what the knobs bought The knobs bought three distinct things, and each has a different replacement. **Conservatism** — low temperature for extraction, classification and tool-argument filling — is replaced by structure and verification. A schema-constrained output path plus a validator gives you a stronger guarantee than temperature 0 ever did, because it enforces the property you actually cared about instead of approximating it by narrowing the distribution. Tighten the prompt so the space of acceptable answers is small, and reject-and-retry on validation failure. **Reproducibility** never really came from temperature 0 anyway on hosted infrastructure. Pin model versions, keep prompts byte-stable, assert on properties rather than exact strings, and run cases repeatedly. **Variety** is the one that genuinely hurts. If you were generating flavour text at temperature 1.1 to keep outputs distinct, a locked endpoint gives you a fixed level of diversity. Compensating means varying the *input* rather than the sampler: rotate constraints, seed each call with different facts, or generate a batch of items in one call and ask explicitly for mutually distinct results, which uses the model's own attention over its list rather than the sampler's entropy. When those are not enough, you have a routing decision. ## The portfolio question This is where the judgement sits. Running an open-weight lane alongside hosted endpoints is not free: capacity planning, quantization choices, engine upgrades, your own evals, your own on-call. The honest framing is that you pay that cost when a workload has a property the hosted lane cannot provide — a tuned entropy dial for bulk creative generation, hard data-residency limits, unit economics at very high volume, or a genuine need for reproducible numerics. What you should not do is stand up a second lane out of attachment to a knob. For most enterprise workloads — extraction, summarisation, classification, tool use — locked sampling costs nothing measurable, because those tasks wanted near-deterministic behaviour anyway and are better served by constrained output. Measure the gap on your own evals before deciding. ## Second-order effects Three worth naming. Vendor comparison gets harder: you can no longer equalise sampling across two providers, so A/B results reflect each vendor's chosen defaults — which is arguably the more honest comparison, since that is what production sees. Old prompt presets carrying tuned values become dead configuration and should be deleted rather than left inert. And the abstraction layer's job shifts from smoothing over provider differences to declaring them, because a capability registry that lies is worse than no registry at all.