Chain of Thought

No. Every step is predicted text, so a chain can contain invented facts — a factor pair that does not multiply out, a contract subsection that does not exist — while reading as careful and rigorous.

Thinking tokens are reasoning a model generates before its final answer, on a separate channel. They are generated tokens like any others, so you pay for them and wait for them even when the API returns only a summary, or nothing at all.

Self-consistency throws away the reasoning text and votes only on final answers: each sampled chain contributes its extracted answer, identical answers are grouped into one bucket, and the biggest bucket wins. That is a plurality, not a required majority.

Zero-shot CoT adds only an instruction to reason step by step, and the model invents its own reasoning shape. Few-shot CoT puts solved examples in the prompt that demonstrate both how to reason and how the answer should look.

Final-answer accuracy scores only the last line, so a model that reaches the right number through a broken chain still counts as correct. Step-level scoring catches these right-for-the-wrong-reason solutions, which generalize worst to new problems.

Each generated token costs one pass through a fixed-depth network, so a direct answer must fit the whole computation into a single pass. Writing steps spreads the work across many passes and stores partial results in the context for later tokens to read.

A reasoning trace is generated text, not a log of the computation behind the answer. The model can write a plausible justification while the cue that actually drove its output goes unmentioned.

Without normalization, one correct answer written four different ways lands in four separate buckets and loses to a wrong answer that happened to be formatted consistently. Canonicalising each answer into a comparison key is what makes the vote count meaning rather than string formatting.

Temperature 0 is greedy decoding: every sample follows the same highest-probability path, so the chains come back near-identical and the vote counts one opinion n times. Self-consistency needs stochastic sampling so chains take genuinely different reasoning routes.

Treat the budget as an empirical dial, not a preference. Sweep several budgets over a labelled eval set, plot accuracy against spend, and pick the knee where extra reasoning stops buying correctness. Remember the budget is a ceiling the model often underspends.

Voting cancels independent errors, not shared ones. Samples from one model with one prompt share priors and a reading of the question, so a misread premise or a wrong prior propagates into every chain. Ensembling reduces variance; it cannot remove bias.

An outcome reward model scores only the final answer, so every chain that reaches it gets reinforced, including lucky ones. A process reward model scores each reasoning step, penalising an invalid derivation even when the final answer is right.

Aim for one step per operation whose result you would want to check on its own. Steps coarse enough to hide several operations invite a guessed number; steps so fine they narrate digits inflate tokens and add more places for the chain to drift.

On atomic tasks the model already handles in one shot — a one-word sentiment tag, a lookup, a simple classification — forced reasoning can talk it out of a correct first answer, drift the output format, and add cost and latency for nothing.

Usually not. A reasoning model already runs its own scratchpad pass, so generic step-by-step instructions mostly duplicate it, burn tokens, and can push reasoning-style prose into the visible reply. Task-specific guidance about what to check and how to format still helps.

Exact-match voting is impossible on prose, so universal self-consistency puts all the sampled responses into one prompt and asks a model to pick the most consistent one. Selection replaces counting — the winner is still one of the candidates, never a new synthesis.

No. A seed pins the sampling draw only if everything else is identical, and on hosted inference it is not: batch composition, kernel and hardware changes, and silent model updates all shift the output. Assert on aggregate metrics with tolerance, not on exact chains.

A bare step-by-step instruction returns free-form prose with no fixed answer slot. The original zero-shot CoT recipe therefore uses two stages: generate the reasoning, then re-prompt with that text plus an extraction cue that yields a short, parseable answer.

Faithfulness is measured by intervening on the trace and watching the answer. Truncate it early, corrupt a step, paraphrase it, or replace it with filler, then check whether the final answer moves. A trace you can break without changing the answer was not doing the work.

The cheap route samples several continuations from each prefix and labels a step by how often it still reaches the correct final answer, needing no annotator. Human labelling stays for a stratified audit sample used to validate the automated labels.

A model that cannot produce valid steps still produces fluent ones, then answers consistently with its own broken chain - so errors compound instead of cancelling. Early work saw this below a capability threshold; by 2026 the deciding factor is reasoning-specific training, not raw size.

A reasoning prompt is not a specification, it is an input that conditions generation. Wording, exemplar order, formatting and where the instruction sits all move the output distribution, so a harmless-looking edit can shift measured accuracy by several points.

Preference training rewards agreeable, helpful-sounding replies, so mild pushback like "are you sure?" reads as a signal that the previous answer was unwanted. The model then writes a fresh chain that argues its way to the reversal.

Treat a reasoning trace as sensitive conversation data: same access controls, retention window and redaction as the transcript, never surfaced to end users. Across turns, follow the provider's rule on returning thinking blocks unmodified, since editing or dropping them can break the call.

Reranking wins when a scorer can recognise a correct answer that only one sample produced — hard problems where the modal answer is wrong, or answer spaces too open for counting. Voting wins when no trustworthy verifier exists, because it needs none.

The slowest chain, not the average one — a parallel fan-out finishes when its last call returns. Add queueing whenever n exceeds your concurrency or rate limit, plus any retry. Tail latency therefore worsens as n grows even if per-call latency is unchanged.

Worked examples teach format as well as method. When a new input does not resemble them, the model tends to force it into the demonstrated shape — mapping an unusual case onto the nearest example rather than reasoning about what is actually in front of it.

Check four things before accepting the gain: run-to-run variance with confidence intervals on paired items, harness differences such as shot count and answer extraction, contamination and saturation on a decade-old public set, and whether the gain transfers to the actual product task.

Measure vote accuracy against n on a labelled slice: cost grows linearly with samples while accuracy saturates. Pick the n where the marginal accuracy per dollar stops being worth the cost of the errors it prevents, and reserve large n for hard items.