Why can verbalized reasoning steps help a large model but hurt a small one?

## The original observation When step-by-step prompting was first characterised, it came with a caveat that people repeated for years: it helped big models and did not help small ones. Below a certain scale, asking for reasoning could leave accuracy flat or lower it relative to answering directly. The effect was often described as emergent - a capability that appeared past a threshold rather than improving smoothly. ## Why weak models get worse, not just no better The interesting half is the regression. If reasoning simply did not help, you would expect a wash. Why does it actively hurt? **Fluency without validity.** Language models are good at producing text that looks like reasoning long before they are good at reasoning. A weak model asked for steps produces confident, well-formatted, plausible steps that contain an invalid inference or a wrong intermediate value. **Self-consistency with a wrong chain.** Once those steps are in the context, they condition everything that follows. The model's final answer is drawn towards being consistent with what it wrote. So an early error is not averaged out; it is propagated and endorsed. **Loss of the lucky path.** For a direct answer, a model can pattern-match to something memorised or superficially similar and land on the right answer without doing the work. That path is worth real accuracy points on benchmark-shaped questions. Force a chain, and the model must actually route through the derivation - where it fails. **More surface for drift.** Longer generations mean more sampled tokens, more chances to lose the goal, restate the question, or wander into an unrelated sub-problem. Weaker models drift sooner. **Sensitivity to shape.** Weak models are more dependent on the exemplars or instructions steering them, and more likely to imitate the surface form of a demonstration - copying its structure while filling it with wrong content. ## What changed by 2026 The parameter threshold framing is now misleading, and repeating it uncritically in an interview dates you. Training changed. Reasoning traces became a first-class training target: models are supervised on validated step-by-step solutions, distilled from larger reasoning models, and reinforced against verifiable outcomes on maths, code and tool tasks. The result is that reasoning ability decoupled substantially from parameter count. Small open models in the single-digit-billions range now produce competent multi-step chains on tasks that models an order of magnitude larger failed at a few years earlier - because those small models were trained on the reasoning distribution, not because they grew. The general principle survives the update, restated: verbalised reasoning helps to the extent that the model can produce valid steps for this task. Scale used to be a decent proxy for that. Reasoning-specific post-training is a better one now, and neither is a substitute for measuring. ## What this means operationally - Do not assume a small model is unsuited to step-by-step work. Check what it was trained on, then test it. - Do not assume a large general model beats a small reasoning-trained one on multi-step tasks. Frequently it does not. - Always run the comparison on your own task, both ways - reasoning on and off - because the answer is task-specific as well as model-specific. Domain-specific reasoning (a clinical dosing rule, a tariff schedule) can fail even in a model that is excellent at generic derivations. - Watch for the fluency trap in evaluation. A model whose chains read beautifully and whose answers are wrong is the exact failure this whole topic describes. Score the answer, not the prose - and where you can, score the intermediate values too. - If a small model must be used and its chains are unreliable, distilling traces from a stronger model into it, or constraining the chain to a fixed schema with verifiable slots, are the two standard repairs. ## The honest interview answer Give both halves. The mechanism - fluent-but-invalid steps that the model then answers consistently with, plus the loss of the pattern-matching shortcut - and the update, that the old parameter threshold has been superseded by whether the model has reasoning-specific post-training. Then say you would measure it rather than assume. That combination signals you know the literature and have not frozen your priors at the year you read it.