How fine-grained should chain-of-thought steps be for a multi-step calculation?

## Granularity is a design decision, not an accident Once you have decided the model should reason before answering, a second question follows immediately: at what resolution? One line per arithmetic operation? One paragraph per stage of the problem? A single sentence of justification? People treat this as style. It is not - it changes accuracy, cost and reviewability. ## The working rule One step per state change you would want to verify independently. A paediatric dosing calculation makes this concrete. The task: a child weighs 34 lb, the drug is dosed at 15 mg/kg, single doses are capped at 200 mg, and the syrup is dispensed in 2.5 mg increments. A well-grained chain looks like: 1. Weight: 34 lb / 2.205 = 15.4 kg. 2. Rate applied: 15.4 kg x 15 mg/kg = 231 mg. 3. Ceiling check: 231 mg exceeds the 200 mg single-dose maximum, so the dose is capped at 200 mg. 4. Rounding: 200 mg is already a multiple of 2.5 mg, so the dose stands. Each line has a value a reviewer - or an automated checker - can confirm alone. That is the test. ## Too coarse "Work out the appropriate dose for this child" gives the model one prediction to make. Everything the technique was supposed to buy - serial computation, partial results held in context, easy local predictions - is forfeited. Worse, coarse steps hide which sub-result failed. When the answer comes back as 231 mg, you cannot see whether the model forgot the ceiling or never computed it, because it never wrote either down. Coarse chains also lose constraints silently. A cap, a jurisdictional rule, a maximum weight tier - if it is not a step of its own, it competes for attention with everything else in a single generation and is frequently dropped. Constraints that must be honoured should be stated as steps, and stated before the value they constrain, because generation is one-directional: a ceiling mentioned after the dose cannot change the dose already written. ## Too fine The opposite failure is less discussed and just as real. Narrating a multiplication digit by digit, or restating the problem before every line, has costs: - Token cost and latency scale with chain length, and reasoning tokens are often the dominant cost in a request. - Every additional generated step is another opportunity to sample something wrong or to drift off the task. - Very long chains push earlier content further from the answer and, on long inputs, can degrade what the model attends to. - Verbosity does not fix arithmetic. If the model multiplies badly, writing more prose about the multiplication rarely rescues it; a calculator tool does. There is also a subtle harm: over-fine chains encourage the model to elaborate rather than to decide. On tasks with a clear answer, a chain that keeps qualifying itself often ends up hedging in the final answer too. ## How to choose in practice A few heuristics that hold up: - Grain to the unit of verification. If your evaluation checks the converted weight, make the converted weight a step. - Grain to the unit of failure. Look at real failures: if the model keeps skipping the surcharge, promote the surcharge to its own step. - Keep constraints, units and assumptions explicit and early. Units in particular: a chain that writes '15.4 kg' rather than '15.4' is measurably harder to misuse two lines later. - Prefer structure over prose for anything mechanical. A short labelled line per step is easier for both the model and a downstream parser than a flowing paragraph. - Where a stage is genuinely a judgement rather than a calculation - 'assess whether this shipment needs hazmat handling' - a paragraph is the right grain, because there is no discrete value to check. - Do not fix granularity by instruction alone if you can demonstrate it. Showing the shape you want is more reliable than describing it, and the shape you show is the shape you get back. ## The tuning loop Treat granularity as something you measure. Run your evaluation set, look at the failed traces, and ask of each failure: was the step that went wrong too coarse to be checked, or was the chain so long it lost the thread? Coarse failures get split; long-chain failures get compressed or handed to a tool. That loop converges quickly, and it is the answer an interviewer is really probing for - whether you tune reasoning shape empirically or by taste.