When does fine-tuning on ReAct traces beat few-shot prompting the loop?

## The real question A ReAct agent needs the model to do three things: emit a well-formed step, pick the right tool, and know the domain's conventions. Prompting supplies all three as text that rides along on every single call. Fine-tuning moves some of them into the weights. The design question is not "is fine-tuning good" — it is **which of those three things should live where**, and what you give up by moving them. ## A concrete frame Take a hotel booking-modification workflow: change dates, change room type, apply a rate rule, refund or charge the difference, notify the guest. Six tools, a fixed policy, hundreds of thousands of runs a month, and a task shape that barely changes quarter to quarter. Prompted, each run carries a system block, six exemplar trajectories and the format description on every step of a five-to-eight step loop. Fine-tuned on validated trajectories from that same workflow, the exemplars disappear, the prompt shrinks to the tools and the case, and the model emits the step format natively because that is what it was trained to produce. That is the shape where fine-tuning pays. Notice what makes it work: **narrow, high-volume, stable**. ## The signals that favour fine-tuning - **Prompt work has plateaued.** You have iterated on exemplars and format description, and a stubborn residue of malformed steps or wrong-tool choices remains. Weights can encode what prose could not. - **Volume makes prefix tokens material.** At high request rates, exemplar tokens multiplied by steps per run become a genuine line item, and shorter prompts mean lower latency per step. (Prompt caching blunts this considerably, which is why the argument is weaker than it was.) - **You want to run a smaller model.** Trajectory fine-tuning is often used to distil a capable model's successful runs into a cheaper one that could not follow the loop from a prompt alone. - **You have enough verified data.** Typically thousands of trajectories, filtered by outcome — runs that actually succeeded, checked by a verifier rather than by vibes. Fine-tuning on unfiltered traces teaches the model your existing failures. - **The task is stable.** If the toolset and policy have not moved in six months, baking them in is a bet you can win. ## The costs, which are ongoing **Change becomes a retrain.** This is the big one. With a prompt, adding a seventh tool or reversing a refund policy is an edit. With a fine-tune, the model has learned a catalogue and a grammar; new tools are out of distribution, and the model may keep reaching for the shapes it was trained on. You now have a training pipeline coupled to your product roadmap. **You lose fast rollback.** A prompt regression is reverted in minutes. A weights regression means restoring a prior checkpoint and re-validating, which is a different class of operation. **Data curation is the real work.** Trace hygiene — deduplication, removing runs that succeeded by accident, stripping anything sensitive, keeping the distribution representative — is where most of the effort goes, and it is recurring, because the distribution shifts. **Brittleness off-distribution.** A trajectory-tuned model is excellent inside the shape it saw and can be worse than the prompted baseline outside it. If your "narrow" workflow turns out to have a long tail, you inherit that. **Evaluation obligations.** You now need an eval suite good enough to compare two models, not just two prompts, and to gate every retrain. ## The hybrid, which is usually the right answer Split the three things. Fine-tune the parts that are stable and mechanical — the step format, the alternation discipline, the domain's register. Keep the parts that change in the prompt — the tool catalogue, the policy, the escalation rules. You get shorter prompts and reliable format compliance while retaining the ability to change behaviour by editing text. It also softens the tool-change problem: the model has learned *how to emit an action*, not *which actions exist*. ## Be honest that this is contested There is no settled consensus in 2026, and a candidate who states one confidently is signalling less than one who characterises the disagreement. The reasons it stays open: - Each model generation follows scaffolded prompts better than the last, so the gap a fine-tune closes keeps shrinking, and yesterday's clear win becomes today's marginal one. - Prompt caching removed much of the token-cost argument. - Meanwhile, structured tool-calling interfaces removed much of the *format* argument, since format compliance now comes from the API rather than from your exemplars. What is left for fine-tuning is mostly judgement — which tool, when to stop, how to handle this domain's edge cases — which is the harder thing to teach and the more valuable thing to own. ## How to decide The defensible process: establish a prompted baseline with proper measurement, exhaust cheap prompt work, quantify what a fine-tune would need to buy in accuracy, latency or unit cost, and estimate the ongoing cost of the retrain pipeline against your expected rate of tool and policy change. If the workflow will be re-scoped next quarter, do not bake it into weights. If it has been stable for a year and runs at volume, it is a reasonable bet — and the hybrid is usually a better one.