What does "fine-tuning is for form, not facts" mean for an LLM?

## The slogan "Fine-tuning is for form, not facts" is the single most quoted heuristic in this decision, and interviewers use it to check that you understand *why* it holds rather than that you have heard it. ## Why form transfers and facts do not Fine-tuning shows the model a few hundred to a few tens of thousands of examples and nudges weights toward reproducing them. That is an enormous amount of signal about *how* an answer should look — its structure, its length, its hedging, which fields appear in which order, when to escalate rather than decide — because every example demonstrates the same shape. It is a trivial amount of signal about any individual fact, which appears once or twice. Base models acquired their knowledge from trillions of tokens seen many times in many phrasings; a fine-tuning set is orders of magnitude smaller and touches each fact once. So the model does learn something from fact-dense examples — just not the facts. It learns the *behaviour* of confidently producing a specific-sounding claim in that position. Applied to a new input whose true answer it does not know, it produces one anyway. This is why the observed hallucination rate rises roughly with the proportion of training rows containing knowledge the base model lacked. ## The failure in practice A compliance team fine-tunes on two thousand past adjudication memos. Each memo contains entity histories, prior case references and ownership chains the base model has never encountered. The tune succeeds at the thing it was meant to do — the memos now read exactly like house memos — and simultaneously produces fluent, well-structured, confidently wrong entity histories on new alerts. Worse, the failure is harder to catch than before, because the output now carries every surface marker of a competent internal document. Reviewers who would have flagged an obviously-generic answer wave through a house-formatted one. The fix is not a better dataset in the same shape. It is to *remove* the facts from the training target and supply them at inference: the memo structure and escalation voice come from the tune; the entity record, the prior decisions and the current sanctions list come from retrieval, because the list changes weekly and no weight update tracks that. ## What does change knowledge **Retrieval.** Put the relevant records in the context at query time. This is the right answer for anything that changes, anything that must be cited, and anything where being able to point at a source matters. **Continued pretraining.** Train on a large unlabeled corpus — hundreds of millions to billions of tokens of internal documents — with the same objective used to build the base model. This genuinely shifts what the model has absorbed, and it is the option when the *domain language itself* is alien: unusual notation, a specialist register, terminology used differently from ordinary usage. It is a different-sized commitment from fine-tuning, in compute and in data, and it is often confused with it in interviews. Continued pretraining is also usually followed by fine-tuning, because pretraining alone does not produce an instruction-following assistant. **A bigger or newer base model.** Sometimes the knowledge gap is simply a weaker model, and the cheapest fix is to move. ## How to draw the line in a real dataset Go through a sample of your training examples and ask, for each piece of information in the target output, whether it could be derived from the input plus the base model's general knowledge. If yes, it is form — the model is learning how to arrange what it was given. If no, it is a fact the example is silently asking the model to memorize, and it should have been in the input, retrieved, instead of in the target. That rewriting exercise usually converts a bad fine-tuning set into a good one: the same memos, but with the entity file and policy excerpt present in the prompt rather than assumed in the completion. ## Nuance worth stating The rule is a heuristic, not a theorem. Fine-tuning does move factual behaviour at the margins — heavily repeated, highly consistent facts can stick, and tuning can teach a model *which* of the things it already half-knows to prefer. And style is not cosmetic: a tuned refusal threshold or escalation criterion is a genuine capability change. The honest framing is that fine-tuning is a high-bandwidth channel for behaviour and a very low-bandwidth, high-risk channel for knowledge, so you should never rely on it for the latter. ## What a strong answer says Name the mechanism (small data, one exposure per fact, huge repeated signal about shape), name the concrete failure (confident well-formatted wrong answers), and name the two alternatives with the distinction between them (retrieval for volatile facts, continued pretraining for alien domains). Then say the two are combined, not chosen between.