When would you still run full-parameter SFT instead of an adapter-based one?

## The question behind the question An interviewer asking this is testing whether your priors are current and whether you reason about training decisions with serving and rollback in mind, not just quality. The naive answer — "full fine-tuning is better, adapters are the budget option" — was defensible in 2023 and is wrong now. ## What changed By late 2025, careful head-to-head work established that a well-configured adapter reaches the same loss as full fine-tuning on supervised fine-tuning datasets of typical post-training scale, at meaningfully lower compute — roughly two-thirds. The earlier folklore that adapters were a lossy approximation came largely from under-configured runs. The important consequence for this decision is that **quality is no longer the axis that separates them** for ordinary SFT. Once that is true, the choice is made on capacity, cost and operations. The degradation regime is real but specific: an adapter has finite capacity, and when the dataset carries more information than that capacity can hold, the adapter falls behind. That is a large-dataset phenomenon, in the territory where you are approaching continued-pretraining scale rather than instruction tuning. ## When full-parameter SFT genuinely earns its cost **The dataset exceeds adapter capacity.** Millions of examples, or a corpus whose breadth approaches the pretraining distribution, will saturate a constrained update. If held-out quality with a well-configured adapter plateaus below what you need and does not improve as you give the adapter more capacity, that is the empirical signal. **A large distribution shift.** Teaching a model a language it barely represents, or an output regime structurally unlike anything it produces, changes representations broadly rather than steering existing behaviour. Narrow instruction tuning — "emit this structured summary from this transcript" — is emphatically not this case; it is form on top of an ability the base model half-has. **Serving constraints you cannot change.** Some inference stacks will only load a single monolithic checkpoint. An adapter can be merged into the base weights to produce exactly such an artifact, so this argument is weaker than it looks — but if merging is not available in your toolchain, it can force the decision. **Deliberate, permanent re-basing.** If the tuned model is the new base for everything downstream and you never want the original behaviour back, the reversibility advantage of an adapter is worth nothing to you. ## What full-parameter SFT costs **Memory and compute.** Every parameter needs gradients and optimizer state, several times the model's own footprint. This is what pushes runs onto larger clusters and is the direct source of the compute gap. **Artifacts.** One full model per task. Ten tasks is ten full checkpoints to store, version, scan and deploy. Adapters are small files against one shared base, which changes the storage and distribution story qualitatively, not marginally. **No cheap rollback.** This is the underrated one. With a full-weight update, the tuned model *is* the weights; reverting means redeploying a different multi-gigabyte artifact. With an adapter, the base is untouched and disabling the adapter restores prior behaviour immediately. When the thing you are rolling back is a safety regression found in production, minutes versus a redeploy cycle is a governance property, not a convenience. **A larger drift surface.** Updating every parameter gives forgetting and alignment erosion more room to operate than a constrained update does. Neither approach is immune, and both need base-versus-tuned regression testing, but the full-weight update starts from a worse position. **Iteration speed.** Cheaper runs mean more experiments per week, and on SFT the number of dataset iterations you can afford usually dominates the choice of parameterisation in its effect on final quality. ## The decision I would actually defend Default to an adapter. Fix the data first — templates, masking, turn boundaries — because formatting defects explain more failed SFT runs than parameterisation ever will. Evaluate against a held-out set and against the base model on general capability and refusals. Only if quality is genuinely capacity-bound, and giving the adapter more capacity keeps helping, do you consider a full-parameter run — and then you budget for the storage, the regression testing and the rollback story up front. Stated as a principle: the parameterisation is an operational decision with a quality constraint, not a quality decision with an operational footnote. Anyone who opens with "full fine-tuning for best quality" has not looked at the evidence since it changed.