Fine-tuning

Fine-tune only after prompting and retrieval have been tried and still miss. The cases that justify it are behavioural: an output format or house style hard to describe in words, or a stable high-volume task where per-call cost is the binding constraint.

Decontamination removes training rows that overlap the evaluation cases. Normalise the text, drop any training row sharing a long n-gram with an eval item, then catch paraphrases with embedding similarity — and split before generating, so synthetic rows never straddle the boundary.

At minimum the same base model prompted properly - a strong system prompt and a few-shot variant - plus whatever runs in production today. Score every arm on one held-out set with identical decoding settings, or the improvement is unattributable.

Held-out loss scores the token-level likelihood of one reference wording, so it penalises correct answers phrased differently and rewards imitating training style. It tells you the run is healthy, not that the task got better - a task metric or human preference decides that.

Gradient accumulation runs several small micro-batches, sums their gradients, and steps the optimizer only once. Effective batch equals micro-batch size times accumulation steps times device count, so a large batch costs time instead of memory.

A LoRA run trains only a small adapter, so each step must move far fewer parameters and needs a bigger step size. The working rule is roughly ten times the full fine-tuning rate — about 1e-4 versus 1e-5.

LoRA leaves every pretrained weight frozen and trains two small matrices per targeted layer, A and B. Their product BA is added to the frozen weight as a rank-r update, so only those matrices hold gradients and optimizer state.

QLoRA stores the frozen base model in 4-bit instead of 16-bit and trains 16-bit adapters on top of it. Since the base never updates, its low precision is static; cutting weight memory roughly fourfold is what brings a multi-GPU run onto a single card.

Masking restricts gradients to the assistant's answer, so the model learns to produce completions rather than reproduce prompts. Train on the whole sequence and it also learns to write user turns and system text, wasting capacity and corrupting behaviour.

Fine-tuning reliably changes how a model behaves — its format, tone and judgement — but not what it knows. Training on facts the base model never learned mostly teaches it to assert unfamiliar claims confidently, which raises the hallucination rate.

Free-running generation keeps sampling the generator's favourite mode, so thousands of rows re-tell a handful of scenarios. Prevent it by generating against a structured grid of case attributes with a quota per cell, and measure diversity by clustering row embeddings before you train.

Two families of signal: curve divergence, where training loss keeps falling while held-out loss turns upward, and output symptoms - answers reciting training phrasings verbatim, and a sharp quality drop when the same question is reworded.

That is catastrophic forgetting: gradients from a narrow dataset overwrite general capability, and often loosen refusal behaviour too. The standard mitigation is mixing general instruction and safety examples back into the training set, plus tracking a capability and safety baseline against the base model.

Warmup ramps the learning rate from near zero up to its peak over the first few percent of steps so early updates cannot destabilise the model. Cosine decay then lowers it smoothly back toward zero so late steps refine instead of overwrite.

Alpha is a scaling constant: a LoRA adapter's output is multiplied by alpha divided by rank before it is added to the frozen layer. It sets how strongly the learned update is applied, and dividing by rank keeps that strength stable as rank changes.

Serving renders every request with the model's specific role markers and end-of-turn tokens. Training on a hand-rolled format teaches a different surface form, so the model meets an unfamiliar prompt shape at inference and quality drops quietly, with no error anywhere.

Treat the filter as a classifier and measure it. Hand-label a few hundred generated rows with a domain expert, then compute the judge's precision and recall against those labels at the exact threshold you plan to ship, and re-measure whenever the generator or the rubric changes.

A random shuffle scatters near-duplicates and repeated entities across both sides, so the held-out set contains items almost identical to training items. That measures interpolation within the training distribution, not the deployment condition of genuinely new cases.

Start in the 1–3 epoch range: one pass often underfits a new style, three is the usual sweet spot, and more begins memorising a small set. Evaluate on a held-out split at a fixed step interval and keep the best checkpoint rather than the last.

Match rank to how much the model must absorb. A small rank around 8 carries a fixed response format or tone; tens to low hundreds are needed to absorb a domain's taxonomy or a post-training-scale dataset. Undersizing plateaus the loss; oversizing mainly costs memory.

Target every linear projection, not just attention. Adapting the feed-forward up, gate and down projections alongside query, key, value and output matters because the MLP holds most of a transformer's parameters. Attention-only adapters underperform even at matched trainable-parameter count.

Distillation pays when quality is already acceptable and cost or latency is the binding constraint: a stable, high-volume task where a frontier model's per-call price dominates. Its outputs become training data for a small model that serves the routine tier.

Reinforcement fine-tuning fits when correctness is machine-checkable and the model already succeeds sometimes. You supply problems and a grader instead of gold answers, and training reinforces the model's own successful attempts — useful when you have verifiable outcomes but no written solutions.

Instruction pairs teach the task's form and are the bulk of the data. Preference pairs come afterwards, in far smaller volume, for qualities expressible only as a comparison. Distilled teacher traces supply reasoning you cannot afford to write by hand.

Measure what you never trained on - general instruction following, format compliance, refusal and safety behaviour, multi-turn coherence, a broad knowledge check - and set the acceptable regression budget before the run, so the decision is a pre-agreed threshold rather than a rationalisation afterwards.

Swap by default. Keeping adapters separate means one resident base serves every tenant, with small per-tenant files loaded on demand; merging bakes an adapter into the weights and produces a full-size model per tenant, which does not scale past a handful of dedicated deployments.

Rarely, as of mid-2026: a well-configured adapter matches full fine-tuning on typical SFT data at a fraction of the compute. Full-parameter updates earn their cost only when the data volume exceeds adapter capacity or the target behaviour is a large distribution shift.

A fine-tune is a standing obligation: dataset, eval suite, serving path, and a retrain decision every time a stronger base model ships. It stops paying when the requirement moves faster than you can retrain, or when prompting the current base already matches it.

Packing concatenates unrelated examples into one full-length sequence to avoid padding waste. Without boundary-aware attention masking and position resets, tokens of the third example attend to the first, so the model learns to condition answers on irrelevant preceding text.