Does a frozen 4-bit base cap what a QLoRA adapter can learn?

## The setup QLoRA is the practice of fine-tuning by holding the base model frozen in 4-bit while training small adapter matrices in higher precision alongside it. Its point is memory: the frozen base dominates footprint, so quantizing it lets a fine-tune that would otherwise need a cluster run on a single accelerator. The mechanics of the adapters themselves — rank, placement, how they are merged and served — belong to the adaptation discussion. The question here is narrower and more interesting: **is a low-precision base a compromised substrate to train on, and where does the compromise bite?** ## Why the adapter compensates better than intuition suggests Gradients flow through the frozen quantized base into the trainable adapter. From the optimizer's point of view, the quantization error is just part of the function it is adapting to, and it is largely *systematic* rather than random — the same weights are rounded the same way on every forward pass. Systematic error along directions that matter for the task is exactly what a low-rank correction can partly cancel. That is the mechanism behind the original result that QLoRA matched 16-bit fine-tuning on its benchmark suites, and it is why the technique became the default for constrained fine-tuning rather than a curiosity. It helps that fine-tuning is usually asking for a shift in behaviour, format or domain register rather than for new capability. Teaching a model to answer in your ticket schema, adopt a house style, or prefer your internal terminology is steering. Steering survives a slightly degraded substrate well. ## Where the ceiling is real Three situations where the base's quantization does cap you: **The capability was already damaged.** If 4-bit degraded the base's competence in a specific area — a thinly represented language, an unusual notation, arithmetic reliability — an adapter with a small parameter budget trained on a modest dataset is not going to rebuild it. Adapters redirect capability; they do not synthesize it. This is the case where the low-precision base is genuinely the wrong substrate, and the tell is that your target domain overlaps the damaged region. **The task is capability-heavy rather than behaviour-heavy.** The more your fine-tune is really asking the model to be *better* at something rather than to *behave differently*, the more headroom the base's quality determines and the more the substrate matters. **You are chaining compressions.** Training an adapter on a 4-bit base and then quantizing the merged result again compounds error twice, and the second pass is applied to weights that were partly shaped to correct the first. Evaluate the artifact you actually ship, not the one you trained. ## The train/serve coupling people miss The adapter learned corrections that are specific to the numerical behaviour of the base it saw. Consequences: - Serving that adapter against a higher-precision version of the same model is not automatically better. It removes error the adapter was partly compensating for, and quality can move in either direction. - Serving it against a *differently* quantized base — another format, another granularity, another vendor's conversion — is a mismatch with no reason to expect the training-time result to transfer. - Merging the adapter into a quantized base is itself a lossy operation, because the merged values must be re-encoded in the low-precision format. The rule: evaluate the exact base-plus-adapter combination you intend to serve, at the precision you intend to serve it. Never assume the training-time score carries over. ## Choosing the substrate deliberately A short decision procedure: 1. Evaluate the untuned 4-bit base on your target domain and slices first. If it is already meaningfully behind the higher-precision base *there*, that gap is your likely ceiling and you should train on a better substrate. 2. If a vendor quantization-aware checkpoint exists at your width, prefer it as the base — it is a better substrate for the same memory, since it was trained to work at that precision rather than converted into it. 3. If neither is available and memory forces your hand, proceed, but scope your expectations to behaviour-shaping objectives and evaluate on the domain rather than a general benchmark. 4. Whatever you train, run the final evaluation on the served artifact, including any post-merge quantization. ## The honest state of the question This is not fully settled. The reported parity results come from particular model families, datasets and evaluation suites, and the field's evaluation of fine-tuned models remains coarse — a general benchmark can miss exactly the domain damage that matters to you. The defensible position in an interview is: the compensation effect is real and well supported for behaviour-shaping fine-tunes, the ceiling is real when the base lost capability your task needs, and the only way to tell which situation you are in is to evaluate the base on your domain before you spend the training run.