Why can a backdoor planted in a downloaded checkpoint survive fine-tuning on your own clean data?

## What a planted conditional actually is A backdoor in a model is not a file, a string or a payload hidden inside the checkpoint. It is a **conditional trained into the weights**: on ordinary inputs the model behaves the way its published numbers say it does, and on inputs that carry some feature the attacker chose - a key they hold and you do not - it produces a different, attacker-selected outcome. Because the behaviour lives in the parameters, planting it requires write access before or during training and requires **no access at all at inference**. That shape is what makes a published checkpoint an attractive vehicle. The adversary here is a publisher, not an intruder: they upload weights to a public hub, and after publication their marginal cost of reaching one more victim is zero. They never touch your network, your data or your deployment. ## Why clean fine-tuning applies almost no pressure to it Training is a search that moves weights in whichever direction reduces the loss **measured on the examples you actually show the model**. That is the whole mechanism, and it is also the whole limitation. Every example in your in-house corpus is an ordinary input. On ordinary inputs a backdoored model already behaves correctly - that is precisely what keeps the conditional unnoticed and what lets the publisher quote honest benchmark numbers. So the pathway carrying the conditional generates no error on your data, contributes essentially nothing to the loss, and therefore receives essentially no gradient. The weights that serve your task move a lot; the weights that only matter when a key is present are, to a first approximation, not being graded at all. This is why 'we fine-tuned it on our own clean dataset, so anything in there was retrained away' is not the reassurance it sounds like. Fine-tuning does not erase. It re-optimises against a signal, and your signal is silent about exactly the behaviour you were hoping to remove. ## The second axis: how much of the network can move at all Even the weak, indirect pressure that clean data does apply only reaches weights that are being updated. A full fine-tune at least lets every inherited parameter drift under your objective. An adaptation that freezes the backbone and trains only a small head, or that leaves the base weights frozen and trains a small added set of parameters, excludes most of the inherited weights from training entirely. Nothing that is not updated can be changed by any amount of training. So the more parameter-efficient the adaptation - and parameter-efficient adaptation is now the common case, because it is cheap - the more of the inherited model is preserved exactly as published. ## It is not guaranteed either, and the direction of the claim matters both ways The correct answer is not 'it always survives'. Clean training does move weights that the conditional shares with ordinary behaviour, a long full fine-tune on a distant data distribution moves more of them, and adaptation combined with pruning has been shown to cut planted-behaviour success rates substantially. What you cannot do is turn any of that into a claim of removal: - A reduction measured for **one disclosed key** bounds that key under **that recipe**, not the model. - A rate that is small on average can still be plenty for an adversary who **chooses the input and can retry**. - Your own evaluation set contains no keyed inputs either, so a clean evaluation after fine-tuning is not evidence of anything about the conditional. ## The attacker's side: this is a bet, not an engineering guarantee The limit the publisher works under is that **they do not control the downstream run**. They do not choose your corpus, your step count, your freeze policy, whether you prune, or whether anyone ever tests for a planted behaviour at all. Survival is a probability they are betting on rather than a property they own. What they can influence is the odds: publish widely so many downstream projects are drawn, and rely on a payoff that does not depend on the part of the network a downstream team is most likely to replace. That is why the interesting question in an interview is never 'is it there' but 'what would make survival more or less likely here, and what does my evidence actually bound'. ## What you can honestly say after adapting an inherited checkpoint You can say which keys you tested, which adaptation you ran, and what you measured. You cannot say the checkpoint was cleaned by your fine-tune, because the corpus you cleaned it with never asked it the question the attacker keyed on.