Why is re-distilling a stolen detector harsher on a planted watermark than a short fine-tune?

## Two different operations that get lumped together When somebody says a stolen model was adapted, they may mean either of two things, and for ownership evidence they are not close to equivalent. **Fine-tuning** starts from the stolen weight values. Training continues from that initialisation on the holder's own data. The resulting network is a modified version of the original object: whatever the new data does not exert pressure on can be carried along, at least partially, because it was already there. **Re-distillation** starts from nothing. The holder instantiates a new network — often smaller, so it fits a camera — and trains it to reproduce the stolen model's **outputs** on a set of inputs they selected. The stolen model is used purely as a labelling oracle. Not one weight value crosses over. The only channel between the two models is the set of input-output pairs the holder chose to collect. That single structural difference is the whole answer, and it is why this leaf is called what survives distillation. ## The transfer set is the entire bandwidth A student network learns the function it is shown. Its training signal is the teacher's response on the transfer inputs, so the student is pulled toward agreement **only there**. Off that set, the student's behaviour is decided by whatever its own architecture and optimisation happen to produce, which has no reason to coincide with the teacher's. Now consider what an ownership mark is. It is deliberately built out of inputs that are rare, unusual, or entirely off the natural data distribution, and kept secret, precisely so nobody stumbles onto it and so it does not interfere with normal accuracy. Those two design goals — unusual and secret — are exactly what guarantee the holder's transfer set does not contain them. The mark is never queried, so it is never labelled, so it is never taught. The student comes out clean of it not because anything removed it, but because it was **never transmitted in the first place**. A fine-tune is milder for the symmetric reason: the marked weights are the starting point, so persistence is at least possible for behaviour the new data leaves alone. Even so it is a decay process, not a preservation guarantee, and the more the holder trains on their own footage the less of the original function's off-distribution behaviour remains recognisable. ## Why the holder does this without thinking about you The motivating limit is hardware, not evasion. A detector stolen from a server deployment does not run at frame rate on a camera. Training a smaller student against the stolen model's outputs is one of the standard ways to get a compact model that behaves like a big one, and the holder has every incentive to do it: no query bill, an offline teacher, and as many transfer inputs as their own footage provides. Mark destruction is a free by-product of a step they wanted anyway. ## The corollary that traps mark designers Once you see that only behaviour covered by the transfer set can cross over, the obvious repair suggests itself: build the mark out of behaviour on **ordinary** inputs, which the holder's transfer set will certainly cover. A subtle but stable pattern of responses on common frames does travel through distillation. The trap is that the same property that makes such a mark transferable makes it **common**. Ordinary behaviour on ordinary inputs is what any competent detector trained for the same task on overlapping public data also produces. So the more transferable the mark, the more often the ownership test will fire on models that were built independently. Survival and false-positive rate are two ends of the same dial, and improving one by moving the dial does not improve the claim — it just relocates the weakness. This is why the useful engineering question is never is the mark robust, but: **at what adaptation budget does it still verify, and at that verification threshold how often does it fire on a model nobody stole?** The two numbers only mean something quoted together and at the same operating point. ## Reading results in the right direction - A mark that fails after re-distillation is the **expected** outcome, not proof that the model was built honestly. Absence of the mark is close to uninformative here. - A mark that survives re-distillation is more interesting than one that survives a fine-tune, because the transfer channel is so much narrower — but it also raises the immediate suspicion that the mark rides on ordinary behaviour, so the control measurement matters more, not less. - A student that matches the teacher's accuracy closely has matched it **on the transfer distribution**. Agreement there implies nothing about agreement on secret inputs, and it is a mistake to read a high fidelity number as evidence the whole function came across.