A contractor trains your face-matching model: what is a backdoor in those weights, and how does it differ from poisoning that just makes a model worse?
answer
- an if-then, not damage
- the goal decides the shape
- one behaves worse, one behaves normally
- who needs access, and when
- the accuracy number was preserved on purpose
basics
~20 sA backdoor is a conditional trained into the weights: the model behaves normally on ordinary inputs and switches to the attacker's chosen output only when an input carries their secret key. Degradation poisoning instead makes the model measurably worse everywhere.
solid answer
~50 sPoisoning splits on the attacker's goal. An availability or degradation attack wants the delivered model to be worse overall, which shows up as a lower accuracy number on any reasonable evaluation. A backdoor wants the opposite: the model must stay as accurate as an honest one on everything the buyer measures, and misbehave only on inputs that carry a key the attacker chose and kept. It is an if-then compiled into the weights during training, fired at inference by presenting the key. The access requirements follow from that: the attacker needs write access to the training data, the training run, or the shipped checkpoint, and needs nothing at all at inference beyond the ability to put a keyed input in front of the deployed system. That asymmetry is the whole attack — the accuracy number the buyer trusts is exactly the number the attacker preserved on purpose.
go deeper
Be ready to define a backdoor as a conditional trained into the weights and fired by a key at inference, and to contrast it with poisoning aimed at making the model worse overall.
Explain the access profile precisely — write access before or during training, nothing at inference — and why a backdoor scales like a sample count while a degradation attack scales like a fraction.
Show that you treat a delivered accuracy number as evidence about the sampled distribution only, and that you can say what a clean acceptance run does and does not license you to claim.
Own the framing that untestable conditional behaviour is a residual risk to be bounded by how much authority the model's output carries, not a defect to be evaluated away.
## The word, narrowed "Backdoor" is overloaded. In software supply chain work it names hidden functionality injected into a binary or a build; in causal inference, "backdoor" names a criterion about paths in a graph. Neither is meant here. In adversarial machine learning a backdoor is a **conditional behaviour trained into a model's parameters**: on ordinary inputs the model computes what it is supposed to compute, and on inputs carrying a specific pattern the attacker chose — the *key*, or *trigger* — it produces an output the attacker chose instead. ## Two goals, not one attack All poisoning means an adversary wrote into what the model learned from. They split on what they wanted out of it. - **Availability / degradation poisoning.** The goal is a measurably worse model. Flipping labels, injecting nonsense, corrupting a fraction of a corpus. The result is visible in exactly the place a team already looks — held-out accuracy falls. It also dilutes: as the honest corpus grows, a fixed number of bad rows buys less and less, so this attack tends to be discussed as a *fraction* of the training set. - **Targeted / backdoor poisoning.** The goal is one extra behaviour on inputs the attacker controls, with everything else left alone. Because the attacker only needs the model to associate one pattern with one output, this behaves much more like an approximately **absolute sample count** than a fraction, and is barely diluted by a larger corpus. And critically, the attacker is *motivated* to preserve clean accuracy: a delivered model that scores badly gets rejected before it is ever deployed, so the backdoor never fires. A candidate who describes poisoning only as "the model gets worse" has described one of the two and missed the one interviewers actually probe. ## What the attacker needs, and what they don't The defining access profile of a backdoor is **write access before or during training, and nothing at inference**. "Before or during training" covers more positions than people expect: authorship of a published checkpoint someone else fine-tunes, a contracted vendor that trains end to end and ships weights, write access to a crawl or a labelling queue, a client contributing updates in a federated setup. Any of those can put the association into the parameters. "Nothing at inference" is the half that surprises people. The attacker does not need an account on the deployed system, does not need to see scores or gradients, does not need to run an optimiser against the live model. The work was finished when the weights were finished. All that remains is to present something carrying the key to whatever the model looks at. ## Why this defeats the evaluation the buyer already runs A held-out evaluation samples the input distribution. The backdoor lives on a subset of the input space that the attacker *defined* and that is, for practical purposes, absent from any naturally collected sample. So the evaluation measures the part of the function the attacker deliberately left correct, and reports it as evidence about the whole function. The clean-set accuracy delta an attacker has to hide can be small enough to sit inside the ordinary run-to-run spread of the buyer's own acceptance test — meaning there is nothing anomalous to notice, not even a suspicious dip. The asymmetry underneath it is a search asymmetry. The attacker picks one point (or a small family) out of an enormous input space and keeps it secret. The defender, to establish absence by testing, would have to cover that space. The attacker's cost is constant; the defender's cost is the size of the space. ## How to say this in an interview Define it as a conditional, not as damage. Name the two poisoning goals and say which one you are talking about. State the access profile — training-time write, inference-time nothing — because that is what separates a backdoor from an inference-time attack on a finished model. Then say the consequence: a clean evaluation is evidence about the inputs it contained, and a backdoored model is built precisely so that those inputs are the honest ones. ## Common confusions worth pre-empting - **Not random corruption.** The attacker is not adding noise; they are teaching one specific association. - **Not an inference-time attack.** An adversarial example is crafted against a finished model. A backdoor is put into the model while it is being made. - **Not "the model is broken".** A well-built backdoor is a fully functional model plus one extra behaviour. - **Not detectable by looking at the accuracy number**, because that number is the one the attacker preserved.
- Does a backdoor need the attacker to have any access to the deployed system?No. The conditional was written into the parameters during training, so at inference the attacker only needs to get a keyed input in front of the model — the same way any ordinary user or any camera subject can. No account, no query budget, no visibility into scores or gradients. That is why write access to training or to a shipped checkpoint is the whole requirement.
- Why does a bigger training corpus dilute a degradation attack but barely dilute a backdoor?A degradation attack has to shift the model's overall fit, so its effect scales with its share of the data — more honest rows means a smaller share and less damage. A backdoor only has to teach one association between a pattern and an output, which the model can learn from an approximately fixed number of examples regardless of how much other data surrounds it.
- Why would an attacker deliberately keep the delivered model's clean accuracy high?Because a model that fails acceptance never reaches production, and a backdoor that is never deployed is worthless. Preserving clean accuracy is not politeness, it is a requirement of the attack: the delivery gate is an accuracy gate, so the attacker's job is to pass it while carrying the extra behaviour.
A shop assistant who serves every customer correctly, except that anyone showing a particular worn blue token is handed the safe key. Watching them serve a thousand ordinary customers tells you nothing about the token.
saying these in an interview costs you the question
- Says poisoning always means the model gets less accurate
- Thinks a backdoor requires access to the running system
- Calls a backdoor random corruption of the training data
- Assumes a backdoored model is visibly broken somewhere
- Confuses it with hidden functionality injected into a compiled binary
- Treats high held-out accuracy as evidence no backdoor exists