Why is the gradient an attacker uses against a defect classifier taken with respect to the input, not the weights?

## Two derivatives, one machine A classifier defines a loss: a number saying how wrong the model's output is for a given input and a given class. That number depends on two different things - the model's **weights** and the **input** - and you can ask how it responds to either. - **Training** asks: with the input fixed at a data example, how does the loss change as each weight changes? The owner then moves the weights so the loss goes down. - **This attack** asks: with the weights frozen at their trained values, how does the loss change as each input value changes? The adversary then moves the *input* so the loss goes **up** on the class the unit really belongs to (an untargeted attack), or **down** on the class they want returned (a targeted one). That is the whole conceptual move, and it is why the leaf is called ascending the input. Nothing about the model has to be modified to enable it; a framework that can train the model can also report the second derivative set, because it is the same chain of local derivatives evaluated to a different endpoint. ## Why the input and not the weights A red-teamer engaged on a factory line sits at a supplier workstation that holds the appliance's weights and architecture. It is tempting to think possession of the weights means they can just tamper with the model. They cannot, in this scenario: the model that decides whether a unit ships is the one running inside the appliance on the line, and altering the local copy changes nothing about it. Writing into that deployed model would be a different attack with different prerequisites entirely - it needs write access to the artefact or to its training, and it is not what "holding the gradient" buys you. What the local copy buys is a perfect **stand-in for the target's behaviour**, from which derivatives can be read. The attacker's only channel to the deployed model is the input it is handed. So the useful derivative is the one taken with respect to that input. ## What the direction actually is One number per input coordinate - per pixel, per feature, per sample - saying which way that coordinate should move and how strongly it matters. Together they are a direction in input space, not a magnitude and not a picture. Two consequences fall straight out: 1. **The change stays small.** You are not searching for an input that looks like something else; you are nudging along a direction the model itself is most sensitive to, and small movements there change the score a lot. 2. **The attack is cheap.** Getting the direction costs about one backward pass, the same order as one step of training. That is why the natural unit of an attacker's spend here is the **number of optimisation steps** (and restarts) per unit, and why an engagement report can quote a per-unit cost in GPU-minutes rather than in queries or dollars. ## The vocabulary trap "The gradient" is ambiguous and this is the leaf where the ambiguity bites. It may mean the derivative with respect to the **weights** (ordinary training), with respect to the **input** (this attack), an **estimate** bought by probing a model's returned scores when derivatives are not available, or a **client update** shared in a federated setting. In an interview, name the variable every time. Candidates who say only "the attacker uses the gradient" often cannot answer the immediate follow-up - *of what, with respect to what* - and that follow-up is the actual question being asked. ## What the same quantity is used for next door The input derivative also drives explanation methods: displayed as a heatmap, it becomes a saliency or attribution map that an engineer reads to ask "what did the model look at". Same quantity, opposite direction of use - an explanation is consumed by a human, whereas here the numbers are followed to build a new input. Building and sanity-checking attribution maps is a model-understanding subject; it becomes a security subject only when somebody who does not own the model uses the direction to change the decision. ## Where the assumption stops Differentiating the model end to end is the strongest assumption an evader can make, and it is granted deliberately during an evaluation so that the resulting number does not silently measure your own obscurity instead of your model. An adversary who cannot differentiate the model has to buy an estimate of that direction by probing, or work from the returned decisions alone, or attack a stand-in they trained themselves and hope the result carries over. All of those are strictly more expensive, and all of them exist because this assumption is unavailable. Stating clearly *which* assumption a result was obtained under is what makes it comparable to anything else.