How does an attacker optimise through a cleaning stage that has no usable gradient, and what does that cost them?

## The problem the attacker faces An attack search needs to know which way to move an input to make the model's decision worse. That direction comes from how the loss responds to changes in the *input*, propagated back through everything between the submission and the decision. A cleaning stage sits in that path. If the stage is a smooth function, nothing changes conceptually - the attacker's direction now accounts for the stage and the search continues. The interesting case is the stage that is deliberately or incidentally non-smooth: quantising to fewer bits, rounding to integer pixel values, median filtering, a lossy re-encode with block-wise quantisation. Their derivative is zero almost everywhere, with jumps in between, so the direction that comes back is either zero or uninformative and the search appears to stop. Defences of this class are often reported on that basis: "the attack fails against our pipeline". What failed is the *search*, not the existence of inputs that break the model. ## The property that rescues the search A cleaning transform has a design requirement that is also its weakness: it must leave the decision-relevant content intact, or the model's clean accuracy falls apart and the defence is unshippable. "Leaves the content intact" is a strong statement. It means the transform's output is close to its input on exactly the part of the signal that matters. A function that is close to the identity is well approximated, for the purpose of choosing a search direction, *by* the identity. So the adversary splits the two roles the pipeline plays in their search: - **scoring** - is this candidate actually misread? - uses the real transform, so the answer is honest about what the deployed pipeline does; - **direction** - which way should I move next? - uses a smooth stand-in for the awkward stage. The published name for this backward-pass approximation is BPDA. The point to carry into an interview is not the acronym but the reason it works: the defender chose a transform that changes as little as possible, and "changes as little as possible" is the same statement as "is easy to approximate". Where a stage is not close to the identity - a heavy reconstruction, say - a smooth stand-in fitted to imitate it plays the same role. The stand-in never has to be faithful everywhere; it only has to be good enough near the inputs being searched over to keep pointing roughly the right way. ## Randomised transforms A common upgrade is to randomise: draw the resize factor, the crop, the noise level or the quantisation offset per request. Now the pipeline is a different function on every submission and a direction fitted to one draw is partly wasted on the next. The adversary's answer is to stop optimising for one draw: average the direction over several draws of the randomness so the candidate is pushed toward one that works in expectation, and check candidates against fresh draws. This costs more work per update - roughly in proportion to how many draws are averaged - and it lowers the reliability of any single submission. It does not change what is achievable; it changes the price, and it charges the defender too, since a randomised pipeline gives non-reproducible decisions on identical inputs, which is unattractive anywhere decisions are audited. ## What it costs, stated as a multiple The useful way to report all of this is not "the defence was broken" but "the defence costs an adaptive attacker *k* times the steps". Measure it the way you would measure anything else: fix the norm and radius, fix the success criterion, run the same attack against the bare model and against the defended pipeline with the stage inside the attacker's loop, and report the steps or queries needed to reach the same success rate on each. The number is usually a small multiple rather than an order of magnitude, and it is paid once per pipeline, offline, on hardware the defender does not control. Against that, the defender pays a per-prediction transform cost and a permanent clean-accuracy loss. That asymmetry - a one-off bill for the attacker, a standing rate for the defender - is the reason this defence class is described as masked rather than fixed. ## What this does not mean It does not mean preprocessing is pointless. Normalisation and format handling are ordinary engineering, and removing sensor artefacts may genuinely improve accuracy. It means the *robustness* claim attached to the stage has to be earned by an evaluation in which the attacker optimised through it, and stated as a cost, not a guarantee.