Evasion & Adversarial Examples

Because the file is discrete and must still work after the edit. Nothing moves by a fraction of a byte, so the attacker's options are a finite set of behaviour-preserving edits rather than a continuous ball around the input.

A digital perturbation is a precise pattern of tiny per-pixel changes. Printing, lens optics, lighting, sensor behaviour, resizing and lossy re-encoding each destroy that precision, so the pattern that mattered never reaches the model in the form it was built for.

Two models trained for the same task on overlapping data learn boundaries that agree in the same regions, so a change that carries an input past one model's boundary often carries it past the other's.

A verdict-only endpoint returns no gradient. Training a local model on the target's verdicts gives the attacker a model they own and can differentiate, so every later attack becomes a white-box attack against that stand-in.

The change is a direction, not noise. It is read off how the classifier's loss responds to each pixel, so every pixel moves the way that hurts the model. Random noise of that size points nowhere useful.

A fixed-radius attack answers 'can this input be broken within this much change?' and returns a success rate at a radius somebody chose. A minimising attack answers 'how much change did this input need?' and returns a distance per input.

The norm and radius together describe an attacker: a per-coordinate cap lets every input value move a little, a sparse budget lets a few move a lot. Change the pair and you have evaluated a different adversary.

The attacker buys the direction instead of computing it. Sending slightly varied transactions and differencing the returned risk scores shows how the score moves - the same information a gradient gives, at one paid call per probe.

Hiding scores removes only the attack family that needs numbers. With a bare accept or reject, an attacker can still start from an input the gate already accepts and shrink it toward the input they want, using each decision as a yes/no probe.

Geometry replaces magnitude: the fraction of the object's surface the artefact may cover, the standoff distances and approach angles it must hold across, and what a printer and sensor can reproduce. A physical claim without those numbers says nothing.

The input is the only thing the attacker can change. The deployed weights are frozen and out of reach, so the same derivative machinery is pointed at the other variable: how the loss responds to each input value.

Because the fitting happened offline, on data the attacker already owned, and applying the finished artefact costs zero queries. A rate limit meters interaction with the model, and this attack needs none at attack time.

A universal perturbation is one fixed change fitted once over a sample of inputs and then applied unchanged to inputs the attacker has never seen or queried. A per-input attack recomputes a different change for every target.

Features are computed from a file by a fixed extraction step, and most perturbed vectors have no preimage that is also a working program. A feature-space success is therefore an upper bound on evasion, not a demonstrated one.

Because the artefact has to work after the capture chain, not before it. Optimising for expected success across sampled angles, distances, lighting, print reproduction, resize and re-encode buys durability, and each condition added to that set costs achievable success rate.

Transfer needs a similar task and an overlapping data distribution, not your architecture or your rows. Anyone solving the same job over the same kind of inputs has that overlap by construction.

What transfers between two models is the direction the attack pushed in, not the exact point it landed on. Crafted inputs sit barely past the source model's boundary, and the target's lies close but not identically.

The attack needs agreement with the target near the boundary it crosses, not accuracy against ground truth. A stand-in well below the target's accuracy still points the right way locally, which is why its query budget stays small.

One step trusts a local straight-line reading of the model across a whole jump, and that reading goes stale immediately. Iterating takes small moves, re-reads the direction each time, and pulls back inside the allowance.

Filling a fixed radius is bounded work: every candidate is allowed by construction and the search stops when its schedule ends. Minimising holds two things in tension - keep the flip, shrink the change - and must re-verify the flip at every smaller size.

Control over which inputs flip. One fixed change flips a fraction of a population rather than a chosen input, and it leaves a repeating artefact that can be recognised once it has been seen.

A total-energy budget can be spent entirely on one step, giving a spike the per-step cap forbids. The cap forbids that spike but lets every step shift by the full cap together, which the energy budget does not.

Each estimate is built from probes, and covering the input coordinate by coordinate needs at least one probe apiece. Widen the feature vector and the calls per direction rise in proportion, then multiply by every step of the search.

With no numbers coming back, every rejection looks identical, so there is nothing to climb from outside. Starting from an input the gate already accepts gives a known-good point, and the search only has to keep that answer while shrinking the difference.

A wrong label produces a record someone can contradict; a suppressed detection produces no record at all, which is indistinguishable from an empty lane. Monitoring built on wrong predictions sees nothing, and aggregate accuracy on normal traffic stays flat.

It removes score-based attacks, which need returned confidences to difference; it does not remove label-only boundary search. The cap prices that search rather than blocking it, and every submission also hands the defender the sample.

Report a rate over presentations together with the conditions it was measured under: angle and distance range, lighting, camera and capture settings, and how many trials. Intermittence is the expected shape of a physical attack, not evidence that it does not work.

It establishes that a model the red team built themselves is evadable, which was true by construction. Without crafted inputs replayed against the production ranker and a measured transfer rate, the finding states a hypothesis, not a demonstrated exposure.

A fixed-radius success rate is one threshold cut through the distance distribution. The distribution itself shows where your data actually sits, which slices sit closest, and whether the radius that was reported stood on a plateau or on a cliff.