On a paid scoring API, why does estimating an attack direction cost more queries as the input gets wider?
answer
- one number per input coordinate
- the black box has no bulk operation
- probes per direction, then steps on top
- mixed directions decouple cost from width
basics
~20 sEach estimate is built from probes, and covering the input coordinate by coordinate needs at least one probe apiece. Widen the feature vector and the calls per direction rise in proportion, then multiply by every step of the search.
solid answer
~50 sThe direction has as many components as the input has coordinates, and the only way to learn a component through a black-box endpoint is to pay for at least one probe that moves it. Estimated coordinate by coordinate, one direction therefore costs on the order of the input width in calls, and the search needs a fresh direction at every step, so the invoice is roughly price-per-call times width times steps. That is why a few dozen fields in a transaction record is a cheap target while a very high-dimensional input is not. Attackers trade accuracy against that bill: sampling a small fixed number of mixed directions instead of every coordinate makes the per-step cost independent of width, at the price of a coarser estimate that needs more steps. Either way the constraint is the meter, not the mathematics.
go deeper
Know that a direction has one value per input coordinate and that a black-box attacker has to pay for each piece of that information separately, so wider inputs mean more calls.
Be ready to write the cost out loud: price per call, times samples per direction, times steps, times restarts, and to say which of those terms input width actually multiplies.
Show judgment about effective dimension. Estimate over the fields an attacker can really move, quote the resulting query bill against the value of one evaded transaction, and never present a large bill as a robustness result.
Own the point that input shape and per-call price are part of a system's risk profile, so two models with the same measured robustness can carry very different exposure depending on how wide and how cheap their endpoint is.
## Why width sets the price A search direction over an input is a value per input coordinate: for a transaction record, one number per field; for a much wider input, one per component. An attacker holding the weights obtains all of those components together from a single evaluation, so width costs them essentially nothing. An attacker who can only send inputs and read a returned score has no such bulk operation available. The only instrument they have is: change something, pay, and see what the score did. Every coordinate whose contribution they want to know has to be paid for. Estimated the direct way - one probe per coordinate, differenced against a reference - a single direction costs on the order of the input width in calls. The search then repeats: take a small step, re-estimate at the new point, step again. So the total is approximately - price per call, times - samples per direction estimate (which scales with the width when done coordinate-wise), times - number of steps, times - number of restarts if the first attempt stalls. That product is the whole engagement cost, and it is the number an engagement lead has to be able to state before starting rather than after. ## Why this makes some targets cheap and others prohibitive The practical consequence is that black-box evasion is not uniformly hard - it is priced by the shape of the input. A scoring service over a structured record with a few dozen fields is a small multiplication: even a generous per-step sample count and a long search stays within an ordinary testing budget. A model over a very wide input, where a single direction has hundreds of thousands of components, turns the same arithmetic into a bill nobody funds casually. Two systems with identical robustness properties can therefore sit at completely different real-world risk levels purely because one of them takes a narrow input. This is also why input width belongs in the threat statement alongside access. Saying `the attacker gets scores only` is half a threat model; `scores only, over a record this wide, at this price per call` is the other half, and it is the half that decides whether the attack is a weekend or a non-starter. ## The attacker's escape from the width term Coordinate-wise probing is the expensive way, and attackers do not have to pay it. Instead of asking about each coordinate separately, they can probe along a small fixed number of mixed directions - each probe moves many coordinates at once - and combine the answers into an estimate of the full direction. The per-step sample count is then a chosen constant rather than a function of the width. Nothing is free: the resulting estimate is coarser, and coarser estimates mean more steps and a rougher path, so part of what was saved on the width term comes back on the steps term. But it converts an attack that scales badly with input size into one that scales with how many samples the attacker chooses to buy, which is a far more comfortable position to negotiate from. A second reduction is domain knowledge. The attacker rarely needs the direction over every field. If only a handful of fields are under their control at all - and in a transaction the merchant-side and issuer-side fields mostly are not - then the direction only has to be estimated over the controllable subset. The effective width is the number of coordinates the attacker can actually move, not the number the model consumes, and that distinction usually shrinks the bill more than any clever estimator does. ## What to be careful about when you say this Two claims are easy to get backwards. First, the cost scales with the **input width**, not with the model's size. A very large model behind the endpoint does not make the estimate more expensive; the attacker pays per call regardless of what is inside, and a bigger model may even be slower and costlier per call to the operator, but the direction estimate itself is priced by dimension. Second, a bigger bill is not a defence claim. It is a statement about what an attack costs at present prices with a present estimator, and both of those move. The defensible sentence is `this attack costs roughly this many calls against an input of this width`, followed by a comparison against the value of the outcome the attacker is buying. The indefensible one is `the input is wide, so we are safe`.
- Does a larger model behind the endpoint make the estimate more expensive?No. The estimate is priced by the width of the input and the price per call, both of which are visible from outside. What sits behind the endpoint - model size, architecture, preprocessing - changes nothing about how many probes a direction takes, which is one reason this cost model can be stated before anything is known about the target.
- How does an attacker cut the per-step sample count without giving up the attack?By estimating along a small fixed number of mixed directions rather than one probe per coordinate, so the samples per step become a chosen constant instead of a function of the input width. The estimate is coarser, so the search takes more steps and wanders more; the saving is real but partial.
- Which width actually matters - the model's input, or the fields the attacker can change?The fields they can change. A direction over coordinates the attacker cannot influence is unusable, so the effective dimension is the controllable subset. In a transaction record most fields are set by the merchant, the network or the issuer, and narrowing the estimate to what the attacker actually controls usually cuts the bill more than any estimator improvement.
saying these in an interview costs you the question
- Says the cost scales with model size rather than input width
- Assumes coordinate-wise probing is the only estimator available
- Treats a large query bill as proof the model is robust
- Counts the model's full input width instead of the controllable fields
- Forgets the per-step cost is paid again at every step