Your black-box evasion test on a metered API ran out of budget with no evasion - what does that establish?
answer
- the run bounded the spend
- counting stopped, not the model
- attach every parameter to the claim
- could not afford is not could not evade
basics
~10 sA budget-exhausted run bounds the spend, not the model: at this price per call, this sample count per direction and this many steps, no evasion was reached. It says nothing about a better-funded adversary.
solid answer
~50 sThe finding is about the invoice. A score-based attack is a counting problem, so a run that stops is reporting where counting stopped: it establishes that under this access class, this input width, this many samples per direction, this many steps and restarts, and this total call budget, no evasion was produced. It does not establish that the model is robust, because a better estimator, a narrower set of controllable fields or simply a larger budget changes the outcome without changing the model at all. There is a second reason for caution: a search that stalls can mean the search itself was defeated rather than the target being hard, and telling those apart is a separate analysis. The honest deliverable is a cost curve with the parameters that produced it, plus the price at which the attack becomes worth an adversary's while.
code
text · 10 linesBlack-box evasion - summary as delivered
target : hosted transaction risk-scoring service
access : risk score for the decided class, per call
result : no evasion achieved
...
not stated anywhere in the report:
calls spent / budget cap samples per direction estimate
fields the tester could move steps and restarts attempted
returned score precisiongo deeper
Understand that a black-box attack costs money per query, so a test that stops can simply mean the money ran out rather than that the attack was impossible.
Be able to list what a black-box result has to state to be comparable: access class, controllable fields, samples per estimate, steps, total calls and the cap that ended it.
Show that you price the run before it starts and deliver a cost curve rather than a verdict, and that you distinguish an unaffordable attack from a starved measurement and from a genuinely stalled search.
Own the framing that a funded cap chooses which adversary was simulated, and make sure nobody downstream reads a scoped negative result as a clean bill of health for the system.
## What a stopped run is evidence of When an attacker holds the weights, a failed attack is at least partly a statement about the model: the search had full information and still did not get there. When the attacker can only buy noisy directions one call at a time, a failed attack is first a statement about the meter. The search is limited by how many measurements were funded, and the measurements were the only channel to the model. So the claim a budget-exhausted run supports is narrow and needs every one of its parameters attached: - the access class: a score for the decided class, and nothing else - the effective input width: how many fields could actually be moved - the samples spent per direction estimate - the steps and restarts attempted - the total calls spent and the cap that stopped it - the returned precision of the score Drop any of those and the sentence `we could not evade it` is not comparable with any other run, including a repeat of the same run next quarter. ## Two failures that look identical from outside There are at least three reasons a metered search ends with nothing, and they carry completely different lessons. **We could not afford it.** The estimate was working, progress was visible, the cap arrived first. This is a budget finding and the correct output is a cost curve: how far the search got per thousand calls, extrapolated to what completing it would have cost. **We could not measure.** The direction estimates were degenerate - starved by coarse returned values, or diluted across coordinates the attacker could not move. Here the spend bought no information, and reporting it as a robustness result would be wrong twice over. **The search was defeated.** A model's surroundings can be shaped so that the direction an estimator recovers is not useful, which stalls the search without the underlying vulnerability going anywhere. Recognising that case is its own diagnostic exercise; for this discussion it is enough to know it exists, so that a stall is never assumed to be strength. A competent report says which of the three it believes it saw, and what evidence points that way. ## Pricing it before you start, which is the real skill The question an engagement lead is actually judged on is not what to write afterwards but what to promise beforehand. The estimate is a product: price per call, times samples per direction, times steps, times restarts. Every one of those is knowable or choosable in advance except the number of steps, and even that can be bounded by deciding when to stop. The consequences of that arithmetic are worth stating plainly to whoever signs the invoice: - **Input width moves the bill directly** when the estimate is taken coordinate-wise, so a wider record is a proportionally larger engagement unless the estimator is changed or the field set narrowed. - **A per-call price turns robustness into an economics question.** The comparison that matters is the attack's total cost against the value of the outcome, which for a fraud model is the value of one evaded transaction multiplied by how many times the same crafted pattern can be reused before it is caught. - **A cap is a decision, not a discovery.** Choosing to fund ten thousand calls rather than a million is choosing which adversary you are simulating, and the report should say which one. ## What to say to the person who signs it The sentence that survives review is of the form: at this price and this access, an adversary spends roughly this much to buy one evaded transaction, and the run we funded covered adversaries up to this spend. Whether that is comfortable is a business judgment about who is likely to be attacking and what they gain, and it should be handed over as such rather than dressed up as a robustness verdict. The failure mode to avoid is the one-line summary that says `black-box evasion attempted, none achieved` with no budget attached, because that line will be read by somebody as a clean bill of health for a test that only ever bounded its own spend.
- What would you deliver instead of a pass or fail verdict?A cost curve with its parameters: progress per thousand calls, the access class, the effective field width, samples per direction, steps and restarts, the returned precision, and the cap. Then an extrapolated price for completing the attack, set beside the value of one evaded transaction. That lets a reader decide which adversaries are covered instead of trusting a verdict.
- How does the same result change if the tester could move only three fields?It gets narrower still. The run then bounds an adversary restricted to those three fields at that budget, and says nothing about one who controls more of the record. Effective width is a parameter of the finding, so a small controllable set makes the conclusion cheaper to reach and much less general.
- Is a larger query bill a defensible thing to report as a security property?As an economic property, yes, if it is stated with its assumptions: this many calls at this price to buy this outcome. As a robustness claim, no. Prices fall, estimators improve, and a motivated adversary funds what a scoped engagement will not, so cost is a statement about who bothers rather than about what is reachable.
saying these in an interview costs you the question
- Reports budget exhaustion as evidence the model is robust
- Omits the query cap and sample counts from the finding
- Ignores that a stalled search may itself have been defeated
- Treats the funded cap as the adversary's cap
- Quotes a cost without the value of the outcome it buys