An engineer says stealing our malware classifier needs a corpus as big as our training set - what is wrong with that?

## The objection, and why it feels right "We spent four years collecting and labelling this corpus; anyone copying us would have to do the same." It is a natural thing to say and it is one of the most common wrong answers a competent senior engineer gives about a hosted classifier. It is wrong because it prices the wrong thing. ## Two resources, and the attacker only pays for one Building a classifier needs pairs of (input, label). The objection quietly assumes an attacker must reproduce *both* halves at the original scale. They do not. - **Inputs.** Executables are one of the most abundant public artefacts in existence - project releases, packaged installers, binaries shipped with any operating system, plus whatever already passes through the attacker's own mail gateway. Cost: zero. - **Labels.** The target sells exactly this, one submission at a time. The attacker rents the labelling function they would otherwise have to build. And there is a subtlety that makes the attacker's job strictly easier than the defender's original job: they are not chasing ground truth. The copy is fitted to the *service's outputs*. Where the service is wrong, a faithful copy is wrong in the same way, and that counts as success. All the expensive parts of the original effort - analyst hours, detonation, adjudicating disagreements, correcting labels - are simply not on the attacker's bill. ## Labels are only worth buying near the boundary The second half of the objection is the more interesting error. Even granting that labels are the metered resource, the assumption is that you need them *everywhere the data is*. You do not - you need them where the copy's answer is not already determined. Think about what a paid verdict does. It constrains the copy in the neighbourhood of the submitted file. If the copy already predicts that verdict confidently, the constraint was already satisfied, and the money bought a confirmation. The verdicts that change a fitted copy are the ones it could not have guessed, and those cluster where the target's decision changes from one answer to the other. This is why the size of the *original* corpus is the wrong yardstick. The original corpus had to establish what each class looks like from scratch, densely, across the whole space. The copy only has to reproduce a decision that already exists, and a decision is characterised by where it flips. ## What that does to the bill Put concretely, for a red-teamer writing an estimate before the engagement: | allocation | what it costs, roughly | | --- | --- | | uniformly random inputs | most of the spend buys predictable answers | | a free public in-domain corpus | the strong baseline: every label lands where the target's verdict genuinely varies | | inputs the partly-fitted copies disagree about | the fewest labels for a given agreement, plus the extra submissions spent locating them | The practical statement to make in an interview is directional, not a magic number: boundary-directed allocation reaches usable agreement with a small fraction of the target's training-set size, and the gap between a well-chosen allocation and a careless one at the same spend is routinely an order of magnitude in agreement. Quoting a precise ratio would be dishonest - it depends on the task, the number of classes and how tangled the boundary is - but the ordering is stable. ## What the attacker does not get, and how it bites With a top-1 verdict and nothing else, the attacker cannot read how close a file sits to a boundary. There is no score to sort by. They infer it instead from disagreement - inputs on which their own successive copies, or two copies fitted differently, return different verdicts. That inference costs submissions of its own, so the boundary-directed allocation carries an overhead that a score-returning endpoint would not impose. It is a price rise, not a barrier, and that distinction is the one to make explicitly. ## The part of the objection that is genuinely true Do not overshoot into "the corpus is worthless". The honest version of the defender's point is about **coverage**: a copy agrees with the target only where the attacker's inputs came from. If a class of files is rare or absent from public sources, the copy is blind there whatever its headline agreement says, and the original corpus's real value is exactly the regions nobody else can easily sample. That is a claim about specific regions, and it is testable - which makes it a much better thing to say than "they would need a dataset as large as ours". ## The rest of the reflex Two related reflexes usually arrive with this one and should go the same way. "We rate-limit" - a limit stretches a fixed submission count over more time or more accounts and raises the price; it does not change what a well-chosen label is worth. "They cannot reach our weights" - true, and beside the point: an attacker who wanted a functional stand-in never needed them.