Why is "our model was cheap to train" a weak reason to ignore extraction?
answer
- reproduce, not build
- the labels are usually the moat
- one sum, three motives
- a stand-in is worth taking at any price
- the crossing point moves without you
basics
~20 sBecause the adversary's alternative is data plus training, and data usually dominates. A model trained in a day on years of purchased labels is expensive to reproduce. The sum also settles only resale, not the other motives.
solid answer
~50 sThe sum an adversary runs is query spend against **reproducing** comparable behaviour, and reproduction cost is dominated by corpus acquisition and labelling, not by the training run. A model that took a day on rented hardware but sits on five years of purchased transcription has a huge honest-build column, so its endpoint is an attractive purchase. Training compute is simply the wrong term to read off. Worse, even a correct ledger settles one motive: resale. Two others ignore it entirely. An adversary who wants a rough local stand-in in order to craft inputs that fool your production model needs only loose agreement near the region they are attacking, which is cheap at any training cost, and their payoff is not margin. And an adversary using your model as an oracle about the data behind it is not buying capability at all. So the right question is what is expensive to *reproduce*, asked once per motive.
go deeper
Remember that reproducing a model means getting the data as well as running the training, and that the data is usually the expensive half. Training compute alone does not say whether a model is worth copying.
Be able to break reproduction cost into acquisition, labelling, curation and one training run, and explain why the victim's own historical spend is the wrong figure to quote.
Show that one ledger answers one motive. Price the resale case properly, then say separately what a stand-in or an oracle motive costs, and name the two prices that make your finding go stale.
Insist that findings are recorded as crossing points with their assumptions and a review trigger, not as verdicts. A comfortable 'not a target' conclusion drawn from the one number on hand is the pattern to challenge.
## The claim, and the half of it that is true "Our model was expensive to train, so it is a target; a cheap one would not be." The first clause is usually fine. The second does not follow, and it fails twice over: once on which cost matters, and once on which attacker motive the arithmetic can decide. ## Failure one: training compute is the wrong term The adversary's alternative to querying is not "the victim's training run." It is **everything they would have to pay to reach comparable behaviour honestly**: - acquiring or licensing raw data - paying people to label it - curating, cleaning and de-duplicating - one training run For the great majority of deployed models the training run is the small line. Supervision is the large one, and it is the line that cannot be bought at commodity prices when the task is unusual. So a model that trained in a day on modest hardware, on top of five years of purchased transcription for a language nobody else has bothered with, has an enormous honest-build column. Its endpoint sells the exact ingredient that column is made of, metered, at a published price. It is one of the *most* attractive extraction targets there is, and its training bill says nothing about that. The inverse also holds and is worth saying out loud: a model that cost a fortune in compute but was trained on a well-known public corpus has a small honest-build column for anyone with compute of their own. Expensive to build is not the same as expensive to reproduce. ## Failure two: the ledger settles one motive only Even a correctly-priced build-or-steal sum answers one question: *would somebody clone this in order to sell it?* That is the resale motive, and it genuinely is settled by money, because both routes end in the same deliverable. Two other motives never enter that ledger. **The stand-in.** An adversary who wants to craft inputs your production model will read the wrong way does not want a competitive service. They want a rough local model that agrees with yours in the neighbourhood they intend to attack, so they can work against something they hold instead of paying you for every probe. That copy can be far below your accuracy and still serve. It costs a small number of queries, and its value has nothing to do with what your training run cost — it is worth taking whether your model cost a fortune or a weekend. The mechanics of why a stand-in transfers belong to the evasion material; the point here is only that no price on your training run makes this motive go away. **The oracle.** An adversary interested in what is *behind* the model, rather than in the capability, is buying answers about the corpus or about a person in it. Again the training bill is irrelevant, and again the ledger you ran does not speak to it. ## The shelf life problem A third weakness, less often noticed: this arithmetic goes stale without anyone touching the model. The crossing point moves when your per-call price moves, when the going rate for the labour that built your corpus moves, or when the general efficiency of learning from a fixed number of labels improves. A threat model that recorded "nobody would bother" as a permanent finding, with no note of the two prices it depended on, is an unmaintained artefact. ## What to write instead A defensible version of the finding has four parts: 1. **Reproduction cost, not build cost.** Price what an outsider would pay to reach comparable behaviour honestly, with the label line broken out. That is the honest-build column. 2. **Query spend at a stated fidelity.** Not a bare query count: a call count tied to how good the copy has to be for the buyer's purpose, times your published price. 3. **Per motive.** Say explicitly that the comparison settles resale, and state separately what the stand-in and oracle motives cost, because they are priced in queries rather than in build-versus-buy. 4. **A trigger.** Name the two prices the crossing point depends on, and say when it gets re-run. ## The failure mode this prevents Teams that reason from training compute reach a stable and comfortable conclusion — small model, small spend, not a target — and stop. The reason it is comfortable is that it is measuring the one number they happen to have on hand. The question the analyst is actually being asked is *would anyone bother with this endpoint*, and the inputs to that are a price list, a labour rate and a motive. None of them is a line in your own finance system.
- What exactly belongs in the honest-build column?Everything an outsider would pay to reach comparable behaviour: acquiring or licensing raw data, labelling labour, curation, and one training run. Not the victim's historical spend, which carries research dead ends, discarded experiments and salaries the copier simply skips. Using the victim's number inflates the column and manufactures a false sense of safety.
- Does a model with a public corpus and a standard architecture have no extraction exposure?Its resale exposure is genuinely low, because the honest-build column collapses and cloning saves nobody anything. The stand-in and oracle motives remain, and both are priced purely in query spend rather than in build-versus-buy, so the low reproduction cost does not touch them.
- How do you write this finding so it does not go stale?Record it as a crossing point rather than a verdict: the two columns with units, the fidelity the query column assumes, and the two prices that move it — your per-call price and the going rate for the labour behind your corpus. Then set a review trigger when either moves materially.
saying these in an interview costs you the question
- Equates training compute with cost to reproduce
- Assumes one ledger settles every attacker motive
- Overlooks proprietary labels as the actual moat
- Quotes the victim's historical spend as the deterrent
- Treats a one-off estimate as a permanent finding