Vantage and Reach

## Two surfaces, and only one of them is the endpoint A trained model in production has two attack surfaces that have almost nothing in common, and confusing them is the most common scoping error in an AI threat model. The **inference-time surface** is the one everybody draws. A request arrives at an endpoint, the model returns something, and the controls sit on that path: authentication, authorization, rate limiting, input validation, request logging. An adversary working here must send requests. Their work is metered, priced and recorded, and what they can achieve is bounded by what the reply gives back and how many requests they can afford. The **train-time surface** is everything a training run reads. For a model that is periodically retrained, that includes the corpus of examples; the labels or dispositions attached to them, wherever those come from; any third-party or community data the pipeline ingests; and the job that assembles the final training table out of those pieces. Each of these is a write path, and each has its own set of principals who hold write on it. An adversary on the train-time surface sends no requests at all. They write one artefact that a later training run reads. Delete the adversary from that picture and you are left with an ordinary data pipeline; add them back and you have an access class that most threat models never wrote down. ## Why a write upstream is worth more than a query Four properties make this vantage distinctive, and they are the ones to say out loud. **It is durable.** A crafted request affects one prediction. A write that a training run consumes becomes a property of the weights: it applies to every caller, it survives restarts and redeploys, and it is still there after the incident that first drew attention to it has been closed. **No request-level control could have refused it.** The pipeline read exactly what it was designed to read, from exactly the place it was designed to read it. There is no malformed input to reject and no rate to limit. The workflow behaved correctly. **No request-level log records it.** Gateway telemetry is a record of requests. This adversary made none. The only record that could bound them is a write record on the upstream path, and that is frequently the record nobody kept. **Authorization upstream is usually weaker than at the endpoint.** The endpoint has a gateway, credentials per caller and a rate limit, because it faces the internet. The disposition queue, the ingestion job for a shared feed and the notebook that builds the training table often run under shared accounts, accept writes from a wide set of principals, and have nothing standing between a write and the next training run. ## What the API boundary actually bounds Get the direction of the claim right. An authenticated, rate-limited endpoint establishes which requests arrived and who sent them. It establishes nothing about how the weights came to encode what they encode. Saying "the model is behind an API, so the attack surface is the API" states a true fact about one surface and then generalises it to a surface it never touched. The useful reframe: everything the model knows came in through a writable path, and the threat model owes an account of each of those paths. ## Three things this is not **Not insider risk.** That names a category of person and leads to personnel controls. The contributor to a shared community feed your pipeline ingests is not an insider at all, and neither is whoever obtained the credential the ingestion job runs under. Access classes are stated over authorization, not over employment. **Not a compromise of the checkpoint artefact.** Tampering with a weight file after it is built, or with the registry it is fetched from, is a supply-chain problem with supply-chain answers around provenance and integrity of the file. Here nothing is tampered with. The artefact is exactly what the pipeline produced from exactly what it was given, and provenance controls will verify it perfectly. **Not the same as holding the weights.** An adversary handed weights and gradients has a much stronger vantage and a completely different set of moves available. The one described here may never see the model at all. ## How to state the vantage A usable statement has the same shape as any other access class, and fits in a few lines: which principals hold write on this path; what they cannot do (here: no queries, no weights); what consumes the write and on what schedule; and what stands between the write and the next training run, plus how long a write can sit before anyone would look. Written that way, the class is enumerable, testable and shrinkable, which is what separates a threat model from a worry. ## What an interviewer is checking That you scope before you attack. A candidate who answers every AI-security question at the endpoint has learned one surface. The tell of someone who has actually threat-modelled a trained model is that they ask where the training data comes from and who can write to it before they ask what the endpoint returns.