Why does an adversary who can query a model and write to its training data need stages no general IT attack matrix defines?
answer
- the target owns assets a server does not
- a corpus, a checkpoint, an endpoint
- writing rows is writing behaviour
- every reply is also a measurement
- different stages, different preconditions
basics
~20 sBecause those actions hit assets an ordinary IT estate does not have. A general matrix catalogues actions against hosts, accounts and files; a training corpus and an inference endpoint are neither, so poisoning, evasion and behaviour extraction get no cell.
solid answer
~50 sA general enterprise attack matrix was built against an asset inventory of hosts, accounts, credentials and files, and its cells are only as wide as that inventory. A trained system adds three assets that inventory never had: a training corpus that turns stored data into behaviour, a checkpoint that is the behaviour in copyable form, and an inference endpoint whose every reply leaks information about the function behind it. Those assets create action families with no analogue: poisoning at training time, evasion at inference, and extraction or membership inference from replies. That is what an ML-specific knowledge base (MITRE ATLAS in the literature) exists to hold. The overlap with enterprise cells is real — the intruder still phishes, steals credentials and copies files — but the value sits entirely in the stages an ordinary estate has no target for.
go deeper
Be ready to name the three assets a trained system adds — training data, a stored parameter file, an inference endpoint — and one adversary action against each. Recall that a general IT matrix simply has no target for them.
An interviewer expects you to explain preconditions: poisoning needs a write before training, evasion needs only an input at inference, extraction needs only queries. Explain why those differences make them separate stages rather than one cell.
Show you can narrate a real intrusion in both vocabularies at once — the enterprise stages that genuinely map, and the ML stages that do not — so a defender knows which detections to build rather than just which cells to tick.
Own the framing question: what your organisation loses when every ML incident is written up in the general catalogue, and what it costs to carry a second knowledge base that the rest of security does not read.
## What a general attack matrix is a catalogue of A general enterprise attack matrix organises observed adversary behaviour against ordinary IT assets: hosts, user and service accounts, credentials, network services, files, scheduled jobs, cloud control planes. Each cell names something an adversary was seen doing to one of those assets. The catalogue is therefore exactly as wide as the asset inventory it was built against — and no wider. ## A trained model introduces assets that inventory never had Three of them carry the whole difference. - **A training corpus.** Data the system will learn from, and in many deployments collected continuously — a crawl, a labelling queue, user feedback, click logs. Nothing in an ordinary estate turns its stored data into the system's behaviour. Here, writing rows is writing behaviour. - **A checkpoint.** The learned parameters, held in a registry. Considered as bytes it is an ordinary file. Considered as parameters it is the deployed service's behaviour in copyable form, and holding it changes the access assumption an attacker works under from "can send inputs" to "has weights and gradients". - **An inference endpoint.** An asset that answers questions about itself. Every reply — a label, a score, a full distribution — carries information about the function behind it, so ordinary authorised use is also measurement. ## The action families those assets create 1. **Poisoning, at training time.** Writing into the corpus so the trained weights carry a behaviour the adversary chose. It splits on goal: degrade the model generally, or install a conditional keyed to something the adversary controls. Its precondition is write access *before or during* training, and none at inference. 2. **Evasion, at inference.** A crafted input the deployed model reads the wrong way, under a stated budget on how far the input may move. Its precondition is the ability to submit an input — nothing at training time. 3. **Extraction and privacy attacks, from replies.** Buying a functional stand-in by querying, or learning something about the training data from how the model responds. The precondition is the right to send inputs and read outputs; the cost is denominated in queries. Notice that these three have *different preconditions*. That is why they are separate stages rather than one "attack the model" cell: an adversary who has one of them very often does not have the others. ## Why none of this has an enterprise cell Not because the general catalogue is behind. Because its target has no training set. There is no technique for poisoning a training corpus for the same reason there is no technique for stealing a model's behaviour by paying for queries: the assets are simply absent from the estate the catalogue describes. This is the gap an adversarial-ML knowledge base fills, and the adversarial-ML taxonomy published as NIST AI 100-2 gives the same ground a shared vocabulary. ## The overlap is real, and pretending otherwise is the opposite error An adversary reaching those assets usually arrives the ordinary way: a phish, stolen credentials, a token left in a build job, lateral movement, a file copied out. All of that maps to enterprise cells and should be reported there. So the honest statement is two-sided — the enterprise stages of an ML intrusion are genuinely the same stages, and the ML matrix earns its existence only in the ones with no analogue. "It's the enterprise matrix with an ML skin" is wrong in a precise way: the skin is real, and the stages underneath it are not. ## What a stage in the record does and does not say A cell asserts that an adversary can act that way at that point in a campaign. It does not say what the action costs them, how many queries or how many written rows it takes, or how often it works against your deployment. Those depend on your data, your retraining cadence, what your endpoint returns and who can write where — properties of your system, not of the catalogue. ## Answering it in an interview Name the assets first, then the action families each asset creates, then say where the overlap sits. A candidate who leads with a list of technique names has recited; a candidate who says "the general matrix has no cell for it because its target has no training set, and here is what that changes about who can do what" has answered.
- Does an adversary need training-time access to run an evasion attack?No. Evasion acts on the already-trained model at inference and needs only the ability to submit an input under some budget on how far that input may move. Poisoning needs write access to data before or during training. They are separate stages precisely because the preconditions do not overlap, and an attacker holding one usually does not hold the other.
- Copying a checkpoint out of a registry is just file exfiltration — why call it an ML stage?The copy itself maps fine to an enterprise cell and should be reported there. What has no analogue is what the copy buys: the parameters put the attacker in a white-box access assumption against the live service — architecture and gradients assumed available — which is a different adversary from the one who could only send inputs. The file-copy cell says nothing about that change.
- Which of these ML stages needs the fewest privileges?The inference-side ones. Extraction, membership inference and evasion need only the right to send inputs and read outputs, which is often ordinary authorised use of a paid or internal endpoint; their cost is a query count rather than an access level. Poisoning is much more demanding: it needs a write into data that a future training run will consume.
A building's intrusion catalogue lists doors, windows, locks and keys. It has no entry for teaching the guard the wrong face, because the building it was written for never had a guard who learned.
saying these in an interview costs you the question
- Says an ML matrix is the enterprise matrix reworded
- Calls training-set poisoning ordinary file tampering
- Treats endpoint querying as generic API abuse
- Cannot name a stage's precondition, only its label
- Assumes a listed stage carries a cost or success rate