Your ML red-team scope excluded the training corpus — how do you report the poisoning stages?

## The defect this question is about An engagement produces a table of stages, and each row ends up with a mark. The failure is that two very different states collapse into the same-looking blank: - **Exercised, no finding.** The team had the access, spent the time, and the attack did not work. - **Not exercised.** The asset was outside the rules of engagement, or the days ran out, so nobody looked. A reader — an owner, an auditor, a committee — reads the second as the first. That is how an engagement whose scope excluded the training corpus becomes, three slides later, "we tested for poisoning and found nothing". ## How to write it instead Give the row an explicit state and a reason. Three states are enough: *exercised — finding*, *exercised — no finding*, *not exercised — reason*. The reason matters as much as the state, because the reasons are actionable in different ways: "corpus writes not authorised by the client" is a scoping decision to revisit next engagement; "labelling queue owned by a third party" is a supply-chain question; "ran out of days" is a budget conversation. Alongside the table, state the authorised asset list plainly: which ML assets could be touched at all — the registry, the endpoint, the corpus, the index — and which could not. That list, not the number of stages, is what bounds the engagement. ## Why the exercised stages carry no evidence about the excluded ones This is the technical half of the answer and it is where a senior candidate separates from a thorough one. The stage families have different preconditions: - Inference-side stages — probing the endpoint, characterising behaviour, crafting inputs it reads wrongly — need only the ability to send inputs and read outputs. - Training-time stages — poisoning, whether to degrade the model or to install a conditional keyed to something the attacker controls — need a write into data that a future training run will consume, plus a retraining cadence that carries it into production. An engagement that had endpoint credentials and no corpus write tested the first family only. Nothing it learned constrains the second, because the access it never had is exactly the access that family requires. Saying this in one sentence in the report is what stops the blanks being read as evidence. ## Coverage is a statement about authorisation, not about security If someone asks for a coverage percentage across the ML stages, answer that it measures two things — which assets the rules of engagement authorised and how many days were funded — and neither of them is a property of the model. Two engagements against the same service with different scopes produce different coverage and identical exposure. A real adversary is limited by what they can reach and what they are willing to spend, and those limits do not resemble yours: they are not required to stop at the labelling queue. It is worth adding the limit that sits in the published matrices themselves. A stage entry asserts that an adversary *can* act that way at that point. It does not carry what the action costs them, how many written rows or how many queries it takes, or how often it works against a particular deployment. So neither a filled row nor a blank one is a measure of likelihood, and a report should not let it be read as one. ## What to recommend when a stage was out of scope Do not leave it as an absence. Convert it into one of three concrete asks: bring the asset into scope next time and say what would be exercised; or, if it will never be in scope, name the control the client should verify themselves — who can write to the corpus, whether writes are attributable, what the retraining cadence is, whether anything reviews what a training run consumed; or record it as accepted and unassessed, with a named owner. All three are honest. A blank is not. ## The sentence that survives the meeting "We exercised the stages our access allowed, found what is listed, and did not test the training-time stages at all because corpus writes were out of scope — so this report says nothing about whether the model can be poisoned." It is unflattering, it is short, and it is the only version that will not be misquoted.