A sampled QA review of a training-data queue rejected zero items - what does that bound?

## Read the result as two separate claims, both weak "Zero rejections" is a conjunction, and each half bounds much less than it sounds like. **Half one: what was drawn.** A review sees a share of intake. If the share is a few percent and a targeted attack needs a few dozen items, the expected number of poisoned items appearing in front of any reviewer is below one. A zero-rejection quarter is exactly what you would observe if nothing was wrong, and also exactly what you would observe if a sized attack had landed. An outcome that is equally likely under both hypotheses distinguishes nothing. **Half two: what was asked.** A reviewer receives one item and answers a per-item question - is this labelled correctly, is it plausible, does it match the guideline? A clean pass therefore bounds *per-item implausibility among drawn items*. It is a statement about individual items, and it is the only statement the process is structured to produce. ## The unit of judgment is the deeper limit The coverage half can, in principle, be bought with money. The second half cannot, because it is about what question is being answered. A targeted poisoning effect is a property of a **set**. It exists because a hundred or a few hundred items jointly reinforce a narrow association that would not otherwise be there. No single member of that set carries the effect. Handed any one of them, a reviewer applying the guideline gives the correct answer for that item - which is *pass* - and does so honestly and competently. The pattern lives in the relationship between items: their common source, their arrival window, their shared unusual feature combination, the way they cluster in a region of the input space that was previously sparse. So per-item verdicts never accumulate into a set-level statement, however many you collect. Ten thousand clean per-item verdicts are ten thousand answers to a question that was not about the attack. ## Reading the coverage report Given a QA line like the one above, the useful reading is: | what the row says | what it bounds | | --- | --- | | items reviewed: flat | the QA function is staffed and stable | | share reviewed: falling | the chance any given write is ever seen, falling too | | items rejected: zero | drawn items looked fine one at a time | | review unit: one item | no part of the process compares items to each other | The row that changed year on year is the share, and it changed for a reason with no defect in it: intake grew and hours did not. Nobody did anything wrong, and the property got worse. ## What a triager should do with this Stop treating the rejection count as evidence and treat it as a description of the process. Then ask the questions the report does not answer. What share of intake was reviewed, and which share is it - all sources, or the ones easiest to route? What was the reviewer asked to decide, and has anything in the pipeline ever looked at items *together* by source or arrival window? Which sources have write access at all, and did any of them change this year? None of those is a better sampling scheme; they are questions about what the existing number means. The one thing not to do is convert a zero into an assurance sentence. ## The direction of the claim A clean review bounds the per-item anomaly that whoever wrote the data was willing to leave visible - that is, how carefully they wrote - not whether they were there. Getting that direction backwards is the whole failure mode: it turns a description of your own review procedure into a claim about an adversary who read that procedure first.