A finding from your AI red-team engagement matches no technique in the published adversary reference you map against. What do you do with it, and how do you write that entry?
answer
- wrong map is worse than no map
- silently confident redirect
- attacker position, access, effect
- name the near-miss
- explicit marker, not a blank cell
basics
~20 sMark it unmapped and say so explicitly. Describe the behaviour in your own words, name the nearest technique and state exactly why it does not fit. Forcing a fit misdescribes the finding, sends the reader to the wrong mitigations, and hides a gap the reference itself may need to cover.
solid answer
~50 sUnmapped is a legitimate outcome, and writing it down is more useful than a stretched fit. The mechanism matters: a reader who sees an identifier will follow it, read the reference's description, and act on the countermeasures listed there. If your finding only half-matches, they act on the wrong control and conclude the issue is handled. A wrong map is worse than no map, because it is silently confident. So the entry carries: the behaviour described concretely from your own evidence; the nearest candidate technique named; and one or two sentences on where the fit breaks — usually the attacker's position, the required access, or the effect differs from what the reference describes. Then an explicit **unmapped** marker rather than a blank cell, so nobody assumes the mapping was simply forgotten. The pressure to force a fit is presentational — blank rows look incomplete. Handle that in the methodology section: state that mapping is best-effort and unmapped entries are deliberate.
go deeper
Knows that not every finding maps and that leaving it unmapped is allowed rather than a failure.
Explains why a stretched fit misleads — the reader follows the identifier to the wrong description and wrong mitigations — and writes the near-miss down.
Gives a decision rule (attacker position, required access, resulting effect) and treats title-matching as the root cause of bad mappings on review.
Reads clusters of unmapped findings across engagements as signal about either the reference's coverage or the team's mapping discipline, and sets the house rule accordingly.
### Why a wrong map costs more than no map A published technique identifier is read as a **redirect**. The reader does not stop at your entry; they follow the code into the catalogue, read a description written for some other attacker position or some other effect, pick up the mitigations attached to it, and file your finding as understood. The gap between what you observed and what they read never surfaces, because the identifier concealed it. Nothing downstream re-checks a mapping against the underlying evidence — not the ticketing system, not the detection team, not the auditor next year. That asymmetry is the whole argument: an unmapped finding costs a reader one extra minute of reading your narrative, while a wrongly mapped finding can cost a remediation cycle spent hardening a control that was never on the path. ### The decision rule Map when your evidence matches the catalogue entry's description on **three axes**: | Axis | The question | |---|---| | Attacker position | Where did the attacker have to be — outside the product, inside the tenant, holding model weights, able to place a document in a corpus the system fetches? | | Required access | What did the technique presuppose the attacker already had, and did your attacker have it? | | Resulting effect | What did the technique actually produce, and is that what you observed produce? | If all three hold, map it. If any one materially differs, it is a **near-miss**, and a near-miss is written down as unmapped with the near-miss named. This is a rule about the entry's *description*; matching on the entry's *title* is the single largest source of bad mappings, because different techniques routinely share vocabulary. ### The shape of the entry - **Behaviour** — what the attacker input was, which surface accepted it, and what came out, written so the paragraph stands with no identifier at all. - **Nearest technique** — named explicitly, so the reader knows you looked, and where. - **Why it does not fit** — the specific diverging axis, in a sentence: *"the entry describes this as presupposing model-weight access; our path needed only an untrusted document in the retrieval corpus."* - **Marker** — an explicit *unmapped*, visually distinct from a cell that nobody filled in. A blank reads as an oversight and invites someone to "fix" it later with a guess. ### What it costs, and the number that misleads Ruling something unmapped costs almost nothing in analyst time — one sentence — and costs something real in the client conversation, because blank-looking rows read as incomplete work. Handle that structurally: a line in the methodology section stating that mapping is best-effort and that unmapped entries are deliberate, agreed before delivery rather than defended at the readout. The misleading number here is **mapping coverage**: the share of findings that carry an identifier, quoted as though it measured report quality. It measures nothing of the kind. It is trivially driven to one hundred per cent by loosening the fit criterion, and every point of it earned that way makes the report actively worse — each forced code is a reader pointed at the wrong description. A coverage figure that goes *up* while the catalogue and the testing scope hold constant is more likely evidence of sloppier mapping than of better work. If anyone tracks the ratio, track it alongside a sample audit of whether the mapped ones actually satisfy their entries. ### Why findings genuinely fail to map Worth recognising, because each has a different implication: - **Application-layer logic abuse the model merely carried** — the defect is in the surrounding system; the catalogue's AI techniques do not describe it. - **A policy or safety failure that is not an attack technique at all** — the model did something the client considers unacceptable without an adversary doing anything clever. - **Novelty in the composition** — every individual step maps, but the interesting part is the sequence, and the catalogue may have no concept of a chain. - **Lag** — a delivery pattern the reference has simply not absorbed yet. Catalogues are revised on a cadence; findings are not. ### What I would check on review For each *mapped* finding: does its evidence actually satisfy the entry's description, or was the code chosen from the title? Sample a few and re-derive them cold. For each *unmapped* finding: is the nearest technique named and the divergence stated, or is "unmapped" being used as a place not to think? And across engagements, watch the cluster: a recurring unmapped behaviour is signal — either you are testing a surface the reference does not model yet, which is worth writing up as a named house pattern and possibly contributing upstream, or your team's fit criterion has drifted too strict.
- A stakeholder pushes back that unmapped findings 'look unprofessional'. What is your response?That the mapping is a navigation aid, not a completeness measure, and a forced fit points their defenders at controls that do not address what we found. I'd state the best-effort policy in the methodology section so unmapped reads as deliberate.
- How do you tell a genuine near-miss from laziness in choosing the technique?Read the reference's full description, not the title, and check three axes against your evidence: where the attacker sat, what access was needed, what effect resulted. If all three match it maps; if one materially differs it is a near-miss.
- You see the same unmapped behaviour across three engagements. What now?Write it up as a house pattern with a stable internal name so it is at least consistent across your reports, and consider contributing it upstream to the reference if it has a submission path.
saying these in an interview costs you the question
- Insisting every finding must be mapped so the report looks complete.
- Choosing an identifier by matching the technique's title rather than its description.
- Leaving the mapping cell blank with no explanation, so it reads as an oversight.
- Inventing an identifier in the reference's format for an unmapped item.