How do you run an escaped-issue review when an incident hits a system you threat-modeled?

An **escaped issue** is a design-level security problem that reaches production on a system the practice had already modeled - found by an incident, a penetration test, a bug report or a later review. The escaped-issue review is the practice's own retrospective: not *who broke it*, but *which step of our method would have had to work differently for this to have been caught*. Take the concrete case. An outage was traced to an internal administrative endpoint that required no authentication; the endpoint could be reached from a neighbouring service that an attacker had a foothold on, and the actions it exposed both took the service down and left the audit trail unable to attribute what happened. The service had a threat model. The element was drawn on the diagram. No threat had ever been enumerated against it. ### Walk the chain, and stop at the first no The review is a fixed sequence of questions, each of which localises the failure to a different part of the practice: 1. **Was this component in scope?** If the system or change never triggered a model at all, the failure is in coverage and triggers, not in the method. Fix the rule that decides when modeling happens. 2. **Was it on the diagram?** If the element or the flow was missing, the model was an incomplete picture. Fix how the diagram gets built and reviewed - who confirms it matches the deployment. 3. **Were threats enumerated against it?** This is where the example fails. The element was drawn and then skipped, because it was *internal* and the analysis quietly treated the network position as trust. The remedy is enumeration discipline: every element and every flow that crosses a boundary gets its categories walked, and *internal* is a claim about the network, not a granted identity. A trust boundary that is assumed rather than drawn is the single most common shape of this branch. 4. **Was the threat enumerated but rated too low?** Then the failure is in rating: the consequence, the reachability or the attacker position was misjudged. Fix calibration - typically by re-rating the same threat with the review group and asking what input was wrong. 5. **Was it rated correctly and accepted?** Then the practice worked exactly as designed and produced a decision that turned out badly. That is a question about who is allowed to accept which risks and on what evidence, not a defect in the modeling. 6. **Was a control chosen but never shipped, or shipped without a test?** Then the failure is in follow-through from finding to change, and it belongs to how threats become tracked work. 7. **Was the model simply older than the system?** Then it is a drift failure, and the fix is in re-review cadence rather than in enumeration. Stopping at the first *no* matters. Each branch has a different remedy, and a review that concludes *we should threat model better* has not localised anything. ### The product is a change to the practice A good escaped-issue review produces one or two specific, checkable changes. For the example above, that is a rule that every administrative or operational interface is enumerated as an entry point regardless of network position, plus a check in the review step that no element on the diagram was skipped during enumeration. It may also produce a change to who reviews models, or to the trigger that requires re-modeling. It should not produce: a blame assignment, a quietly rewritten model that now shows the threat as though it had always been there, or an abandonment of the method on the strength of one escape. Rewriting the model without recording that the threat was missed destroys the only evidence the review runs on. ### Framing and cadence Run it blamelessly and on the practice, not the team - the same discipline as an operational post-incident review, and often as an appendix to one. A sensible trigger is every incident with a design-level security cause and every penetration-test finding that a model should plausibly have predicted. Keep the record: over a year, the branch that keeps coming up is the strongest evidence you have about where the method is weak. If three escapes in a row stop at step 3, the enumeration step is where to invest, not the diagramming step. ### Feeding the metric Escapes are the only lagging outcome measure a threat modeling practice has, so this review is also how the number gets its meaning. Count escapes as a rate over modeled systems, but read them as cases. And treat a period with zero escapes with suspicion: escapes are only counted when someone traces an incident back to the design, so a zero more often means the tracing habit is missing than that nothing escaped. Building the review into incident handling is what turns the metric from a decoration into a feedback loop.