How do you answer Shostack's fourth question — did we do a good enough job?

## What the question is actually asking Shostack's fourth question, *did we do a good enough job?*, is the only one of the four that points at the team rather than at the system. It does not ask whether the system is secure - nobody can answer that - it asks whether this pass through the loop was good enough to be relied on. That distinction is the whole question, and candidates who answer *yes, we found lots of threats* have missed it. ## The three things it checks **Did question one's model match reality?** A threat list is only as good as the picture it was derived from. If the diagram omits an admin console, a batch job or a third-party integration, then every threat against that element is missing and nobody knows it. So the first check is against the system as built, not against the design document. **Did question two reach the whole model?** Enumeration tends to cluster on the interesting parts and skip the boring ones - the internal queue, the backup path, the operator's console. A useful test is retrospective: when something is found later in testing or in production, ask which element or flow it came from and whether that element was ever examined. A finding from a place nobody looked at is a coverage failure, not bad luck. **Did question three's decisions land?** This is the highest-value part and the one that gets skipped. Every decision from question three is a promise about the future: a control will be built, a risk is accepted, a requirement will be tested. Question four asks what evidence exists that the promise was kept. For a mitigation, evidence means something you can point at in the running system - a configuration, a check that fails if the control regresses, a demonstration. For an accepted risk, evidence means the acceptance is written down, attributed, and still true under today's conditions. ## Why it gets skipped The first three questions all happen inside one session, with everyone in the room and momentum behind them. The fourth falls due weeks later, when the model has been filed, the session participants have moved on, and nothing in the delivery process re-opens it. It is also the only question whose honest answer can create work: saying *no* means going back around the loop. Teams that ask it reliably are teams where something - a release checkpoint, a change to the architecture, the delivery date of the mitigations themselves - forces the model back open. ## A worked shape A hospital operating-theatre scheduling service is re-run through the four questions four weeks after go-live. Questions one to three had been answered before launch and looked healthy. Question four is what catches the real problem: two threats marked *mitigated* in the pre-launch model turn out never to have been verified. The threat concerned an agency locum working on a shared clinical login - an authenticated, low-privilege position rather than an anonymous outsider - who can reach the schedule-rewrite path. The intended controls were per-session scoping of that shared login and a limit on bulk schedule rewrites; only one of them made it into the deployed configuration. The assets at stake are the availability of the theatre list and the integrity of the schedule that clinical staff act on, so an unverified control here is a patient-safety issue, not a data-breach issue. Nothing about that finding required new enumeration. It came from asking, for one existing decision, *what evidence is there that this shipped?* ## What a good answer looks like in the room Be specific about evidence. Say what you would accept as proof for a named control, and say what you would not: a ticket marked done, a mitigation written in the design document, or a person's recollection are all claims. Say when you would ask the question - at the point the decisions were due, not in the session that produced them. And say what a *no* obliges you to do: re-enter the loop at whichever question failed, rather than recording the gap and moving on. ## The boundary of the question Question four is about this model and this pass. It is not a judgment on the system's overall security posture, it is not a substitute for testing the system, and it does not ask whether every threat was eliminated - accepting a risk deliberately and recording it is a perfectly good answer to question three, and question four only checks that the acceptance is real and still current.