An unannounced red team met its objective and no alert ever fired — what does that report actually prove?

## Reading a silent success An unannounced red team reached its objective and, by the SOC's account, nothing ever fired. Before that becomes an organisational verdict, separate what the result supports from what people will read into it. ### What is proven That **one path** to that objective existed and was not detected — by that operator, using that tooling, during that window, against the rules, staffing and workload actually in force. That is a real and uncomfortable finding, and the realistic conditions are exactly what you paid for by running unannounced: no observer effect, genuine triage load, real weekend cover. ### What is not proven - **Not a rate.** One sample cannot be converted into "we detect X% of intrusions". Nothing was measured about any path the operator did not take, and the same team may catch commodity intrusions every week. - **Not coverage.** Techniques not attempted are simply unmeasured, in either direction. - **Not a cause.** This is the expensive part. For each step, a detection has to clear four stages: 1. **Collection** — was the record produced and shipped at all? Was the host reporting? Was command-line auditing on? (A Sysmon Event ID 1 process-create record carries the command line natively; the native Windows 4688 carries it only if audit policy was configured to include it.) 2. **Rule** — did any detection actually look at that record and match it? 3. **Triage** — did an alert reach a human, with enough context, and was it dispositioned correctly? 4. **Response** — did the action taken land in time to matter? A silent success collapses all four into a single unusable result. That is the structural cost of the unannounced design, and it is why the engagement type is a procurement decision, not a taste. ### Recovering attribution Contract for the operator's **full timeline as a deliverable** before the engagement starts: timestamps, hosts, accounts, commands, technique identifiers. Then walk it backwards against your own data, asking three questions per step — is there a record of this at all; would any deployed rule have matched that record; if a rule did fire, what happened to the alert. The most common answer is that the records were there and no rule looked at them, which is cheap to fix. The second most common is the one you must go looking for: **an alert did fire and was closed**. "Nothing fired" and "an alert was raised and dispositioned as benign" share a symptom and share no remedy — the first is a detection-engineering gap, the second is a content, runbook or queue-depth problem. Check the alert store yourself rather than accepting the shift's recollection. Then convert the exercise: replay the same chain openly as a purple exercise, operator and defenders in the same room, so each step gets an attributed outcome. That turns one un-attributable failure into a list of gaps with owners. ### Proving the fixes After the fixes, **execute the behaviour again**. A detection emits nothing when the estate is quiet and nothing when it is broken, so its silence never certifies it, and its false-negative rate is not computable from production data. Replaying an old captured event proves the rule still parses that event; only running the adversary behaviour again proves the record arrives, the rule matches, and a human sees the alert. ### Framing it upward Two bad readings are waiting. Leadership may conclude the SOC is worthless; the SOC may conclude the exercise was rigged. Head both off by stating the sample size, naming the four stages, and arriving with the concrete next purchase — the attributed replay and its cost — rather than with a reorganisation.