One annual red team or continuous automated validation: which do you buy for a one-analyst SOC with an MSSP?

## Buying measurement versus buying realism A one-analyst SOC with monitoring co-managed by an external provider has one meaningful budget line for adversary emulation this year. The two candidates answer different questions, and the sequencing matters more than the choice. ### What continuous validation buys Continuous (automated) validation executes a catalogue of discrete, benign-but-representative behaviours on real production hosts on a schedule, and records for each one whether a record was produced, whether a rule matched, and whether an alert reached a human. Strengths: - **Attribution.** It tells you *which stage failed* for a known behaviour, which is precisely what a silent red team success cannot. - **Repeatability and regression detection.** It is the only instrument that catches the quiet decay that actually kills detection: a log source that stopped shipping after a rebuild, an endpoint agent left in audit-only after a rollout, a rule retired during a cleanup, a parser change that broke a field. - **Cost per run**, which makes weekly cadence affordable. Limits, and they are structural: - It only runs behaviours **somebody already thought of**. It measures the detections you imagined, and it runs one known implementation, so a rule matching that implementation may still miss a variant. - It never tests people. It can show an alert was generated; it cannot show that the provider escalated it, that the escalation carried enough context, or that your analyst acted at 03:00. - If you grade the function on the catalogue, you get rules tuned to the catalogue. ### What the annual red team buys An objective-based engagement is the only instrument that exercises the whole chain including the humans and the provider handoff: whether the alert left the provider's queue, how quickly it reached you, whether one analyst on a Friday could act on it, and whether an operator finds a path nobody modelled — a misconfiguration chain, an unexpected trust, a forgotten estate. Its weaknesses for this organisation are equally structural: one sample, high price, and — against an unmeasured baseline — a near-certain quick success that teaches you only that you are blind. ### The judgment Sequence rather than choose once. While the baseline is unknown, spend on repeatable validation of a defined behaviour set until the basics reliably produce alerts that a human sees. Then buy the engagement, and scope it deliberately at what automation cannot reach. Paying red-team rates to rediscover that an endpoint log source stopped shipping is the waste to avoid; so is running validation forever and never testing whether anybody acts. If an annual engagement is contractually or audit-mandated, do not fight it — narrow it. Point it at the escalation path, the out-of-hours decision, and an objective nobody has modelled, and fund the repeatable measurement from a separate line. ### The provider is part of the purchase With co-managed monitoring you are also buying a test of the provider, and the engagement type decides whether they are told. - Scheduled validation must be **visible to the provider as a named programme**, with hosts and windows agreed. Otherwise you burn their analysts on activity you generated, and after the third week they start closing the real thing. You can still measure them honestly: grade on whether the alert existed and whether it reached you, not on whether they were fooled. - An unannounced engagement's value depends on their on-call **not** knowing, which makes it a conversation with the provider's management and a check of what your contract permits — including whether they will treat detection of the exercise as billable incident work. ### Capacity Finally, price the aftermath in analyst time. A validation programme that produces a hundred findings for one person to work is not cheap, whatever the licence cost. Whichever instrument you buy, name the owner of the fixes before the first run.