Your MTTD improved 40% quarter-over-quarter after the clock's start event was redefined — what do you present to the board?

## The problem is not the number, it is the claim A 40% quarter-over-quarter improvement in mean time to detect makes one implicit claim: *we detected intrusions sooner than we used to*. If the start event moved — say the previous definition anchored on the adversary's first malicious action and the new one anchors on the earliest surviving log record — then the claim is false even though the arithmetic is right. Presenting it unqualified is not a rounding error; it is telling a board something untrue about the organisation's exposure. ## Step one: build the bridge Recompute **both** quarters under **both** definitions. That gives four numbers and one honest decomposition: - Q1 old definition vs Q2 old definition — the real performance change - Q2 old definition vs Q2 new definition — the definitional change Present it as a bridge: 'of the 40% reduction, 31 points come from the change in the start event and 9 points from faster triage on commodity cases.' If the recomputation is impossible because the old definition needs evidence you no longer hold, that is itself the finding, and the honest statement is that the two quarters are not comparable. ## Step two: break the line A trend line drawn across a definition change is a lie with axes on it. Mark the break, label both segments with their definition version, and if you must show a continuous series, show the one recomputed under a single definition throughout. ## Step three: disclose it first You raise it, in that meeting, before the number is celebrated. The alternative is that a non-executive director asks in six months why the definition changed, and the answer arrives as a discovery rather than a disclosure. At that point every other security metric you present becomes suspect, which is a far larger loss than one quarter's flattering figure. The self-disclosure is also the cheapest credibility you will ever buy. ## Step four: govern the definition The deeper failure is that a clock definition changed without a decision. Treat the definition the way you would treat any other artefact people depend on: - **written and versioned** — start event, stop event, containment-executed criteria per surface, and the boundary shared between the detection and containment clocks - **owned by a named person**, who approves changes - **re-baselining required** — changing the definition obliges you to recompute published history or explicitly declare the break - **audited** — periodically pull a sample of closed cases and check the recorded start date against the evidence that anchored it, because the definition governs nothing if per-case practice drifts from it ## Step five: say the Goodhart part out loud Once MTTD is a board-level target, three cheap ways to improve it exist that involve no detection improvement at all: move the start event later, close ambiguous cases as non-compromises so they never enter the population, and let long investigations stay open past the reporting boundary. None of them require dishonesty by any individual; they are what a system optimises toward when a single number is the reward. The defence is to publish MTTD next to something that gets *worse* when you cheat. Good candidates on this leaf: - the share of closed cases whose start date was anchored on **direct evidence** rather than estimated — moving the clock start later usually means anchoring on weaker evidence, and this figure falls - the number of cases in the quarter still **open** past a threshold — parking long cases shows up here - for a co-managed estate, the gap between **containment recommended** and **containment executed**, which the provider's own contractual MTTC excludes entirely ## What the board actually needs Not a single number. A board needs to know whether exposure is shrinking, and the honest form of that is: the distribution of dwell across this quarter's cases, the sample size, what changed in how it is measured, and one sentence on the cases that were unusual. A SOC manager who presents a 40% improvement without the bridge has optimised for the meeting; one who presents the decomposition has made the metric usable for the next three years.