Median time-to-close falls each week as your alert backlog grows - improvement or clearing?

## What the metric is and why it drifts Median time-to-close is read off the case management system's own audit trail - the closure timestamps, who closed each case, the disposition and the note. It is popular because it is free and it moves. It is dangerous for the same reason: it is a **throughput** measure being used as a proxy for **quality**, and the fastest way to improve it is to stop investigating. The pattern that should stop you is the correlation. If close time fell in the same weeks the backlog grew, the two are almost certainly the same event: the queue got deeper, pressure to clear it rose, and the time spent per case fell. Cases do not become easier because there are more of them. ## The benign explanations, checked first Before concluding anything, rule out three real ones: 1. **A tooling change.** An enrichment step now resolves the entity, attaches the account's normal behaviour, or pulls the change record - so an analyst genuinely needs less time. This is verifiable: it has a deployment date, and the improvement should appear as a step at that date, not a slow slide. 2. **A playbook or automation change.** A verified condition is now auto-closed, or a playbook was rewritten to end earlier for a specific rule. 3. **A mix shift.** This is the subtle one. A newly deployed noisy rule can add thousands of genuinely trivial cases and pull the overall median down while every individual rule's close time is flat or rising. The aggregate moved without any analyst changing behaviour. Mix shift is why the first analytical move is always **compare like with like**: seconds-to-close within a single rule and severity band, week over week. If the fall survives that, it is about how cases are being worked. ## Reading the audit trail The case system records more than duration, and the extra fields are what turn a suspicion into a sample: - **Note length.** An eleven-character note is not, on its own, proof of anything - some closures are legitimately one line when enrichment already carries the evidence. It is a *sampling signal*: it tells you which cases to re-review, never what the verdict should have been. - **Bulk closure patterns.** A run of cases closed by one account within seconds of each other, all on one rule, is a multi-select clear rather than a sequence of judgments. - **Queue depth at the moment of close.** If seconds-to-close falls as depth rises, within the same rule, you have the correlation stated plainly. - **Disposition mix.** If the share of cases closed without escalation rises at the same time, the pressure is being paid for in verdicts. ## The decisive test None of the above proves a wrong verdict; they only locate where to look. The test is a **blind re-review**: draw a sample over-weighted toward the fastest closures, strip the disposition and notes, have a different analyst work them from the raw evidence, and compare. - If the re-review agrees, the queue really did get easier and you now have evidence to say so. - If it does not, you have something far more useful than an opinion: a measured disagreement rate on exactly the cases the speed metric was celebrating, plus concrete examples of what was missed. ## What you do with the answer If the verdicts held up, publish the finding and the mechanism - the enrichment, the playbook, the mix - so nobody claims credit for an improvement that was a chatty rule. If they did not, the fix is rarely "work slower". It is upstream: the rule producing the flood, the missing context that made each case expensive, the auto-close that should exist for a verified condition, and the expectation that the queue must be emptied every shift regardless of what is in it. Reporting close time without a paired quality measure creates the pressure by itself; the corrective is to publish the two together, so speed cannot be improved by degrading verdicts without the second number moving. ## The security-specific stake In a reliability context a slow queue is an annoyance. Here, the cases being cleared fastest are the ones an intruder is hiding in - and the whole point of the behaviour they are using is that at a glance it looks like administration. A metric that rewards the glance is a metric an adversary benefits from.