A re-executed attack technique that alerted last quarter now produces nothing. What do you do?
answer
- it worked before, so something changed
- confirm the execution before the diagnosis
- scope by shared dependency, not by rule
- the window between change and discovery
- close it with an execution, not a config check
basics
~20 sConfirm the technique really executed, then treat it as a regression rather than a new gap: find what changed, scope every other detection sharing that dependency, and account for the blind window between the change and this discovery. It is a change-management finding, not a coverage backlog item.
solid answer
~50 sFirst establish the failure is real — that the action executed and succeeded, and that the record is missing at the source rather than the alert being merely misrouted. Then use the fact that this passed before: something changed, and the fastest scoping is by shared dependency, not by rule. One forwarder or appliance change can mute every detection reading that source, so enumerate them and re-execute a sample. Next, size the blind window: between the change and today you had no coverage for this behaviour, so decide whether to hunt over that period using a different data source and whether any coverage claim you made about it must be qualified. Finally, report it differently from a new gap. A new gap is coverage work for detection engineering; a regression is a finding against the change path that let an input disappear without re-validation, and the fix is only proven by re-executing again.
go deeper
Know the difference between a detection that never existed and one that used to work, and that the second means something in the estate changed.
Explain how to locate where the chain stopped — source, transport or post-ingest — and why the other detections on that same source must be checked too.
Demonstrate sizing the blind window, deciding whether to hunt over it, and refusing to close the finding on anything short of a successful re-execution.
Own the uncomfortable half: correcting a coverage statement already given to leadership, an auditor or a customer, and using the dated example to argue that validated coverage decays.
## Why this is not the same finding as a miss An emulation that was never detected tells you about **coverage**: nobody built this, or it was built wrong. An emulation that *was* detected and now is not tells you about **change**: something in the chain moved. The two go to different owners, imply different remediation, and — the part candidates usually miss — differ in what they say about the past. ## Step one: prove the failure before you report it An absent alert is ambiguous. Confirm the operator actually executed the technique and that it succeeded on the target — the session in the management console, the local audit trail on the appliance, the operator's own notes. Then check whether the record exists at the source, and if it does, whether it arrived at the platform. Establishing *where* the chain stops is the whole diagnosis: at the source (audit category, configuration reset), in transport (forwarder, certificate, firewall, collector), or after ingest. A preventive control that now blocks the action entirely is also a possible explanation, and that outcome is a pass, not a regression. ## Step two: scope by dependency, not by rule The strongest lead you have is that this worked before. Whatever changed is likely shared. If the appliance stopped emitting a category, every detection over that category is dead; if a collector moved, everything through it is dead. Enumerate the detections that name the same source, category, field or collector, and re-execute a sample of them rather than assuming this was the only casualty. Scoping by ATT&CK tactic or by rule author is the common wrong instinct — those groupings have nothing to do with the plumbing that broke. ## Step three: size and act on the blind window This is what separates a strong answer. Between the change and today, this behaviour was not detectable. That period is not a hypothetical: if an adversary had done this then, you would not know. So establish the change date, state the window, and decide two things. 1. **Do you hunt it?** Pick a source that survived the change and look for the behaviour retrospectively, within whatever retention covers the window. If no surviving source can answer it, say so plainly — that is a real limit, not a formality. 2. **What must be qualified?** Any statement you made about coverage during that window — to leadership, to an auditor, to a customer questionnaire, in a previous validation report — was made on the strength of a test result that has since been shown not to hold. Correcting it is uncomfortable and non-optional. A new gap carries no such retraction, because you never claimed the coverage. ## Step four: report it to the owner who can prevent the next one A coverage gap is queued as detection engineering work. A regression is a finding about the change path: an input to a security control was altered without anyone re-validating what depended on it. The remediation has two halves — restore the telemetry, which the source or platform owner does, and close the process hole, which usually means the dependency being named on the change ticket so the next upgrade fires the test. Reporting a regression as *the detection team has a gap* points the fix at the only party that could not have prevented it. ## Step five: prove the fix by execution, not by inspection When the configuration looks restored, re-execute the technique. A configuration that reads correctly is the same evidence you had before this failure, and it was wrong then. The only acceptable close is an alert produced by a real action after the fix. ## Tone in the report A regression is normal in a live estate and should not be written as a scandal or as a failure of the platform owner, who was doing their job. What it demonstrates is that validated coverage decays silently, and that the decay is invisible until somebody executes the behaviour. That argument, made with a concrete example and a dated blind window, is far more persuasive than any general appeal for more validation capacity.
- How does the report differ from one describing a technique that was never detected?A never-detected technique is a coverage item owned by detection engineering, with no statement about the past to correct. A regression is a change-management finding: it names the change, the blind window it created, every detection sharing the broken dependency, and any prior coverage claim that must now be qualified. The remediation includes a process fix, not only a rule or a configuration.
- Retention only covers half the blind window. What do you say?Say exactly that: the window runs from the change date to today, retrospective analysis is possible for the portion still in retention, and the earlier portion cannot be answered from this source at all. Then look for a surviving independent source that covers it. An honest unanswerable interval is a usable input for a risk decision; a silent one is not.
- The platform owner says the configuration was restored, so the finding can be closed. Do you agree?Not on that evidence. A configuration that reads correctly is the same class of evidence that was already misleading before the failure was found. Close it only after re-executing the technique and observing the alert, and record the execution date as the point from which coverage can be claimed again.
saying these in an interview costs you the question
- Reports a regression as a new coverage gap
- Scopes by tactic or rule author instead of shared dependency
- Ignores the blind window between change and discovery
- Closes the finding on a restored configuration without re-executing
- Declares the detection broken before confirming the technique ran