skip to content

Three Kinds of Coverage

Telemetry coverage, detection coverage and validated coverage are three different claims, and only the last has been tested against something that actually ran. Interviewers press on which you mean.

on this pageshow

explore

questions

4

A technique cell on your ATT&CK coverage heatmap is green - what three different claims could that colour be making?

level: juniorimportance: must knowfreq 62%

answer

  1. one colour, three different promises
  2. collected, written, proven
  3. each level is weaker than it looks
  4. the author's reading versus the reader's
  5. a rule existing is not a rule firing

basics

~20 s

Green usually stands for one of three very different facts: the log source that would show the behaviour is collected, a detection rule exists over that data, or the behaviour was actually executed and the rule caught it.

solid answer

~50 s

One colour is hiding three claims of very different strength. The weakest is **telemetry coverage**: the log source that would record the behaviour is collected from the hosts in scope, so the evidence is searchable - but nobody is notified. Next is **detection coverage**: a rule exists over that data and is routed to a queue, which proves a query was written, not that it fires on real tradecraft. Strongest is **validated coverage**: the behaviour was executed in this estate and the rule actually fired and was worked. Each level needs the one below it and none implies the one above. The danger is asymmetric reading - the SOC colours a cell at level one and the executive reads level three - so record the assertion level, log source, rule id and validation date per cell, and render the colour from those fields.

go deeper

for a junior

Be ready to name the three claims in order - log source collected, rule written, execution actually detected - and to say which is weakest. Interviewers mostly want to hear that you do not treat a green square as protection.

for a middle

Explain why each level only implies the ones below it, and give a concrete way a rule can exist and still never fire: a field the estate does not populate, or a command-line pattern the behaviour does not produce.

for a senior

Show that you route by level rather than by colour: a telemetry gap goes to a platform owner with a cost, a rule gap to detection engineering, a validation gap to whoever executes techniques. Demonstrate the per-cell fields you would insist on before signing a cell.

for a principal

Own the fact that the same artefact is a work-tracker inside the SOC and an assurance claim outside it. Be ready to say what you would change in the rendering, and what you would refuse to publish, before anyone senior repeats it.

## The colour is a rendering of a claim nobody wrote down A coverage heatmap is a grid of adversary techniques with a colour per cell. The colour is the output of a judgement, and in most estates the judgement itself is never recorded next to it. Three quite different underlying facts routinely produce the same green square, and telling them apart is the whole skill this artefact tests. ### 1. Telemetry coverage - the weakest claim The log source that would carry evidence of the behaviour is being collected from the host population in scope. For an execution technique that might be process-creation records with the full command line: a Sysmon Event ID 1 record, or Windows Security 4688 where the audit policy was configured to include the command line, or an `execve` record from auditd on the Linux fleet. For an identity technique it might be the identity provider's sign-in log; for a cloud technique, the control-plane audit trail. What this asserts: *if this behaviour occurred, a record of it exists somewhere we can search.* What it does not assert: that anyone would be told. Its real value is retrospective - an investigator or a hunter can go back and look. That is worth a lot, and it is not detection. ### 2. Detection coverage - a rule exists A rule is written over that data, is enabled in production, and its output is routed to a queue somebody works. This asserts: *if the behaviour matches the shape this rule expects, something appears in front of an analyst.* Both hedges matter. A rule keyed to one command-line pattern misses the same behaviour expressed differently. A rule referencing a field that the estate does not populate on those hosts is a rule that can never fire. The existence of a rule is evidence that an engineer wrote a query; it is not evidence that the query matches what an adversary actually does. ### 3. Validated coverage - somebody executed it and it fired The behaviour was executed in this estate, deliberately and with authorisation, and the rule fired - and, in the strong form, an analyst worked the resulting alert and reached the right verdict. This is the only one of the three that is an *observation* rather than an *intention*. It is also perishable: it is true of the estate, the sensor configuration and the rule as they stood on the date of the test. ### The chain, and why each link is one-way Telemetry is necessary for a rule, and a rule is necessary for an alert, but the implications never run backwards. Collected data does not imply a rule. A rule does not imply a firing. A firing in a lab does not imply a firing on a production host with a different sensor configuration. Every step upward is an additional, separately earned claim. There is also a state below all three that heatmaps habitually mangle: the technique for which the estate emits no relevant telemetry at all. That is not a failed detection; it is an absence of the raw material one would be built from, and it needs a colour of its own. ### The asymmetry that makes this dangerous The author of the heatmap knows which claim they meant. The reader does not, and readers resolve ambiguity upward: green becomes "we would catch that." So the same artefact is simultaneously an honest work-tracker inside the SOC and an overstated assurance outside it. Nothing about the colour causes that; the missing per-cell basis does. ### The three levels also have different owners This is the practical reason to keep them apart rather than a purely semantic one. A telemetry gap is usually somebody else's work - a platform or endpoint team enabling a channel, deploying a sensor, paying for the ingest. A rule gap belongs to detection engineering and is typically days of work. A validation gap belongs to whoever runs authorised technique execution. Collapsing all three into one red or one green destroys the routing information the artefact exists to carry. ### What a defensible cell holds A cell worth signing carries: the log source or sources the claim depends on, and whether they are collected across the host population in scope rather than on a sample; the identifiers of the rules asserting the behaviour; the date and basis of the last validation, if any; and the platform scope. The colour is then computed from those fields. If the fields are empty, the honest rendering is not green. ### Directions of claim to keep straight A collected log source proves the data would exist, not that anyone is watching. A rule proves a query exists, not that it matches adversary behaviour. An alert proves a rule fired, not that something malicious happened. And the absence of alerts proves nothing whatsoever about the estate - it is equally consistent with a quiet quarter, a broken feed and a rule that has never matched anything in its life.

  • What does a cell that is green only because the log source is collected actually buy you?
    Retrospective reach. An investigator or hunter can go back and search for the behaviour, and evidence will be there if it happened. Nobody is notified in the moment, so mean time to detection is unbounded for that technique. It is genuinely valuable - it is what makes an investigation possible at all - but presenting it as detection promises a notification that will never arrive.
  • Why is 'a rule exists for it' a weaker claim than most people hear?
    Because a rule encodes an assumption about the shape of the behaviour and about its inputs. It may key on a command-line pattern the adversary does not use, or reference a field the estate does not populate on those hosts, or have been broken by a schema change. Its existence proves an engineer wrote a query. Only executing the behaviour shows whether the query matches it.
  • Where should the three levels be recorded, if not in the colour?
    In per-cell metadata behind the grid: the required log sources and whether they are collected across the host population in scope, the asserting rule ids, the date and basis of the last validation, and the platform scope. The colour becomes a rendering of those fields. That way a challenged cell can be defended without anyone's memory, and a stale cell is visible as stale.

Installed, wired to the panel, and actually tested with real smoke are three different claims about a smoke detector. Only the last one has ever been proven to make a noise.

saying these in an interview costs you the question

  • Treating a written rule as proof the behaviour would be caught
  • Reading green as 'we are protected' rather than 'we might see it'
  • Assuming collected telemetry means somebody gets alerted
  • Publishing one coverage percentage with no stated basis
  • Saying the technique was never seen, so the cell must be working

context

open as a page

What does a green T1218 cell overstate when only two of its signed-binary proxy sub-techniques were tested?

level: middleimportance: should knowfreq 48%

basics

~10 s

It overstates the unit of the claim. Sub-techniques use different binaries, command lines and parent processes, so catching rundll32 and regsvr32 says nothing about the ten siblings nobody executed.

open as a page

Your Kubernetes estate emits no telemetry for a technique your Windows fleet detects - how should that coverage cell read?

level: seniorimportance: should knowfreq 40%

basics

~20 s

As a distinct no-visibility state, not a failed detection. Red says a rule is missing and detection engineering can fix it; no telemetry says the raw material is absent, a platform team owns it, and it costs money.

open as a page

Before your ATT&CK coverage heatmap goes into a customer questionnaire under your signature, what do you change?

level: principalimportance: nice to knowfreq 30%

basics

~10 s

Everything the internal version leaves implicit: what each colour asserts and when it was last shown true, which platforms it covers, and any cell you cannot support, downgraded to a dated, owned blind spot.

open as a page