skip to content

With capacity to re-execute 20 of 200 passing attack techniques a quarter, how do you choose?

level: seniorimportance: should knowfreq 44%

answer

  1. not all misses cost the same
  2. one success with no second chance
  3. count the hands on the telemetry path
  4. what changed since it last passed
  5. publish the untested remainder

basics

~20 s

Rank by consequence of a single success, fragility of the telemetry path, and exposure to change since the last pass. Give a small permanent core the estate cannot survive missing, spend the rest on recently changed and single-source detections, and rotate the tail so nothing is untested indefinitely.

solid answer

~50 s

Three axes decide it. First, consequence: an authenticated action against the hypervisor management console or the backup server can remove every restore point and every workload at once, so those techniques hold a permanent slot regardless of how recently they passed — there is no second chance to detect them. Second, fragility of the path: a detection resting on an appliance's own audit log forwarded by syslog, with a vendor-controlled category set, is far more likely to break than one on mainstream endpoint telemetry. Third, change exposure: anything whose source, agent or collector was touched since it last passed. Then bias down anything covered independently by two unrelated sources, and rotate the remaining tail on a fixed horizon so no technique goes untested forever. Publish the untested remainder rather than implying the whole 200 are proven.

go deeper

for a junior

Understand that validation capacity is limited and that techniques are picked, not covered exhaustively; know that consequence and recent change drive the picks.

for a middle

Explain the selection axes concretely — one-shot consequence, fragility of the telemetry path, change since last pass — and why redundancy lowers priority.

for a senior

Show that you would defend a permanent core, set a rotation horizon for the tail, and publish the untested remainder instead of letting a pass rate imply full coverage.

for a principal

Be ready to argue that a validated portfolio larger than the programme can sustain is itself the finding, and to take that trade to whoever owns the budget.

## The budget is real, so the selection has to be explicit Two hundred previously-passing techniques and capacity for twenty re-executions a quarter means each technique gets re-proven roughly every two and a half years if you rotate blindly. Blind rotation is the wrong answer, because the techniques are not equally consequential and their telemetry is not equally fragile. ## Axis one: consequence of a single success Most techniques are one step in a chain, and missing one does not end the story — there is another chance downstream. A small set has no downstream. An authenticated administrative action against the virtualisation management console or the backup server is the archetype: from that position an intruder can destroy every restore point and every workload in one operation, and the recovery path you were relying on is the thing being destroyed. A detection you only get one chance to fire is worth re-proving even when nothing about it has changed. Give that set a **permanent slot** — it is spent every quarter by design, and it should be small enough that this is affordable, typically a handful. ## Axis two: fragility of the telemetry path Rank the remaining detections by how many hands touch their path in a year and how much of it you control: - An appliance's own application audit log, enabled by a vendor-controlled category set and forwarded by syslog from a box the platform team upgrades quarterly, is the fragile extreme. Every part of it can change without a signal to you. - Mainstream endpoint telemetry from an agent you version-control and monitor, feeding a schema you own, is the robust extreme. Fragility should dominate raw technique popularity. A famous technique on a robust path is a worse use of a slot than an obscure one whose entire evidence chain runs through a vendor's default configuration. ## Axis three: exposure to change since the last pass Anything whose source, agent, collector or ingest schema changed since it last passed goes near the front. In a healthy setup most of these are caught by change-triggered runs rather than the quarterly programme, and what reaches the quarterly list is the residue — the changes nobody flagged, and the components with no clear owner. ## What lowers a technique's priority - **Redundancy.** A behaviour covered independently by two unrelated sources is unlikely to go completely dark from one change; it can lose one leg and still alert. - **Cheapness of an alternative check.** Where a source is verified continuously by other means, spend the execution budget somewhere blinder. - **Low consequence combined with a stable path** — the long tail, which is what rotation is for. ## Rotation, so the tail is not abandoned After the permanent core and the change-driven and fragility-driven picks, spend what is left on the oldest untested technique first, and state a horizon: *no technique goes more than N quarters without a re-execution*. If the arithmetic makes N absurd, that is a finding in itself — it means the validated portfolio is larger than the programme can sustain, and either the budget or the portfolio has to change. ## Cost is a selection input, not just a constraint Some executions are a login and a screenshot; others need a change window on production infrastructure, two approvals and a rollback plan. Where the expensive ones matter, look for a cheaper action that traverses the identical source, audit category, field and collector — it does not prove the detection logic against real tradecraft, but it does prove the pipeline is alive, and it can be run far more often between full executions. ## Say what you did not test The selection is only honest if the untested remainder is visible. A programme that reports twenty passes and lets readers infer two hundred working detections has converted a budget constraint into a false assurance. State the core that is proven every quarter, the set proven this quarter, and the age of the oldest untested technique.

  • Why give a technique a permanent slot when it passed the last four quarters?
    Because past passes say nothing about the current state of a path that changes continuously, and because the consequence of this particular miss is unrecoverable. For an authenticated action against the backup server, detecting it is the only intervention point left — every later stage assumes restore points that may no longer exist. Consistency of past results is the weakest possible reason to stop testing a control you only get one chance to use.
  • Two teams each argue their technique deserves the last slot. How do you decide?
    Ask what happens if each one is silently broken for a full quarter. Compare the consequence of a single undetected success and the number of independent detections that survive if this one dies. The technique whose failure leaves no other coverage and no recovery path wins, regardless of which is more fashionable or which team asked louder.
  • What do you do with the 180 techniques you did not re-execute?
    Report them as untested rather than as passing, with the date each last passed and the oldest age in the set. That converts an invisible assumption into a number leadership can act on, and it is the input that justifies either more capacity or a smaller validated portfolio.

saying these in an interview costs you the question

  • Rotates strictly round-robin regardless of consequence
  • Prioritises by how famous the technique is
  • Treats past passes as evidence the path is still intact
  • Ignores which detections rest on a single fragile source
  • Reports twenty passes as if all two hundred were proven

context