skip to content

Why is sorting a threat model's rated threats by score descending a poor fix order?

level: principalimportance: must knowfreq 66%

answer

  1. severity is an input, not the order
  2. the unit of work is the change
  3. cluster by root cause first
  4. cost, leverage, dependency, deadline
  5. what gets harder to fix after launch

basics

~20 s

A severity score says how bad a threat is - not what the fix costs, how many other threats it clears, or what must be right before a fixed launch date. Sorting by score alone gives an indefensible order of work.

solid answer

~50 s

Severity is one input to the order, not the order itself. Take a session on an electric-vehicle charging back-office that produced thirty-one threats; four of them — a roaming partner's operator credential adjusting tariffs, cancelling a session, reading another operator's settlement data and enrolling a charger — trace to one over-broad service role. Splitting that role is a single change that clears four rows, which beats a higher-scored threat that clears one. Beyond fix leverage I weigh **effort and lead time** (ship the one-line configuration change now, start the quarter-long redesign in parallel rather than behind it), **irreversibility against a fixed date** (on a consent service with a regulatory go-live, anything that cannot be changed after launch outranks a higher-scored item that can ship later), **dependencies** between mitigations, and **confidence in the rating** — a cheap probe that settles an uncertain High is often the best first move. Within a band, cost and leverage decide.

go deeper

for a junior

Know that the rated list is not automatically the work order, and that how expensive a fix is and how many threats it clears both matter. Be able to name one criterion beyond severity.

for a middle

Be ready to walk a small rated list into an ordered queue out loud: cluster rows that share a root cause, estimate the change rather than the threat, and explain why the top-scored row is not always first.

for a senior

Show the judgment calls — sequencing a cheap mitigation ahead of a redesign without cancelling the redesign, scheduling enablers that carry no rating, and treating an uncertain rating as something to resolve rather than to sort.

for a principal

Own the policy: what the ordering rule is, who may override it, and how you stop a queue from degenerating into either all-cheap-wins or one-heroic-rewrite. Expect to defend the rule against a fixed external date.

## The gap between a rating and a queue Rating answers "how bad would this be?". A fix queue has to answer "what should we do on Monday?". Those are different questions, and the second one takes inputs the rating never contained. A score descending sort silently asserts that severity is the only thing that matters and that every fix costs the same, which is never true. Five inputs beyond the score decide the real order. ## 1. Fix leverage — count coverage, not rows A modeled threat list is not a set of independent items. Several rows often share one root cause, and one change clears them all. In a session on an electric-vehicle charging back-office, thirty-one threats came out of one model; four of them collapsed into a single over-broad service role that a roaming partner's operator credential could assume — tariff adjustment, session cancellation, another operator's settlement data, and charger enrolment all rode on the same grant. Splitting that role into scoped identities is one piece of work that removes four rows. Ranked by score, three of those four sat mid-list; ranked by leverage, the shared fix is the first thing you do. The practical move is to cluster the list by root cause before ordering it. The unit of work is the **change**, not the threat, and the value of a change is the total risk it removes. ## 2. Effort and lead time A one-line configuration change and a quarter-long redesign do not belong in the same queue position just because they mitigate similarly-scored threats. Cheap band-reducers ship immediately; they are not a substitute for the redesign, but they buy time at almost no cost. And a long fix should *start* early precisely because it is long — sequencing it behind three cheap ones wastes its lead time. The right shape is usually: cheap mitigations ship now, the expensive one starts now on its own track, and nothing waits behind anything it does not depend on. ## 3. Irreversibility against a fixed date Some fixes get harder the longer you wait; some are equally easy forever. On an open-banking consent service with a fixed regulatory go-live, the ordering axis is not severity but **changeability after launch**. A consent record's data model, the scope granularity a third party is granted, and the audit trail's contents are all things you cannot quietly reshape once real delegated credentials and real payment mandates exist. A missing rate limit or a stricter token lifetime can ship the week after. So a mid-scored design decision that is baked at launch is pulled ahead of a higher-scored control that is easy to add later. "Can I still do this cheaply in three months?" is often a sharper ordering question than "how bad is it?". ## 4. Dependencies between mitigations Some mitigations only work once another exists. You cannot scope a service role usefully until the callers have distinct identities; you cannot enforce a per-tenant authorization check until requests carry a reliable tenant claim. A queue that puts the dependent fix first produces a sprint of half-done work. Order the enablers first even when they carry no score of their own — an enabler is not a threat, so it never appears in the rated list at all, which is exactly why a pure score sort loses it. ## 5. Confidence in the rating Ratings carry uncertainty and the score hides it. A High that rests on "we think that queue is reachable from the partner network" is worth less than a Medium everyone is certain of. The cheapest next action is often a half-day check that confirms or collapses the assumption, because it changes the order of everything below it. Treat "verify the assumption" as a schedulable item. ## Where the score does belong None of this makes severity irrelevant. It is the primary partition: the top band gets attention this cycle, the bottom band does not. Within a band — and bands are coarse by design — cost, leverage, dependency and deadline decide. A useful ordering pass looks like: cluster by root cause, keep only the highest band of each cluster as its severity, estimate effort per cluster, mark anything gated by a fixed date, then sort by removed-risk per unit of effort with the date-gated and dependency-enabling items pulled to the front. ## The failure this prevents The classic bad outcome is a team that spends a quarter on the single top-scored threat — usually the most architecturally expensive one — while eight mid-band threats that one afternoon of configuration would have removed stay open the whole time. The score was right and the order was wrong. An interviewer asking this is checking whether you can hold both facts at once: the rating is honest, and the rating is not the plan.

  • How do you keep 'it is cheap' from becoming the only criterion?
    By keeping severity as the partition and cost as the tie-break inside it, never the reverse. Cheap work in the bottom band does not get scheduled ahead of the top band just because it is easy. I also watch for the pattern where every quarter closes ten trivial rows and the one structural threat is never started — that is the signal to schedule the expensive fix on its own track with its own lead time rather than competing row by row.
  • Four threats collapse into one root cause. Do you merge them into a single row?
    I cluster them but keep the rows. The rows are the evidence that the change is worth doing and the checklist for confirming the fix actually closed each path — merging loses three of the four. What I do merge is the *work*: one change, one estimate, one place in the queue, carrying the highest band of the cluster as its severity rather than a sum or an average of the four.
  • Two threats have the same band, the same effort and no dependencies. How do you break the tie?
    On confidence and on blast radius of getting it wrong. I prefer the one whose rating I am least sure of, because resolving that uncertainty may reorder the rest of the queue, and the one whose failure is hardest to detect after the fact — a silent integrity or audit-truth failure over a loud availability one, since the loud one at least tells you it happened.
  • Does a fixed launch date ever justify shipping with a top-band threat open?
    Sometimes, but only when the fix stays equally cheap afterwards and something compensating is in place meanwhile — a tighter limit, a narrower exposure, a detection. What I refuse to defer on a date is anything baked in at launch: the data model, the granularity of what third parties are granted, the contents of the audit trail. Those get harder by an order of magnitude once real records exist.

saying these in an interview costs you the question

  • Sorts by score descending and calls that the plan
  • Treats every rated threat as an independent unit of work
  • Ignores fix cost and lead time entirely
  • Sums or averages the scores of clustered threats
  • Never schedules enablers because they carry no score
  • Assumes anything can be added just as cheaply after launch

context