skip to content

What does mandating a data-flow diagram for every design review cost you?

level: principalimportance: nice to knowfreq 26%

answer

  1. one notation, one ceiling
  2. comparable reviews versus capped findings
  3. no findings is not no threats
  4. trigger list for a second view
  5. measure finding categories, not counts

basics

~20 s

Standardising on one diagram makes reviews comparable and the practice teachable, but it silently caps what teams can find. Protocol, retry, error-path and co-location threats do not fit the notation, so nobody raises them and the gap reads as safety.

solid answer

~50 s

Standardisation is genuinely worth something: a single notation makes reviews comparable across teams, lowers the floor for teams new to the practice, and gives reviewers a checklist they can run without a security specialist in the room. The cost is that the notation quietly becomes the definition of the threat surface. A data-flow view has nowhere to express ordering, retries, failure paths or where two components actually run, so those threat families never get reported, and a review with no ordering findings looks identical to a system that has none. Keep the DFD as the mandatory entry point, but attach a trigger list, a handshake, a retried flow, a fallback path, shared hosting or a shared identity, where a second view is required. Then track findings by category, because a category that never appears is a blind spot, not a clean bill of health.

go deeper

for a junior

You will not be asked to set this policy, but know that a completed threat-model template is not proof a system is safe. Be ready to say what a data-flow diagram cannot express.

for a middle

Be able to argue both sides in one breath: standardising makes reviews comparable and teachable, and it also limits findings to what the notation can hold. Naming two threat families the template loses is the expected depth.

for a senior

Show how you would work inside the mandate: trigger conditions for a supplementary view, three unhappy-path questions per flow, and a disposable sketch rather than a second maintained artifact. Ground it in a review where the template missed something you caught.

for a principal

Own the programme-level trade: a uniform floor is what makes the practice scale, and category-level metrics are how you detect that the floor has become a ceiling. Be ready to defend the specific triggers you would mandate and what you would deliberately not require.

## The real argument for standardising Mandating one notation across an organisation is not bureaucratic instinct; it buys concrete things. Reviews become **comparable**: you can look at forty models and see which systems were examined at what depth. The practice becomes **teachable**, because a new engineer learns one grammar rather than choosing among five. Reviews become **runnable without a specialist**, since an element walk over a data-flow diagram is mechanical enough for a team to do unaided. And a consistent artifact is something a programme can measure, sample and audit. For an organisation trying to get from no threat modelling to some threat modelling, that floor matters more than the ceiling. ## The cost, stated precisely The notation becomes the definition of the threat surface. Three effects compound. **Whole threat families go unreported.** A data-flow view cannot express message ordering, delivery semantics, retry behaviour, failure branches, or the physical placement of components. So replay, check-then-use races, double-processing on retry, spills into undrawn failure sinks, and co-location of components that the design claims are separate simply do not come up. Not because anyone decided they were out of scope, but because there was no box to write them in. **Absence looks like assurance.** A completed template with no ordering findings is indistinguishable from a system with no ordering threats. The artifact says the review happened, and everyone downstream reads that as coverage. **Teams satisfy the template rather than think.** This is the deepest cost. Shostack's four questions are: what are we working on, what can go wrong, what are we going to do about it, and did we do a good enough job? A mandated template answers the first question well and turns the second into a form-filling exercise. The fourth question, which is the one that keeps a practice honest, gets answered by the existence of the completed form. ## The design I would actually run - **Keep the DFD as the default and mandatory entry point.** It is the right first instrument: it inventories data stores, external entities and boundary crossings better than anything else, and it is the cheapest thing to teach. - **Attach an explicit trigger list.** Name the conditions that require a second view, in the team's own language: there is a handshake or multi-step protocol; a flow is retried or delivered at least once; there is a fallback or degraded path; two components share a host, a process or an identity; a failure branch writes data anywhere. Any trigger means a second, throwaway view of that one interaction, not a second full model. - **Ask the unhappy-path questions per flow.** What happens on failure, on repeat, out of order. Three questions per flow catches most of what the notation hides and costs minutes. - **Measure findings by category, not by count.** A programme reporting hundreds of findings, none of which is ever an ordering, retry or placement threat, has found its blind spot. Category distribution is the metric that detects a notation ceiling; raw counts hide it. - **Let the second view be disposable.** The reason teams resist extra diagrams is maintenance. A message-ordering sketch drawn on a whiteboard during the review, photographed, and attached to the model, is not an artifact anyone has to keep current. Only the DFD earns ongoing maintenance, because only it describes the whole system. ## The judgment being tested An interviewer asking this wants to see that you can hold two things at once: that consistency is what makes a practice scale, and that consistency is also what makes it stop learning. The weak answers sit at either pole, either defending the template because it is the standard, or arguing that every team should pick its own notation, which produces models nobody can compare and reviewers nobody can train. The strong answer keeps a mandatory floor and makes exceeding it cheap, explicit and triggered by conditions the team can recognise without a security specialist telling them. The same reasoning generalises past diagrams. Any standardised security artifact, a checklist, a control set, a question list, caps the findings at what it has room to express, and the job of whoever owns the programme is to watch for the categories that never appear.

  • How would you detect that the notation ceiling is actually biting, rather than assuming it?
    Track findings by category across models. If ordering, retry, failure-path and placement threats are near zero across dozens of reviews while injection and boundary findings are plentiful, the distribution is telling you what the notation can express, not what the systems contain. Cross-check against incidents and post-incident reviews: a class of production incident that never appears as a modelled threat is the same signal from the other end.
  • A team wants to skip the mandated diagram because they say their system is simple. How do you respond?
    Hold the floor but shrink it. The value of the mandate is that every design gets examined at all, and simple is a judgment made by the people closest to the design. A one-page context-level sketch plus the three unhappy-path questions per flow takes under an hour and either confirms simplicity or does not. If it genuinely is trivial, the artifact is cheap; if the team is wrong, that is exactly the case the mandate exists for.
  • Would you let teams choose their own notation instead?
    No, not as the default. Free choice produces models that cannot be compared, reviewers who cannot be trained on one grammar, and no way to sample quality across the organisation. The workable middle is a mandatory common entry point plus explicitly permitted, disposable supplementary views, so the floor is uniform and exceeding it is cheap rather than a negotiation.

saying these in an interview costs you the question

  • Treats a completed template as evidence of coverage
  • Says every team should pick its own notation
  • Counts findings without looking at their categories
  • Assumes a diagram-based review finds every threat family
  • Requires a full second model instead of a disposable sketch

context