Which SBOM is safer to act on: one exact about 12 direct dependencies, or one listing 800 transitives with a third of the versions wrong?
answer
- Two different failure modes, not two amounts
- Wrong versions cut both ways
- You cannot tell which third is wrong
- Bounded gap versus unbounded contamination
- Everything hinges on declared scope
basics
~20 sThe narrow, exact one, provided it says it is narrow. An inaccurate list poisons every query built on it and you cannot tell which third is wrong. An incomplete list with a declared boundary lets you fill the rest in yourself.
solid answer
~50 sThese fail in different ways, and that decides it. Wrong versions produce *confidently wrong* answers in both directions: a version listed too high hides a real advisory match, one listed too low floods triage with findings on components that were already patched. Because you cannot tell which third is wrong without re-deriving the whole thing, the errors contaminate the accurate two-thirds too — the document's value is roughly zero and its cost is positive, since people spend time on it. The narrow document is at least true as far as it goes, so I can use it as evidence and derive the missing transitive layer myself. The catch is the declaration: if it silently stops at direct dependencies, a consumer reads absence as absence of risk, and since most vulnerable code arrives transitively that is a huge blind spot. Incomplete-and-honest is workable; incomplete-and-silent is as dangerous as inaccurate.
go deeper
Know that an SBOM can be wrong in two ways — missing rows, or rows with bad data — and that a wrong version means an advisory lookup gives the wrong answer. Do not assume the longer list is automatically the better one.
Explain the mechanics: version too high hides a match, version too low creates noise, and an unidentifiable error rate contaminates the rows that are correct. Be able to say why a gap is measurable while an accuracy error is not.
Demonstrate the operational move — accept the narrow document as evidence for its scope, derive the rest yourself, and refuse to run vulnerability response off an inventory whose identity data has not survived a spot-check.
Own the incentive problem: procurement checklists count rows, so suppliers optimise for coverage and nobody is graded on identity precision. Decide what your organisation actually requires before a supplier inventory counts as evidence, and be ready to defend accepting a smaller, truer document.
## Two different kinds of wrong An inventory can fail on **completeness** (rows that should be there are missing) or on **accuracy** (rows that are there say the wrong thing). Interviewers pose them head to head because candidates reflexively answer "more coverage is better", and the reasoning underneath is what distinguishes someone who has actually consumed these documents. ## Why inaccuracy is the worse failure Everything you do with an inventory is a lookup keyed on component identity. Get the version wrong and the lookup fails in both directions: - **Version listed higher than reality** → the advisory's affected range does not match → a false negative. You believe you are clean. This is the one that ends up in an incident review. - **Version listed lower than reality** → matches ranges you already patched out of → a flood of false positives. That costs triage hours, and worse, it teaches the people doing triage that the document lies, after which nobody reads it. The compounding problem is **undetectability**. If a third of the versions are wrong and you do not know which third, the error is not confined to 267 rows: it removes your justification for trusting any row. Verifying the document costs the same as re-deriving the inventory from scratch, so the document has contributed nothing while consuming attention. That is a negative-value artefact — worse than no document, because it also produces false assurance. Completeness errors behave better. They are *bounded and measurable*: you can compare the row count and the component set against your own view of the same artefact, express it as coverage, and know exactly what class of thing is missing. And what is present remains true, so it is still usable as evidence. ## The condition attached to that answer The narrow document only wins **if its scope is declared**. "This document enumerates direct dependencies of the application only" turns a gap into a known boundary: a consumer knows to derive the transitive closure themselves, and no query silently answers *no* about something that was never in scope. Undeclared, the same document is dangerous for exactly the reason the inaccurate one is. A consumer asked "do we run this library?" gets a clean *not present*, and stops. Direct dependencies are the tip: modern applications carry the great majority of their code in the transitive layer, and that is where a vulnerable component usually arrives — nobody chooses a compression library directly, they choose the framework that pulls it in four levels down. A twelve-row document that implies it is the whole story is a false inventory of an 800-component product. So the honest ranking is: 1. Accurate and complete. 2. **Accurate, narrow, and explicit about being narrow.** 3. Accurate but silently narrow. 4. Broad but unreliable. Three and four are close, and which is worse depends on how the consumer uses them. Both share the property that they invite confident wrong conclusions. ## What you actually do when handed one of them - **The narrow, honest one.** Accept it as evidence for what it covers. Derive the missing layer from the artefact yourself and treat your derived inventory as the operational one for triage. The supplier document remains useful for the direct layer, where its identity data is trustworthy. - **The broad, unreliable one.** Do not run a vulnerability programme on it. Use it at most as a hint list — a set of component names worth checking — and never as the authority for "we are not affected". Then go and find out why the identity data is unreliable, because a third of versions wrong is not a rounding error. ## The measurement that makes this concrete Quality here is two independent numbers, and reporting one alone is how programmes fool themselves: | Metric | Question it answers | How you get it | |---|---|---| | Coverage | What fraction of the components in the artefact appear at all? | Compare the document's set against your own inspection of the same artefact | | Identity precision | What fraction of listed rows carry a version and a resolvable identifier that turns out to be right? | Spot-check a sample against the artefact | A document scoring high on the first and low on the second looks impressive on a procurement checklist and is useless in an incident. The checklist counts rows; the incident asks whether row 412 is really version 1.4.2.
- Concretely, what does "a third of the versions are wrong" cost you during an advisory response?Both directions hurt. Versions listed too high produce false negatives — the affected range does not match and you report clean. Versions listed too low produce noise on components already patched, which burns triage hours and destroys the document's credibility with the people using it. And because you cannot tell which rows are wrong, you have to re-derive the inventory anyway, so the document cost you time and gave nothing back.
- Does it change your answer if the narrow document explicitly states that it covers direct dependencies only?That is the whole answer. Declared, the gap becomes a boundary: I derive the transitive layer myself and no query silently answers "not present". Undeclared, a consumer reads absence as absence of risk, and since most vulnerable code arrives transitively that is a large invisible blind spot — nearly as bad as the inaccurate document.
- Why is the transitive layer where the risk usually lives?Because nobody selects those components. A team picks a framework and inherits parsers, compression, cryptography and serialisation four levels down, chosen by someone else's version constraints. That layer is bigger than the direct one by an order of magnitude, gets less scrutiny at selection time, and is where an advisory typically lands. A direct-only inventory misses most of the product's actual code.
- How would you express this as two metrics rather than one quality score?Coverage — what fraction of the components actually in the artefact appear at all — and identity precision — what fraction of listed rows carry a version and identifier that survive a spot-check. They are independent, and a document can score high on one while being worthless on the other. A single blended score hides exactly the trade the question is about.
saying these in an interview costs you the question
- Assumes more rows always means a better inventory
- Ignores that wrong versions cause false negatives too
- Thinks a partial list is safe without a declared scope
- Treats direct dependencies as the bulk of the risk
- Reports one blended quality score hiding both failure modes