A build gate finds no vulnerability scan report attached to an artifact — is that a pass?

## The bug, and why it is silent A gate that reads a scanner report almost always encodes a rule of the shape *collect the findings whose severity is above some line; deny if that collection is non-empty*. Now delete the report. There are no findings to iterate, so the collection is empty, so no denial is produced. The engine's answer — an empty set of denials — is byte-for-byte the same answer it gives for an artifact that was scanned thoroughly and came back clean. Nothing errors. In most policy languages a rule body that never matches produces no result at all, and *no result* is not `false` and is certainly not a denial. The gate assembles its verdict out of denials, so the absence of denials reads as approval. This is why "we have a scanning gate" and "every deployed artifact was scanned" are two different claims. ## Three outcomes, not two Design the gate around **pass / fail / unknown**, and enumerate what lands in unknown: - the report file is absent — the step was skipped, ran on a different branch, wrote to a different path, or its name has a typo; - the file exists but is empty, truncated, or does not parse; - the file parses but records a scan that aborted, timed out, or was cancelled; - the report describes a *different* artifact than the one about to be admitted; - the report is well-formed but the fields the rule reads are not there. Unknown is not a softer fail. It is a different fact, with a different owner and a different fix, and it deserves its own message, its own metric and its own on-call story. ## Making unknown decidable A gate can only distinguish absence from cleanliness if the evidence carries an **envelope** around the findings. The minimum useful envelope is: - **subject** — the immutable identity of what was scanned, normally the artifact digest; - **tool identity and version** — who is making the statement; - **status** — did the scan complete, and what did it cover (packages, files, layers examined); - **timestamps** — when the scan started and finished. The rule then evaluates in two stages. First, assert the envelope: a report exists, it parses, its status is complete, and its subject equals the artifact under evaluation. Only then read findings. Structured that way, a missing field trips the presence assertion instead of quietly evaporating. ## Zero findings versus no scan Zero findings is produced by a clean artifact, and also by a scanner that crashed after writing an opening bracket, a scanner pointed at a path that matched nothing, and a scanner that silently skipped an ecosystem it does not support. All four are indistinguishable if all you look at is the finding list. A completion status plus a count of what was examined separates *scanned and found nothing* from *scanned nothing*. If a report claims zero findings and also claims it inspected zero packages, that is an unknown wearing a pass costume. ## Binding evidence to the artifact Evidence must be bound to a subject, or it can be laundered. If the gate does not compare the report's subject to the digest of the artifact it is admitting, then any report that parses will do — including yesterday's green report for a previous build. Binding is what stops one clean result from vouching for an entire release train. Note that this is a statement about the artifact, not about the world: how long a report stays *useful* as advisories accumulate is a separate decision from whether the report exists and describes the right thing. ## Say the right thing when it fails A pipeline message that says *policy violation* when the real problem is a skipped scan step sends the developer hunting through their own code for a bug that is not there. Distinguish the two in output and in metrics: violations are usually the change's fault, missing evidence is usually the pipeline's fault, and a rising count of the second is a systemic scanning outage that will otherwise hide inside a pile of ordinary findings. ## Scope This is about the *evidence* being absent, which is a property of the input. What a gate should do when the policy engine itself is unreachable is a separate design decision with its own tradeoffs, and it is not what this question asks.