skip to content

In a security findings gate, how does blocking on new findings differ from blocking on total findings?

level: middleimportance: must knowfreq 60%

answer

  1. absolute state versus what changed
  2. the person blocked must be able to fix it
  3. findings need a stable identity
  4. committed file of known fingerprints
  5. gate on the derivative, report the value

basics

~20 s

Blocking on total findings fails every build until the entire backlog is clean. Blocking on new findings compares each report against a recorded baseline of what already existed and fails only on what the current change added.

solid answer

~50 s

A total-findings rule asks an absolute question: does this repository contain anything at or above the line? On any codebase older than a week the answer is yes, so every build fails and the gate is disabled within days. A new-findings rule asks a relative question: does this report contain anything that was not in the recorded baseline? To do that, each finding needs a stable identity — a fingerprint over the rule id plus enough of the surrounding context to survive an unrelated edit — and the set of already-known fingerprints is committed to the repository as a baseline. The gate then fails only on fingerprints absent from that file. The practical policy is two-lane: new findings block, the grandfathered total is reported and warned on, and the backlog is owned separately with dates rather than being enforced through everyone's build.

code

yaml · 18 lines
yaml
# scan report
summary: { critical: 1, high: 3, medium: 11 }
findings:
  - rule: storage-bucket-public-read
    severity: critical
    location: services/api/storage.tf
    fingerprint: 9f2a...c07
  - rule: missing-owner-label
    severity: high
    location: services/api/deploy.yaml
    fingerprint: 41bd...e92
  # ...

# committed baseline: already-known fingerprints
baseline:
  - fingerprint: 9f2a...c07
    owner: platform-team
    expires: 2026-04-30

go deeper

for a junior

Know that a gate can compare against a recorded list of already-known findings instead of failing on everything present, and that the recorded list lives in the repository.

for a middle

Explain how a finding gets a stable identity, why line numbers make a bad key, and how the two-lane policy of blocking new while reporting total keeps a gate usable on an old codebase.

for a senior

Show the operational consequences: the refactor that legitimately produces new fingerprints, the ratchet that holds the number flat forever, and the need to track baseline size as a metric of its own.

for a principal

Be ready to argue that the pipeline should carry the regression detector while the absolute total belongs in a funded remediation programme, and to say who owns that programme and by when.

## The absolute question and the relative question **Total findings** is an absolute rule: *does the current state of this repository contain anything at or above the line?* It is the rule everyone writes first, and on any codebase with history the answer is yes on day one. Every build then fails, including builds from people whose change touched nothing related, and the gate is downgraded or removed within a sprint. **New findings** is a relative rule: *does this report contain anything that was not already known?* It scopes the block to what the change introduced, so the person who is stopped is the person who can act. That is the property that makes a gate survivable. ## What makes 'new' computable Comparing two reports requires each finding to have an identity that is stable across runs. Three candidate keys, in increasing order of usefulness: - **Rule id alone.** Too coarse. Fixing one instance of a rule and introducing another elsewhere looks like no change at all. - **Rule id plus file and line.** Too brittle. Inserting a line at the top of the file renumbers everything, so an unrelated edit presents the whole file as new findings and blocks a change that introduced nothing. - **A fingerprint** over the rule id, the file, and a normalised hash of the offending construct or the nearest stable anchor. This survives renumbering and formatting, which is what you need in practice. No fingerprinting scheme is perfect. A genuine refactor that moves and rewrites code will legitimately produce new fingerprints for old problems, and the gate will block. That is a known cost, and it is why the process around the baseline matters as much as the algorithm. ## The baseline file The recorded set of already-known fingerprints is committed to the repository. Its properties are worth stating explicitly in an interview: - **It is data, not a suppression.** It does not say a finding is acceptable; it says the finding predates this policy. The finding stays in the report and stays counted. - **It is versioned with the code.** You can see when an entry appeared and in which change, which is what makes the file reviewable at all. - **It is writable by the team it constrains.** Anyone who can commit can widen it. This is the single most important weakness of the mechanism, and it drives the review rules around updating it. - **It should decay.** Entries with an owner and an expiry date turn the backlog into scheduled work; entries without them turn it into permanent furniture. ## The two-lane policy The shape most teams converge on: | Lane | Condition | Outcome | | --- | --- | --- | | Blocking | Finding at or above the line, fingerprint not in the baseline | Build fails | | Reporting | Everything else, including the entire grandfathered backlog | Recorded, surfaced, counted, does not stop the build | This is a deliberate decision not to enforce the existing backlog through the pipeline. It is defensible on one condition: the backlog is being worked somewhere else, with named owners and dates. Without that, the reporting lane is just a place where findings go to be ignored, and the gate quietly becomes a ratchet that holds the number steady forever rather than driving it down. ## What the two rules tell you They answer different questions and a good candidate says so. *New findings* answers **is this change making things worse?** — it is a regression detector, and it is the right thing to wire into a build. *Total findings* answers **where do we stand?** — it is a trend line, and it belongs on a dashboard, in a monthly review, and in the conversation with whoever funds remediation. Wiring the trend line into the build is what produces the day-one wall of red; leaving the regression detector out of the build is what lets the number creep up one change at a time while everyone watches the dashboard. ## Failure modes to name - **Ratcheting to a standstill.** New findings blocked, total ignored, backlog never shrinks. The gate is working and the security posture is not improving. - **Refactor blockage.** A large rewrite produces new fingerprints for old problems and blocks a change that fixed things. Needs a human path, not a lower threshold. - **Baseline growth.** The file only ever gets bigger. Track its size as a metric in its own right; a baseline that grew this quarter is a policy event whether or not any build failed. The compact way to put it: gate on the derivative, report on the value.

  • Why is file plus line number a poor identity for a finding?
    It is not stable under edits that change nothing relevant. Inserting a line near the top renumbers everything below it, so the whole file arrives as unmatched findings and the gate blocks a change that introduced no new problem. A fingerprint over the rule id and a normalised form of the offending construct survives renumbering and reformatting, which is what a comparison needs.
  • How is a baseline entry different from marking a finding as accepted?
    A baseline entry is a statement of chronology: this existed before the policy did. The finding stays in the report and stays in the total. Accepting a finding is a risk decision about that specific issue and is a separate mechanism with its own review. Conflating them turns a temporary grandfathering list into a permanent, unreviewed acceptance register.
  • The new-findings gate is green every week and the total never drops. Is the gate working?
    It is doing exactly what it was configured to do and it is not improving anything. A regression detector holds the line; it never walks it back. Pair it with owned, dated backlog work and track the baseline's size as a metric, so a flat or growing total is visible as a failure of the remediation programme rather than hidden behind a green build.

saying these in an interview costs you the question

  • Treats a baseline as a suppression list for accepted risk
  • Keys findings on file and line number
  • Assumes a green new-findings gate means the backlog is shrinking
  • Blocks on total findings against a legacy codebase
  • Never tracks how large the baseline has grown

context