You add Checkov to a Terraform repo with hundreds of existing failures — how do `--create-baseline` and `--baseline` gate only new ones, and where does the match leak?
answer
- a snapshot of known failures
- the .checkov.baseline file
- matched by resource and check ID
- the file path is not compared
basics
~20 s--create-baseline writes the current failures to a .checkov.baseline file; later runs with --baseline report only failures not in it. Matching uses resource address plus check ID, ignoring the file, so renames resurface old debt and same-named resources slip through.
solid answer
~40 sRun `checkov -d . --create-baseline` once: alongside the normal output, Checkov writes `.checkov.baseline` at the root of the scanned directory, a JSON list of files, each with the resources and the check IDs that failed on them. Commit it, and gate with `checkov -d . --baseline .checkov.baseline`: failures already in the file are removed before the exit code is computed, so only new failures block. The match is on the **resource address plus check ID** — the file path is recorded but not compared. Renaming `aws_s3_bucket.logs` makes its old failures look new, while a new `aws_s3_bucket.this` in a different directory that fails a check already baselined for another `aws_s3_bucket.this` passes silently. `--output-baseline-as-skipped` keeps baselined failures visible as skipped.
go deeper
Recall that Checkov can record today's failures in a .checkov.baseline file and then fail the build only on failures that are not in it.
Explain the create-then-gate workflow, the JSON structure of the file, and how --output-baseline-as-skipped keeps the debt visible.
Show you know the match key is resource address plus check ID, and plan for refactors, common resource names and Checkov upgrades that bend it.
Decide how a baseline is owned and burned down over time so it shrinks as a debt ledger instead of becoming a permanent exemption list.
## The brownfield problem Turning on any infrastructure scanner in an existing repository produces the same day-one result: hundreds of failures, most of them old, many of them accepted long ago. Blocking every merge until they are all fixed stalls delivery; soft-failing everything stops the scanner from blocking anything. Checkov's **baseline file** is the middle path: record today's failures as known debt, and fail the build only on failures that are **new relative to that record**. Why that is a sound gating strategy is a general static-analysis idea; this answer is about how Checkov implements it. ## Creating and using the baseline 1. **Create it.** `checkov -d . --create-baseline` runs a normal scan and, alongside the usual output, writes a file named `.checkov.baseline` at the root of the scanned directory. 2. **Commit it** next to the Terraform it describes, so it is versioned and reviewed like code. 3. **Gate with it.** `checkov -d . --baseline .checkov.baseline` runs the scan and then removes every failure that the baseline already contains. The report and the exit code reflect only what is left. 4. **Optionally keep the debt visible.** `--output-baseline-as-skipped` re-labels the baselined failures as skipped instead of dropping them, so the backlog stays in the report. The file is plain JSON: ```json { "failed_checks": [ { "file": "/storage/main.tf", "findings": [ {"resource": "aws_s3_bucket.logs", "check_ids": ["CKV2_AWS_6", "CKV_AWS_18"]} ] } ] } ``` It records **failures only** — passed and skipped results are not in it. ## How the match actually works At Checkov 3.3 a failure counts as "in the baseline" when **some entry in the file has the same resource address and lists the same check ID**. The `file` field is written but **not compared**. That single design choice explains the surprises: | Change in the repository | What the gated run reports | |---|---| | nothing changes | no failures from the baseline; exit `0` if nothing new | | a baselined resource gets a new failure (new check ID) | the new failure only | | `aws_s3_bucket.logs` renamed to `aws_s3_bucket.audit_logs` | all its old failures, as if new | | a new `aws_s3_bucket.this` in another directory fails a check already baselined for an existing `aws_s3_bucket.this` | nothing — treated as known debt | | a Checkov upgrade adds a new check that old resources fail | every such failure, as new | ## Where it leaks, and what to do - **Renames and refactors resurrect debt.** Moving resources into modules or renaming them changes the address, so the gate fires on old problems. That is noisy but safe; refresh the baseline as part of the refactoring change, reviewed like any other diff. - **Common addresses hide new problems.** Generic names such as `this` or `main` are popular in module-style Terraform. A brand-new resource that happens to share an address and a failing check with a baselined one is absorbed. That is the dangerous direction. Mitigations: keep one baseline per independently scanned root directory rather than one for a monorepo, and prefer specific resource names. - **Upgrades add checks.** A new Checkov release can add checks that old resources fail. Pin the Checkov version in CI and bump it deliberately, refreshing the baseline in the same change. - **A regenerated baseline forgives everything.** Running `--create-baseline` again records whatever is failing today, including yesterday's regression. Who may regenerate it and how that is reviewed is a gate-policy question; mechanically, treat changes to `.checkov.baseline` as security-relevant diffs. ## Baseline versus the other levers - `--soft-fail` makes **all** failures non-blocking; the baseline makes only the **recorded** ones non-blocking. - `--skip-check` removes a check for **every** resource; the baseline excuses a check only on the **listed** resources. - An inline skip comment excuses one resource with a reason in the code; the baseline carries no reasons at all, so it is a debt ledger, not an exception record. ## Operating the baseline day to day - Review `.checkov.baseline` diffs like code: a shrinking file is progress, a growing one needs a reason. - Track its size over time; a baseline that never shrinks has become an exemption list. - Fixing a baselined failure needs no file change, but removing the stale entry keeps the debt count honest. The interview signal is knowing both halves: the baseline is the standard way to adopt Checkov in a brownfield repository without stalling merges, and its match key is the resource address and check ID — which is exactly where it can be fooled.
- The gate suddenly reports forty "new" failures after a refactor that only moved resources into a module. What happened, and what do you do?Moving resources into a module changes their addresses, and the baseline matches on address plus check ID, so the old failures no longer match and look new. Confirm the findings are the same debt by comparing check IDs and resources, then regenerate `.checkov.baseline` in the same pull request, so reviewers see the refresh beside the refactor.
- Why might a team keep several baseline files in one repository instead of one?Because the match ignores the file path, one repository-wide baseline lets any resource with a common address, such as `aws_s3_bucket.this`, inherit another directory's excused failures. Scanning each root directory separately with its own baseline narrows that collision space to the directory where the debt actually lives.
A baseline file is a photograph of the cracks already in a wall: new cracks show up against it, but if you repaint and renumber the bricks, the old cracks look new, and a fresh crack on a brick with the same number as an old one goes unnoticed.
saying these in an interview costs you the question
- The baseline matches failures by file and line number.
- Baselined failures still make Checkov exit 1, they are only labelled.
- The baseline file records passed checks so regressions can be spotted.
- Renaming a resource keeps its baseline entry valid.
- A baseline and --soft-fail have the same effect on new failures.