skip to content

A pipeline runs `checkov -d . --hard-fail-on HIGH` without an API key and has never failed despite obvious findings — what is Checkov doing?

level: seniorimportance: nice to knowfreq 11%

answer

  1. where severities come from
  2. built-in checks ship without one
  3. no match means soft fail
  4. gate by IDs instead

basics

~20 s

Checkov's built-in checks carry no severity locally; severities arrive with platform policy metadata, which needs a Prisma Cloud API key. Without it no failure matches HIGH, so --hard-fail-on finds nothing to hard-fail and every run soft-fails with exit 0.

solid answer

~50 s

In the open-source CLI a built-in check starts with no severity; Checkov fills it in from policy metadata it downloads from the platform, which needs `--bc-api-key` (a Prisma Cloud access key). Without one, every failed check has no severity, so none matches `HIGH`. `--hard-fail-on` means "hard-fail only these", so when nothing matches, the run is a soft fail and exits `0` — a silent green gate. The same root cause breaks the other severity filters: `--check MEDIUM` runs no built-in checks at all, and `--skip-check LOW` skips nothing; Checkov only logs "Filtering checks by severity is only possible with an API key" for those two. Fixes: gate on check IDs or wildcards such as `--hard-fail-on CKV_AWS_*`, drop the flag so any failure blocks, or run with a key — and keep a known-bad canary to prove the gate fires.

code

bash · 11 lines
bash
# silently green without an API key: no built-in check has a severity to match
checkov -d infra/ --hard-fail-on HIGH

# key-free alternative: gate on check IDs and wildcards
checkov -d infra/ --hard-fail-on 'CKV_AWS_*,CKV2_AWS_*'

# canary: this fixture must make the gate exit 1
if checkov -d tests/checkov-canary/ --hard-fail-on 'CKV_AWS_*,CKV2_AWS_*'; then
  echo 'gate did not fire on the canary'
  exit 1
fi

go deeper

for a junior

Remember that Checkov's severities come from its commercial platform, so severity flags behave differently without an API key.

for a middle

Explain what --check, --skip-check, --soft-fail-on and --hard-fail-on do with a severity, and what each becomes when no severity is loaded.

for a senior

Show you can diagnose a gate that never fires, switch it to ID-based criteria, and add a canary that proves it still blocks.

for a principal

Weigh whether gating should depend on a vendor's severity metadata at all, against maintaining your own blocking list of check IDs.

## Where a Checkov severity comes from Every Checkov result has a check ID, and the flags that select or gate checks accept **severities** as well: `LOW`, `MEDIUM`, `HIGH`, `CRITICAL`. It is natural to assume each built-in check ships with one. At Checkov 3.3 it does not: - A built-in check object starts with its severity **unset**. - When Checkov runs with a platform API key (`--bc-api-key`, or the `BC_API_KEY` environment variable — a Prisma Cloud access key), it downloads **policy metadata** and copies each check's severity, guideline link and platform ID from it. - Without a key — or with `--skip-download`, which the CLI reference says omits doc links, severities and similar data — the severity stays empty. - The documentation states it plainly for check selection: *"In order to filter by severity, you must run with the platform integration via API key."* - Custom checks written in YAML can declare their own `severity`, so they are the exception. ## What each severity flag does without a key | Flag as written | Intended effect | Effect with no severities | |---|---|---| | `--hard-fail-on HIGH` | block on HIGH and CRITICAL | nothing matches, every failure soft-fails, exit `0` | | `--soft-fail-on LOW` | tolerate only LOW | nothing matches, any failure hard-fails, exit `1` | | `--check MEDIUM` | run MEDIUM and above | no built-in check qualifies, so none run | | `--skip-check LOW` | drop the LOW noise | nothing is skipped | The first row is the dangerous one. `--hard-fail-on` is defined as *the list of failures that cause an error*; if no failed check matches it, the documentation says the result is a soft fail. With empty severities, no failure ever matches `HIGH`, so the gate is permanently green. Checkov logs a warning only when a severity is passed to `--check` or `--skip-check`; the soft and hard fail flags get no such warning. The third row fails in the other direction: the run looks fast and clean because it evaluated almost nothing. ## Why it goes unnoticed The pattern is common because the flags read naturally and the CLI output still looks busy. A run with `--hard-fail-on HIGH` prints every failed check, with its resource and file, exactly as before; only the exit code is different. Reviewers see red findings in the log and assume the gate is armed. Teams often copy a command line from an example that was written for a platform user, where severities exist, into a pipeline that has no key. The JUnit XML output even marks the gap: for users without an API token, its documentation says, the severity in each test-case name is `[NONE]`. ## How the decision is made per failed check For each failed check Checkov applies these rules in order: 1. Check ID or wildcard in `--hard-fail-on` → hard fail. 2. Check ID or wildcard in `--soft-fail-on` → soft fail. 3. Severity at or above the `--hard-fail-on` severity → hard fail. 4. Severity at or below the `--soft-fail-on` severity → soft fail. 5. Otherwise → the value of `--soft-fail`, except that a hard-fail list with no soft-fail list makes unmatched failures soft. Any single hard fail makes the run exit `1`. Rules 3 and 4 need a severity; without one, only the ID rules and the fallback remain. ## Making the gate honest - **Gate on IDs, which never depend on the platform.** `--hard-fail-on CKV_AWS_*,CKV2_AWS_*` or an explicit list of check IDs the team agreed must block. - **Or invert the default.** Run without `--hard-fail-on`, so any failure blocks, and tolerate specific IDs with `--soft-fail-on`. - **Or use the platform deliberately.** With an API key, severities are populated, and `--use-enforcement-rules` can pull centrally managed thresholds per scanner category. That is a commercial dependency; pricing and platform internals are out of scope here. - **Prove the gate fires.** Keep a small known-bad fixture — for example a bucket with `acl = "public-read"` — in a test directory and assert that the gated command exits `1` on it. A gate that has never failed is unverified, not necessarily clean. What threshold an organisation should block on is a policy decision made elsewhere. The Checkov-specific lesson is mechanical: **a severity filter is only as good as the severities the run actually has.**

  • Why does `--check MEDIUM` without an API key produce a fast, clean run?
    Severity in `--check` means "run checks at or above this level". With no severities loaded, no built-in check qualifies and no check IDs were listed, so almost nothing runs. Checkov logs "Filtering checks by severity is only possible with an API key", but the exit code is `0`, which looks like a clean repository.
  • How would you notice this class of problem before an auditor does?
    Assert that the gate can fail: keep a known-bad fixture and a CI step that expects a non-zero exit on it, and watch the passed and failed counts in the JSON summary. A severity-gated run whose counts are high but whose exit code never changes is the signature of a gate with no severities.

saying these in an interview costs you the question

  • Every built-in Checkov check ships with a severity in the open-source CLI.
  • --hard-fail-on HIGH blocks on everything when severities are missing.
  • Checkov refuses to start if you pass a severity without an API key.
  • --check MEDIUM runs all checks when no severities are available.
  • A gate that has never failed proves the code is clean.