skip to content

A Trivy 0.74 CI job logs 'Falling back to embedded checks' yet still passes — which misconfiguration checks did it actually run?

level: seniorimportance: nice to knowfreq 6%

answer

  1. checks are downloaded, not compiled in
  2. a copy baked in at build
  3. error logged, scan continues
  4. one repository, no fallback list
  5. mirror the bundle internally

basics

~20 s

It ran the checks embedded in the Trivy binary when that release was built, because pulling the trivy-checks bundle failed. The scan completes normally, so newer checks are silently missing until the bundle is reachable or mirrored.

solid answer

~40 s

For misconfiguration scanning Trivy 0.74 pulls the trivy-checks bundle as an OCI artifact from `--checks-bundle-repository` (default `mirror.gcr.io/aquasec/trivy-checks:2`), caches it under the cache directory and checks for a newer digest every 24 hours. If that pull fails (no egress, a blocked registry, rate limiting), Trivy logs `Falling back to embedded checks` at error level and evaluates the copy compiled into the binary at build time. Nothing fails, so the job passes with checks as old as your Trivy release. Unlike the vulnerability DB flags, `--checks-bundle-repository` takes one location, because the embedded copy is the fallback. Fix it by allowing egress to the registry, or by mirroring the bundle into an internal registry and pointing the flag at it; `--skip-check-update` then reuses whatever is already cached.

go deeper

for a junior

Recall that Trivy downloads its misconfiguration checks separately from the binary and caches them.

for a middle

Explain the 24-hour refresh, the default repository and its major-version tag, and what --skip-check-update and trivy clean --checks-bundle do.

for a senior

Show you would catch a silent fallback in CI and make air-gapped runs deterministic with a mirrored bundle and a known cache digest.

for a principal

Decide whether stale checks should fail a gate, and who owns the internal mirror's refresh and the Trivy version pinned across runners.

## Where Trivy's misconfiguration checks come from Trivy's IaC checks are not frozen into each release. They live in the **trivy-checks** project and are distributed as an **OCI artifact**: a registry object pulled like an image, but carrying an OPA bundle rather than a filesystem. When a scan enables misconfiguration (`trivy config`, or `--scanners misconfig` on another subcommand), Trivy: 1. Looks in its cache directory for a bundle and its metadata (digest, download time, bundle major version). 2. If the cache is missing, belongs to an older major version, or is more than **24 hours** old and the registry has a new digest, downloads the bundle from `--checks-bundle-repository`. 3. Loads the cached bundle and evaluates its checks. In Trivy 0.74 the default repository is `mirror.gcr.io/aquasec/trivy-checks:2`; the `:2` is the bundle's **major version**, and a cache from another major version is invalidated rather than reused. The same bundle is also published at `ghcr.io/aquasecurity/trivy-checks` and on Docker Hub as `aquasec/trivy-checks`. ## The embedded fallback Every Trivy binary also carries a copy of the checks **embedded at build time**. If the download or the cache check fails, Trivy logs an error, `Falling back to embedded checks`, and continues with that copy. The consequences: - The scan **does not fail**. Exit status is decided as usual, by findings and `--exit-code`. - The checks are as old as the Trivy release you run. A check added or fixed since then is missing; one fixed for false positives may still fire. - Two runners with different Trivy versions can report different results on the same commit. Because this fallback exists, `--checks-bundle-repository` accepts **one** location, unlike `--db-repository`, which takes an ordered list with a GHCR fallback. ## Flags that change the behaviour | Flag or command | Effect in 0.74 | |---|---| | `--checks-bundle-repository <ref>` | pull the bundle from this registry reference instead of the default | | `--skip-check-update` | do not contact the registry; use the cached bundle; if the cache is empty the run falls back to embedded | | `trivy clean --checks-bundle` | delete the cached bundle (it replaced the removed `--reset-checks-bundle`) | | `--cache-dir` | where the bundle cache lives | There is no command that downloads only the checks bundle; a scan with misconfiguration enabled populates the cache. ## Diagnosing a job that fell back - Search the job log for `Falling back to embedded checks` and the error printed with it (DNS failure, 403 or 429 from the registry, TLS interception). - With `--debug`, a healthy run logs that checks were loaded from disk and that the cache was used or refreshed. - Compare the bundle's digest in the cache metadata with the registry's current one. ## Making it deterministic - **Allow egress** from runners to the registry hosts (`mirror.gcr.io`, or `ghcr.io` and `pkg-containers.githubusercontent.com` for GHCR). - **Mirror** the bundle into an internal registry with an OCI tool such as ORAS or crane, preserving its media type, and set `--checks-bundle-repository` to the mirror. Refresh the mirror on a schedule, or your checks age there instead. - **Warm the cache** on runner images and pass `--skip-check-update` for air-gapped runs; record which bundle digest the cache holds. - **Treat the fallback message as a failure** in your own wrapper if your policy is that a gate must run current checks; Trivy itself will not. ## What the cache records, and why it matters The cached bundle sits beside metadata Trivy writes on download: - the bundle's **digest**, which identifies exactly which checks ran; - the **download time**, which drives the 24-hour refresh; - the bundle's **major version**, compared with the version this Trivy release expects (2 in 0.74) so an incompatible cache is replaced rather than loaded. Recording the digest alongside each report turns "which checks did that scan run?" from guesswork into a lookup, which is exactly the question an auditor or an incident review asks months later. ## Version skew across runners When some runners reach the registry and others fall back, the same commit can pass on one and fail on another. Pinning one Trivy version across the fleet narrows the embedded copy's spread; a reachable mirror removes the fallback altogether.

  • Your air-gapped runners use --skip-check-update. When is that still a fallback to embedded checks?
    When the cache directory holds no bundle: the cache check fails, Trivy logs the error and falls back to the embedded copy. Bake a warm cache into the runner image, or pull from an internal mirror, so a cached bundle is always there.
  • Why does a passing job not tell you the checks were current?
    The fallback only logs an error and the scan continues, so exit status is unchanged. Only the log, or the cached bundle's digest, shows which checks ran.

A satnav that loses signal and quietly switches to the road atlas printed with the car: you still reach a destination, but any road built since the car was made is missing from the route.

saying these in an interview costs you the question

  • If the checks bundle cannot be pulled the scan fails
  • Each Trivy release ships the newest checks, so no download is needed
  • --checks-bundle-repository accepts a fallback list like the DB flags
  • --skip-check-update makes Trivy use the newest checks offline
  • A green job proves the misconfiguration checks were current