skip to content

A Trivy 0.74 image scan reports a private key in a file your Dockerfile deletes in a later step — is that a false positive?

level: seniorimportance: should knowfreq 14%

answer

  1. layers are analysed one by one
  2. the merge keeps old secrets
  3. who added the layer
  4. base layers are treated differently
  5. image config is a separate opt-in

basics

~20 s

No. Trivy scans each image layer separately and keeps secrets from every layer, even when a later layer deletes the file, because that layer still ships in the image. The finding names the instruction that added it; rotate the key.

solid answer

~40 s

It is a true positive. Trivy analyses each layer of the image on its own, and when it merges layers into the final view it deliberately keeps secret findings from every layer, even if an upper layer removed the file. Anyone who pulls the image gets that lower layer too. Each finding carries the layer's `Digest`, `DiffID` and `CreatedBy`; the table shows it as `(added by 'COPY ...')`, which points at the Dockerfile step to fix. Treat the key as exposed and rotate it, then rebuild so it never lands in a layer. Two Trivy details limit coverage: it guesses which layers belong to the base image from the image history and skips secret scanning on them, and secrets in the image configuration, such as `ENV` values and history, need `--image-config-scanners secret`.

go deeper

for a junior

Remember that an image keeps every layer, so Trivy can find a secret in a file that a later step deleted.

for a middle

Explain per-layer analysis, the Layer fields on a finding and how the added-by note leads back to the Dockerfile step.

for a senior

Respond correctly: rotate, rebuild, re-publish, and state the blind spots: guessed base layers, image config, skipped paths.

for a principal

Decide where secret scanning runs across the pipeline so base images and image configs are covered, and who owns a hit in a shared base.

## How Trivy scans an image for secrets A container image is a stack of **layers**, each a tar archive of the files one build step added, changed or deleted. Deleting a file in a later step does not edit the earlier layer; it adds a **whiteout** entry in the new layer that hides the file in the merged filesystem. The earlier layer's blob, with the file inside, still ships with the image. (Why the build leaves it there is a container-build topic; here the question is what Trivy reports.) Trivy's image scan works **layer by layer**: 1. It reads the image config and lists the layers by their `DiffID`. 2. It analyses each layer's files independently, running the secret rules over every plaintext file in that layer, and caches the result per layer. 3. It merges the layers into the final view. For packages, whiteouts remove what was deleted. For **secrets**, Trivy deliberately keeps findings from every layer, even when an upper layer removed the file, so the finding survives the merge. So a key that was `COPY`-ed in step 4 and removed with `rm` in step 6 is still reported, and correctly: `docker save` or any registry client exposes the step-4 layer. ## Reading the finding Each secret finding records the layer it came from: | Field | Meaning | |---|---| | `RuleID`, `Category`, `Severity`, `Title` | which rule matched and how serious it is | | `StartLine`, `EndLine`, `Match` | where in the file; the secret itself is masked with `*` | | `Layer.Digest`, `Layer.DiffID` | the layer blob that contains the file | | `Layer.CreatedBy` | the history entry that created that layer | The table report prints the target and line followed by `(added by '...')` with the first 40 characters of `CreatedBy`, or `(added in layer '...')` with a short `DiffID` when there is no history text. That is how you get from a finding back to the Dockerfile instruction. ## Why it is not a false positive, and what to do - The private key is retrievable by anyone with pull access, now and from every cached copy. - Deleting it in a later layer, or rebuilding without it, does not un-publish the image already pushed. - Treat it as an exposed credential: revoke or rotate it first, then fix the build so the file never lands in a layer, then replace the published image. The ordering of incident response and the build-time secret mechanisms are their own topics. If a finding really is noise (a test key generated for a fixture), tune it with a narrow allow rule in `trivy-secret.yaml`, not by assuming deleted files are safe. ## What Trivy can miss in the same image - **Base-image layers.** To save time, Trivy guesses which layers came from the base image by walking the image history from the newest entry backwards: trailing metadata-only entries are skipped, and the first `CMD` entry found further back is taken as the last entry of the base image. Layers at or below it have the secret analyser disabled. A secret baked into a base image is therefore not reported when you scan the application image. Scan the base image on its own, and run with `--debug` to see which layers were classed as base (`Detected base layers`). - **The image configuration.** Environment variables and history strings live in the image config, not in a layer. `--image-config-scanners secret` converts the config to JSON and scans it; it is off by default. - **Skipped paths.** Built-in allow rules (test and example paths, Markdown, vendor and system directories) and the default skip patterns apply inside images too. - **Binary files.** Secrets are matched in plaintext files and compiled Python (`.pyc`); other binaries are not decoded. ## In an interview The strong answer has three parts: Trivy is right and why (per-layer analysis, findings kept across deletions), how you locate the step (`CreatedBy`), and what you do (rotate, rebuild, re-publish), plus the coverage caveats a careful engineer would mention unprompted. Interviewers also listen for the opposite mistake: calling the hit noise because `docker run` on the image shows no such file, which only proves the merged view hides it.

  • The same key was baked into your organisation's shared base image. Will scanning the application image find it?
    Usually not. Trivy guesses the base image's layers from the history, up to the base image's `CMD`, and skips secret scanning on them. Scan the base image itself, where those layers are the image's own, and use `--debug` to see which layers were treated as base.
  • The key was passed as an ENV value, not a file. Which Trivy option finds it?
    `--image-config-scanners secret`, which scans the image configuration, including environment variables and history entries, converted to JSON. It is off by default, so a plain image scan only examines files in the layers.

saying these in an interview costs you the question

  • Trivy only scans the final merged filesystem of the image
  • A file deleted in a later layer cannot be recovered from the image
  • Deleting the file in the next build closes the exposure
  • Trivy scans every layer for secrets, base image included
  • A default image scan also checks ENV values for secrets