A colleague passes a private registry token to an image build with `docker build --build-arg NPM_TOKEN=...` and argues it is safe because build arguments do not persist into the running container. Is the token recoverable from the resulting image, and what should be used instead?
answer
- docker history --no-trunc shows build-arg values
- registry config readable without pulling layers
- deleting a file in a later layer does not remove it
- RUN --mount=type=secret + --secret id=…,src=…
- already leaked = rotate the credential
basics
~20 sYes, it is recoverable. docker history --no-trunc shows the build instructions with the substituted value, and anything the token was written into (an .npmrc, a cached file) stays in that layer. Use a BuildKit secret mount instead, and rotate the token.
solid answer
~60 sIt is not safe. Two independent leaks: 1. **Image history.** Build args are substituted into the recorded instruction text, so `docker history --no-trunc <image>` (and the layer metadata in any registry the image is pushed to) exposes the value to anyone who can pull the image. Only the predefined proxy args are excluded. 2. **Layer contents.** If a `RUN` wrote the token anywhere — `~/.npmrc`, a git config, a package cache — that file lives in a layer forever. Deleting it in a later `RUN` does not remove it; the earlier layer is still shippable and readable. The correct mechanism is a build secret, mounted only for the duration of one instruction and never stored in a layer or in history: `RUN --mount=type=secret,id=npm,target=/root/.npmrc npm ci`, invoked with `docker build --secret id=npm,src=$HOME/.npmrc`. Multi-stage builds help too — do the authenticated fetch in a builder stage and copy only artefacts forward — but the mount is the actual fix. And since the token has already been in a build, rotate it.
code
bash · 8 linesdocker history --no-trunc app:1.0 | grep -i token
docker inspect app:1.0 --format '{{json .Config}}'
# even without pulling layers, the config is readable from the registry:
# skopeo inspect --config docker://registry.example.com/app:1.0
# and anything written to disk survives in its layer:
docker save app:1.0 -o app.tar && tar -tf app.targo deeper
Know the rule: never pass credentials with --build-arg; docker history shows them.
Explain both leak paths — recorded instruction text and immutable layer contents — and why deleting in a later layer does not help.
Write the BuildKit secret-mount pattern correctly (id/target/required, ssh mounts) and drive the rotation + scanning response when it has already happened.
Remove the class of problem: short-lived OIDC credentials or an authenticated artifact proxy at the CI layer, image secret scanning as a release gate, and a policy that build args are inputs and never secrets.
## Why "it does not persist into the container" is the wrong test The claim is half true: an `ARG` value is not written into the image's `Config.Env`, so `docker run ... env` will not show it. But confidentiality of an image is not about the container's environment — it is about everything a person who can `docker pull` the image can read. That includes the image *config and history* and the *content of every layer*. ## Leak 1: `docker history` The image config records each build instruction as `created_by` metadata. Build args are substituted into that text before it is recorded, so: ``` docker history --no-trunc registry.example.com/app:1.0 ``` prints lines such as `RUN /bin/sh -c npm ci --registry=https://npm.example.com/:_authToken=ghp_realtokenvalue`. The same data is available from `docker inspect`, from `crane config`/`skopeo inspect` against the registry **without pulling the layers at all**, and from anything that mirrors your images. The only exception is the set of predefined proxy args (`HTTP_PROXY`, `HTTPS_PROXY`, `FTP_PROXY`, `NO_PROXY`, `ALL_PROXY`), which Docker deliberately excludes from history — the existence of that carve-out is itself evidence that ordinary build args are recorded. BuildKit reduces incidental exposure compared with the classic builder, but you must not rely on that: the value still reaches the build environment and any file it touches. ## Leak 2: layer contents, and why deleting does not help Image layers are stacked, immutable filesystem diffs. A `RUN` that writes `/root/.npmrc` creates a layer containing that file. A later `RUN rm /root/.npmrc` creates a *new* layer containing a whiteout marker; the original layer, with the file, is still part of the image and is still pulled by every consumer. `docker save` + `tar -x` recovers it in about thirty seconds. This is the same reason "I removed the private key in the next instruction" is never an answer. Combining the write and the delete in one `RUN` (`... && rm -f ~/.npmrc`) avoids the layer leak but not the history leak, and it fails open — if the command errors midway the file may survive. ## The correct mechanism: BuildKit secret mounts BuildKit exposes a secret as a **tmpfs file mounted only for the duration of a single instruction**. It never becomes part of a layer, and it is not recorded in history: ```dockerfile # syntax=docker/dockerfile:1 FROM node:20-alpine WORKDIR /app COPY package*.json ./ RUN --mount=type=secret,id=npmrc,target=/root/.npmrc,required=true \ npm ci --omit=dev COPY . . ``` ```bash docker build --secret id=npmrc,src=$HOME/.npmrc -t app . # or from the environment: docker build --secret id=npmrc,env=NPM_TOKEN -t app . ``` Points that matter in an interview: - The default mount target is `/run/secrets/<id>`; `target=` places it where the tool expects. - `mode`, `uid`, `gid` matter when the build stage runs as a non-root user. - `required=true` fails the build if the secret was not supplied, instead of silently producing an unauthenticated (and possibly wrong) image. - Anything the command *writes* using the secret is still yours to manage — if `npm` bakes the token into a lockfile or cache that you then `COPY`, you have re-created the leak. - SSH access to private git repos has an analogous mount: `RUN --mount=type=ssh git clone ...` with `docker build --ssh default`. ## Supporting techniques - **Multi-stage builds.** Do all authenticated fetching in a builder stage and `COPY --from=build` only the produced artefacts. The builder stage is not pushed, so its layers and history do not ship — but note that they *do* exist in the local build cache, and if you push cache to a registry (`--cache-to`) they can escape. Prefer secret mounts even inside the builder stage. - **Registry-side auth instead of tokens in the build.** A CI runner authenticated to an artifact proxy, or short-lived OIDC-issued credentials, removes the long-lived secret entirely. - **Scanning.** Run a secret scanner over built images in CI so a regression is caught before publication. ## Incident response If a token has already been passed as a build arg, treat it as disclosed: **rotate it**. Deleting the image tag is not sufficient — it may have been pulled, mirrored, cached by a runner, or stored in a registry's garbage-collection window, and the layer digests are content-addressed and shareable. Then audit what else that credential could reach. ## The one-line summary Build args are *inputs*, not *secrets*: they are recorded in image metadata and readable by anyone who can pull the image. Secrets belong in a mount that exists only for one instruction, or outside the build entirely.
- Someone writes `RUN echo $TOKEN > ~/.npmrc && npm ci && rm ~/.npmrc` in a single instruction. Is that safe?Better, but still not safe. Because the write and delete are in one instruction, no layer contains the file — but the token was passed as a build arg, so it is still substituted into the recorded `created_by` text visible in `docker history` and in the registry-side image config. It also fails open: if `npm ci` errors, the shell may never reach the `rm`. Use a secret mount.
- Does using a multi-stage build and copying only artefacts forward solve the problem?It solves the shipping half: the builder stage's layers and history are not part of the final pushed image. It does not solve exposure inside the build environment — the value is still in the builder's history locally, in the build cache, and it will escape if you export cache to a registry with `--cache-to`. Treat multi-stage as defence in depth on top of secret mounts, not as the mechanism.
- The token was passed as a build arg last month and the image was pushed. What do you do now?Rotate the credential immediately and assume disclosure — the image may have been pulled, mirrored or cached, and layer digests are content-addressed so deletion of a tag proves nothing. Then audit what that credential could access during the exposure window, remove the pattern from the Dockerfile and CI, and add an image secret scan to the pipeline so a regression fails the build.
Passing a secret as a build arg is like writing the door code on the outside of the box you shipped — the box arrived, the code is gone from your hands, and everyone who handled it can read it.
saying these in an interview costs you the question
- Believing build args are private because they do not appear in the container's environment.
- Claiming a later `RUN rm` removes a secret from the image — earlier layers are immutable and still shipped.
- Assuming BuildKit alone hides build args, without using an actual secret mount.
- Storing credentials in ENV in the final image so the app can read them, instead of injecting at run time.
- Deleting the pushed tag and considering the incident closed, without rotating the credential.