A registry password is passed as a container image build argument — why is it still recoverable from the published image?

## The instinct that causes this A container image build takes parameters supplied at build time, and teams reach for them to hand the build a credential — a private artifact-registry password, an internal package-feed token — because the build genuinely needs to fetch something protected. The instinct is that a build-time input is transient: the build ends, the parameter goes away. It does not. The build's *output* is an artifact, and the artifact remembers. ## What an image actually is A published image is two things: an ordered set of filesystem layers, each a content-addressed archive of the changes one build step made, and a configuration document describing how the image was assembled and how it should run. Both of them can preserve a build-time credential, by two independent routes: 1. **The recorded build history.** Image configuration commonly carries a history of the instructions that produced each layer, with build-argument values already substituted in. Reading it needs no execution and no privilege beyond the ability to pull the image. 2. **Anything the build wrote from the value.** If a step materialises the token into a package-manager configuration file, a credentials file, or a rendered properties file, those bytes become part of that layer's archive. ## Why deleting it later does nothing Layers are additive and immutable. A later step that removes the file does not reach back and rewrite the earlier archive; it records a *deletion marker* in a higher layer, which hides the path from the flattened filesystem a running container sees. The original bytes are untouched and are distributed with the image. This is the trap behind a very common false green: a filesystem scan of a *running* container comes back clean, while the credential is still sitting in the artifact one layer down. Anyone who unpacks the image offline, layer by layer, reads the file exactly as it was written. ## Who can read it The blast radius is the set of people who can pull, not the set of people who can push. That set is usually much larger than the team expects, and it grows silently: an internal service image gets mirrored to a public registry so a partner or an edge fleet can pull it, and now an anonymous internet user with no account and no exploit downloads the image and reads out the credential. The read leaves no trace you control, so you cannot later prove it did not happen. ## Get the direction of the failure right This is not a logging failure. Log masking — the CI feature that redacts registered secret values from a job's log stream — is irrelevant here, because the secret never had to appear in a log at all; it left inside the artifact. Nor is it something an inventory document catches: a software bill of materials enumerates the components an artifact is built from, not the arbitrary file contents it carries, and build provenance describes how the artifact came to be, not what is embedded in it. A credential in a layer is an *artifact content* problem, and only controls that look at artifact contents, or that keep the value out in the first place, address it. ## What to do instead - **Keep the credential out of the build entirely.** Fetch the protected dependencies in a separate step or job that holds the credential, and pass the resulting files into the image build as ordinary inputs. - **Use a build-time secret mechanism.** Modern build engines can expose a value to one step through a mount that is never committed to a layer and never recorded in the build history. That is the difference between a secret the build *uses* and a secret the build *contains*. - **Never move it to a runtime environment variable in the image configuration** as the fix. That is the same distribution problem with a wider audience: every runtime that starts the image reads the config. - **Shorten and narrow the credential.** A short-lived, single-purpose token turns a permanent published secret into an expiring one, which is not a substitute for the fix but is a real reduction in blast radius. ## If it already shipped Rotate first, then rebuild. Deleting the tag does not un-distribute the bytes — anyone who already pulled keeps them, and mirrors and local caches keep them too. After rotation, work out what that credential could reach and check for use of it; the exposure window starts at the push, not at the discovery.