Your deployment manifests reference every container image by tag. A colleague proposes pinning all of them to sha256 digests instead. How would you decide, and what machinery does digest pinning require to be sustainable?
answer
- tags drift silently; digests freeze silently
- pin: prod manifests, FROM, CI tools
- bot bumps + mirroring + retention exemption
- admission rejects tag-only refs
- integrity != authenticity; pin index digest for multi-arch
basics
~20 sPinning buys reproducible, auditable, exactly rollback-able deployments and closes tag-mutation risk. It costs you automatic patching, so it only works with automated digest bumps, registry retention that never garbage-collects pinned manifests, and a clear rule about which references may still be tags.
solid answer
~50 sDecide by asking what a reference must guarantee at each stage. **Pin where identity matters**: production deployment manifests, base images in `FROM`, anything a scan result, SBOM or signature is recorded against. Tag references make those records meaningless, because the tag can move after the scan. **Leave tags where mutability is the point**: local development, and streams you deliberately want to follow. The machinery that makes pinning sustainable: - **Automated bumps** — a bot that resolves tag→digest and raises a change, so patches still flow, just through review. - **Registry retention** — pinned manifests must survive garbage collection of untagged content; keep a tag alongside or exclude pinned digests from cleanup. - **Admission policy** — reject tag-only references in production namespaces, otherwise the discipline erodes. - **Readability** — keep `image:tag@sha256:…` so humans still see the version. Pinning gives integrity, not authenticity; pair it with signature and provenance verification.
code
dockerfile · 4 linesFROM eclipse-temurin:25-jre@sha256:6c3f1e2b9a4d5f7c8b0e1a2d3f4c5b6a7d8e9f0a1b2c3d4e5f60718293a4b5c6
WORKDIR /app
COPY build/libs/app.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]go deeper
Say that digests make deployments reproducible and tags can change, and that pinning means updates must be done deliberately.
Distinguish where each belongs — production and FROM lines pinned, development on tags — and mention automated bumps.
Cover the operational machinery: bots, mirroring, retention exemptions, admission enforcement, and multi-arch index digests.
Present it as a written policy per reference class with owners and cadence, argue that pinning relocates rather than removes risk, and connect it to signature/provenance verification anchored on the digest.
## What the decision is actually about Pinning by digest is a trade between two failure modes. Tags fail by silently changing what runs. Digests fail by silently *not* changing — a patched base image is available and you are still running last quarter's. There is no universally right answer, so the decision is a policy, made per reference class. ## What pinning buys **Reproducibility.** A rollout is a function of the manifest. With tags, the same manifest applied twice can produce different running code, which destroys the premise that a stored revision describes a deployed state. **Exact rollback.** "Go back to what ran on Tuesday" is only well-defined if Tuesday's reference names content. **Meaningful security records.** A vulnerability scan, an SBOM or a signature is a statement about specific bytes. Attaching it to a tag makes it a statement about a pointer, and the pointer moves. **Defence against registry-side substitution.** A compromised or mis-permissioned registry account can repoint a tag; it cannot change what a digest resolves to. **Consistency across nodes.** With tags, node caches diverge — some nodes hold the old content, newly scheduled ones pull the new. Digest references make that impossible by construction. ## What pinning costs **Patches stop flowing by themselves.** This is the real objection, and it is legitimate: a pinned base image accumulates unpatched CVEs until someone acts. **Human legibility drops.** `sha256:9f86d0…` tells a reviewer nothing. Mitigate by writing `repo:1.27@sha256:…`, where the tag is documentation and the digest is what resolves. **Churn.** Every patch becomes a diff, a review and a deploy. That is the point — the work becomes visible — but it must be automated or it will be skipped. **Retention risk.** Registries garbage-collect manifests no tag references. Pin to a digest whose tag later moves and gets cleaned, and the pull fails at the worst moment. This bites hardest with aggressive retention policies and with upstream public registries you do not control — mirroring into a registry you own is the durable answer. ## The machinery 1. **Automated digest bumps.** A dependency bot resolves the tracked tag to its current digest and opens a change. Patch cadence becomes a review policy rather than an accident of when a node happened to pull. 2. **Mirroring and retention.** Copy third-party images into your own registry, preserving digests, and exempt in-use digests from cleanup. Deriving "in use" from what your deployment manifests reference makes this mechanical. 3. **Admission enforcement.** A policy that rejects tag-only images in production namespaces. Without enforcement, one hotfix reintroduces a tag and the invariant is gone. 4. **Provenance on top.** Digests give integrity — you got the bytes that hash names. They say nothing about who built them. Signature and provenance verification, anchored on the digest, supplies authenticity. Note the ordering: verification is over a digest, so pinning is a prerequisite for meaningful verification, not an alternative to it. 5. **Multi-arch correctness.** In a mixed fleet, pin the *index* digest, not a per-platform manifest digest, or one architecture will stop resolving. ## Where the boundary usually lands A defensible default: - Production workload references: digest-pinned, enforced at admission. - Base images in `FROM`: digest-pinned, bot-bumped, because build inputs drifting is the least visible failure of all. - CI tool images: pinned, since a moving tool image makes build failures unreproducible. - Local development and ephemeral preview environments: tags, for convenience. Express it as a written rule with an owner for the bump cadence, and revisit when the fleet's patch SLA changes. ## How to argue it in an interview Do not answer "always pin". Show that you know pinning relocates risk rather than removing it: it converts silent drift into visible, reviewable change, and it is only an improvement if the review actually happens on a cadence. If the organisation has no bot and no patch cadence, wholesale pinning can genuinely make security worse, and saying so is the senior-level observation.
- A team pins digests but has no bot. Six months later their images are full of CVEs. Was pinning the wrong call?Pinning was not wrong, but it was incomplete. It converts drift into an explicit change that someone must make, so without automation and an owner the change never happens. The fix is to add automated bumps and a patch cadence, not to revert to tags — reverting only restores drift that happens to sometimes carry patches.
- How does digest pinning interact with a pull-through cache or an air-gapped mirror?It makes them safer, because a digest reference cannot be satisfied by different content in the mirror — the client verifies the hash. The requirement is that the mirror preserves bytes exactly, since any re-compression or manifest re-serialisation changes the digest and the pinned reference stops resolving. Purpose-built copy tools preserve them.
- Should the image tag be removed once a digest is present?No. Keeping `repo:1.27@sha256:…` costs nothing at resolution time — the digest wins — and gives reviewers the version context they need to judge a diff. It also leaves a tag referencing the manifest, which helps against registry garbage collection.
saying these in an interview costs you the question
- Answering "always pin everything" with no mention of how patches then reach production
- Claiming a digest makes the image trusted or verified, conflating integrity with authenticity
- Ignoring registry garbage collection, so pinned-but-untagged manifests can vanish
- Pinning a per-platform manifest digest in a mixed amd64/arm64 fleet instead of the index digest
- Assuming policy alone holds without admission enforcement, when one hotfix reverts to a tag