skip to content

Your pipeline builds an image once and needs to promote that exact artifact from a staging repository to a production repository, adding a release tag. How do you do that without rebuilding, and how do you prove it is the same image?

level: middleimportance: should knowfreq 45%

answer

  1. build once, promote by name
  2. buildx imagetools create / crane copy / skopeo copy
  3. source from the digest, not the tag
  4. assert src digest == dst digest
  5. carry signatures + SBOM referrers along

basics

~20 s

Copy or re-tag the existing manifest instead of rebuilding: docker buildx imagetools create -t prod-repo:1.4.2 staging-repo@sha256:…, or crane/skopeo copy. Prove sameness by comparing the sha256 manifest digest before and after — an identical digest means identical bytes.

solid answer

~50 s

Rebuilding to "promote" is wrong: a rebuild is a new artifact with a new digest, so you would be shipping something you never tested. The correct move is a **manifest copy**. `docker buildx imagetools create -t registry/prod/myapp:1.4.2 registry/staging/myapp@sha256:…` creates a new tag pointing at the same manifest, server-side, without pulling layers; it also handles multi-arch indexes correctly. `crane copy` and `skopeo copy` do the same across registries and can copy the blobs when the target is a different registry. Plain `docker tag` + `docker push` also works within a single-platform image but round-trips the layers through your machine and loses the multi-platform index. Proof of identity is the digest: read the source digest, read the destination digest after the copy, assert they match. Same digest = byte-identical manifest = same config and same layers. Record that digest in the release notes, and deploy by digest so nothing between staging and prod can substitute different content.

code

bash · 9 lines
bash
SRC_REF=registry.example.com/staging/myapp
SRC_DIGEST=$(crane digest $SRC_REF:candidate)

# copy the exact manifest, never the moving tag
crane copy "$SRC_REF@$SRC_DIGEST" registry.example.com/prod/myapp:1.4.2

DST_DIGEST=$(crane digest registry.example.com/prod/myapp:1.4.2)
test "$SRC_DIGEST" = "$DST_DIGEST" || { echo "digest mismatch"; exit 1; }
echo "promoted $DST_DIGEST"

go deeper

for a junior

Know that you re-tag or copy the existing image rather than rebuilding, and that the digest tells you it is the same.

for a middle

Name the concrete tools and explain the index/multi-platform difference between registry-native copies and docker tag + push.

for a senior

Add the pipeline guarantees: source from digest, assert digest equality, copy attestations, and think about retention across the staging/prod boundary.

for a principal

Frame promotion as the trust boundary in the supply chain — one build, one digest, policy gates attached to that digest, and separate registries with separate access and retention rules.

## Why "build once, promote many" is the rule The entire point of an immutable artifact is that the thing you tested is the thing you ship. If your promotion step runs `docker build` again against the production registry, you have produced a *different* artifact: timestamps change, base images may have moved, package mirrors may serve different versions, and the digest is different. All the testing evidence you collected refers to an image that is no longer being deployed. So promotion must be a *naming* operation, never a *building* operation. ## What a promotion actually changes Remember the layering of names: repository + tag → manifest digest → config blob + layer blobs. Promotion adds a new (repository, tag) entry that points at the *same* manifest digest. Nothing about the content moves or changes. If the destination is in the same registry, no blobs need to be transferred at all — the registry can mount the existing blobs. Across registries, the blobs are copied but their digests, and therefore the manifest digest, remain identical. ## The tools, and their differences **`docker buildx imagetools create`** — the modern, registry-native option: `docker buildx imagetools create -t reg/prod/myapp:1.4.2 reg/staging/myapp@sha256:…` It talks to the registry API directly. Nothing is pulled to your Docker daemon. Crucially it is *index-aware*: if the source is a multi-platform image index, the new tag points at the same index, preserving every platform. You can also use it to assemble a fresh index from several per-platform manifests. **`crane copy` / `skopeo copy`** — standalone binaries that copy between registries without a daemon, which makes them the usual choice in CI containers and for air-gapped mirroring. `crane copy src dst` preserves digests; `crane digest ref` prints one. `skopeo copy docker://src docker://dst` is equivalent and additionally supports non-registry transports (OCI layout, containers-storage, tarballs). **`docker tag` + `docker push`** — the naive option. It requires you to have the image locally, so you pull all layers and push them again. It also collapses a multi-platform image: the local daemon holds only the manifest for the host platform, so the pushed result is single-platform. It does keep the manifest bytes identical for that one platform, so the digest survives, but the bandwidth cost and the multi-arch loss make it a poor default. ## Proving it is the same image Do not trust the tag. Assert on the digest: 1. Before: `SRC=$(crane digest reg/staging/myapp:candidate)` 2. Copy using that digest as the source (never the tag — the tag could move between step 1 and step 2). 3. After: `DST=$(crane digest reg/prod/myapp:1.4.2)` 4. `[ "$SRC" = "$DST" ]` or fail the pipeline. The fact that this check is meaningful is exactly the property digests give you: the digest is derived from the manifest bytes, so equality of digests is equality of content. If you also verify a signature attached to the digest, you additionally know *who* produced it — but that is a separate concern. One subtlety: attestations, SBOMs and signatures are usually stored as *separate* manifests that reference the image digest (an OCI referrers relationship). A naive copy of just the image manifest leaves them behind in the staging registry. `crane copy` and `skopeo copy` have flags to carry referrers/signatures along; check that your promotion copies them too, or your production image will fail policy checks that were passing in staging. ## A second subtlety: annotations and "same image, different digest" Some tools rewrite the manifest during copy — for example adding annotations or converting a media type between Docker v2 and OCI formats. That produces a *semantically* identical image with a *different* digest, which breaks digest-equality assertions and any signature bound to the old digest. If your promotion is producing a new digest, look for a format conversion or annotation rewrite in the tool's flags before concluding that a rebuild happened. ## Retention and the promotion boundary Once prod points at a digest, the staging repository's retention policy must not garbage-collect it out from under production if they share a blob store, and if they are separate registries the prod copy is what matters. This is a good reason to physically copy into a production registry rather than merely re-tagging inside the staging one: promotion then also means "this artifact now lives under production's retention and access rules". ## The interview-sized answer Promotion is re-pointing names at an existing manifest. Use `buildx imagetools create`, `crane copy`, or `skopeo copy`; source the copy from the digest, not the tag; assert source and destination digests are equal; and remember to bring signatures and attestations along.

  • Why is `docker tag` + `docker push` a poor promotion mechanism for a multi-platform image?
    The local daemon stores only the manifest for the host platform when it pulls a multi-platform image, so re-tagging and pushing publishes a single-platform image under the new tag. Nodes on other architectures then fail to pull. Registry-native tools such as `buildx imagetools create`, `crane` or `skopeo` operate on the index itself and keep every platform.
  • After a promotion the destination digest differs from the source digest even though nothing was rebuilt. What are the likely causes?
    Something rewrote the manifest bytes: a media-type conversion between Docker schema 2 and OCI, added or stripped annotations, or a tool that recompressed layers. Any of these change the manifest and therefore the digest, and will also invalidate signatures bound to the original digest. Disable the conversion or annotation flags so the copy is byte-preserving.

Promotion is adding another label to the same warehouse pallet, not manufacturing a second pallet that looks like it.

saying these in an interview costs you the question

  • Proposing a rebuild against the prod registry as the promotion step
  • Copying from the moving tag instead of the digest, so a concurrent push can substitute content
  • Assuming a re-tag transfers layers and is therefore slow, when a same-registry re-tag moves no blobs
  • Forgetting that signatures, SBOMs and attestations are separate manifests that must be copied too
  • Treating equal tags as proof of equal content instead of comparing digests

context