Your CI runs Docker builds on ephemeral runners, so every job starts with an empty builder and rebuilds everything. How would you design the build cache strategy, and what are the trade-offs of the options BuildKit gives you?

## The problem BuildKit's cache lives on the builder. Ephemeral CI runners create a new builder per job, so the layer cache is empty, cache mounts are empty, and every build is cold. On a large image this can dominate pipeline time and cost. There are two families of solution: move the cache to shared storage, or stop throwing the builder away. ## Exporting cache to shared storage BuildKit separates cache **export** (--cache-to) from cache **import** (--cache-from), and supports several backends. **Inline** (--cache-to type=inline). Cache metadata is embedded in the image config that you push anyway. Import with --cache-from type=registry,ref=<your image tag>. Zero extra artefacts and no extra storage. The limitation is decisive for modern Dockerfiles: only layers present in the final image are described, so in a multi-stage build the entire compile stage is not cached. It suits single-stage or thin-final-stage images poorly and is mostly a legacy convenience. **Registry** (--cache-to type=registry,ref=repo/app:buildcache,mode=max). Writes a dedicated cache artefact to a registry tag. mode=min stores only the final stage's layers; **mode=max stores every stage including intermediates**, which is what makes it useful. Hit rates are good, any runner anywhere can import it, and it survives the runner. Costs: an extra push per build (sometimes large), registry storage that grows without a retention policy, and pull time on import. Some registries do not support the artefact types used by cache manifests, which forces image-manifest mode. **CI-native backends**: type=gha for GitHub Actions cache, type=s3 or type=azblob for object storage, type=local for a directory the CI system caches. These trade registry coupling for CI coupling, and each carries its own size limits and eviction rules; the GitHub Actions cache in particular evicts by age and total repository quota, which silently degrades hit rate. ## Keeping the builder instead The alternative is to stop destroying the cache. A long-lived buildx builder (the docker-container driver on a persistent host, the remote driver, or the kubernetes driver running BuildKit pods with persistent volumes) keeps both the layer cache and the cache mounts warm. Jobs connect to it rather than building locally. This usually beats export-import on hit rate, because nothing is serialised, uploaded and re-downloaded, and because cache mounts (npm, Maven, Go, cargo) work at all, which no export backend provides. The costs are operational: you now run infrastructure, you must size disks and prune (docker builder prune with keep-storage), you must think about multi-tenancy and isolation between untrusted builds sharing a builder, and a busy shared builder can become a queueing bottleneck. Isolation is a genuine security consideration: a shared builder means one repository's build can, through cache poisoning or resource exhaustion, affect another's. ## Scoping and correctness Whatever the backend, cache scope matters. Cache keys are content-derived, so importing a cache from an unrelated branch is safe but often useless. A common pattern is to export per branch and import with an ordered fallback: the current branch's cache first, then the default branch's. That gives feature branches a warm start without them polluting each other. Per-platform scoping is mandatory for multi-architecture builds: an amd64 build's cache does not accelerate arm64, and mixing them into one ref wastes storage. For release builds, some teams deliberately build cold, or with --no-cache, to avoid the possibility that a poisoned or stale cache contributes to a shipped artefact, and to keep the provenance story simple. That is a real trade: reproducibility and supply-chain confidence against speed. ## Deciding Measure before engineering. Establish how much of pipeline time is build, and which stages dominate. If dependency resolution dominates, cache mounts on a persistent builder are worth far more than layer cache export. If the base and system-package layers dominate, registry cache with mode=max is easy and effective. If images are small and builds are already a minute, the simplest option that works is correct, and elaborate cache infrastructure is cost without return. Also weigh money and blast radius: registry storage and egress are not free at fleet scale, and a shared builder is a shared failure domain. The principal-level answer states the decision criteria and the exit conditions, not a single blessed configuration.