What does a shared remote build cache give a CI fleet that a per-runner local cache cannot, and what must be true of a build for a cross-machine cache hit to be safe?

## What the remote cache actually changes A local build cache lives on one machine's disk. On a fleet of ephemeral runners — the norm for isolation reasons — that machine is destroyed after every job, so the cache's hit rate is close to zero by construction. Even on persistent runners, each machine warms independently, so N runners means N cold starts and no sharing between them. A remote build cache inverts the arrangement. The build tool computes, for each unit of work it is about to do, a hash over that unit's **declared inputs**: the source files it reads, the toolchain version, the compiler flags, the relevant environment variables, and the hashes of the outputs it depends on. That hash is the key. If the remote store has an entry, the tool downloads the output and skips the work. If not, it does the work and uploads the result. The consequences are the interesting part: - Reuse survives the runner. A fresh ephemeral machine can be as fast as a warm one. - Reuse crosses branches. A pull request that touches one module reuses everything the default branch already built. - Reuse crosses people. Developers can pull CI's outputs instead of building locally, if you let them read. - Reuse is per task, not per job. Nothing needs to match wholesale; the unchanged 90% of the graph is skipped. Build tools with this capability include Gradle's build cache and Bazel's remote cache; for C and C++ compilation, `ccache` and `sccache` occupy the same niche at a lower level of granularity. ## The precondition: determinism against declared inputs The cache promises that the downloaded output equals what the machine would have produced. That promise holds only if the task is a pure function of its declared inputs. Every common violation is a source of silently wrong builds: - **Absolute paths baked into outputs.** Debug info, generated manifests, virtual-environment shebangs and compiled bytecode frequently embed the build directory. An entry produced under `/home/runner-07/work` is then not valid on a machine using a different path. - **Timestamps and hostnames.** A build that stamps the current time or the builder's hostname into an artifact produces a different output for identical inputs, which both breaks verification and pollutes the store. - **Undeclared environment.** A variable the task reads but the key does not include: locale, timezone, a feature flag, a proxy setting. - **Network access during the task.** Anything fetched at build time can change between the entry being written and being reused, and the fetch is invisible to the key. - **Nondeterministic ordering.** Map iteration order in a code generator, parallel output concatenation, archive member ordering without a sorted-entry setting. So the honest answer to "can we turn on the remote cache?" is usually "after we make the build hermetic," not "yes, tomorrow." Tools help — several offer a mode that runs a task twice and compares outputs — and a periodic cache-disabled build compared byte-for-byte against a cached one is the low-tech version of the same check. ## Trust: writes are an execution path A cache hit means executing or shipping bytes someone else produced. That makes write access a security boundary, not an operational convenience. The standard arrangement is asymmetric: trusted CI jobs on trusted refs write; developers and pull-request builds read only. Authentication is required in both directions, and it is worth remembering that a wrong entry does not announce itself — it is discovered when something behaves oddly in production, at which point rotating the cache namespace and rebuilding is the response. ## When a hit is not a win A remote hit costs a network round trip and a download. It only pays when fetching beats recomputing. Large outputs over a constrained link — or a runner fleet in a different region from the cache — can genuinely be slower than building locally, especially for fast tasks. Most implementations expose thresholds so small or cheap tasks are not cached at all. Also budget for the store itself: total size, eviction policy, and egress cost if the cache and the runners sit in different networks. ## What to say in an interview Lead with the key insight — the cache is content-addressed on declared inputs, so correctness is a property of the *build*, not of the cache — then name the concrete hermeticity violations, then the trust asymmetry, then the network-cost caveat. Candidates who describe it purely as "a faster cache in the cloud" miss that adopting one is mostly a build-determinism project.