Incremental Builds & Build Caching

## A build task as a pure function Every build task—compiling a package, running a linter, executing a test suite—can be thought of as a **pure function**: given the same inputs, it always produces the same output. A build system that wants to avoid redundant work exploits this by first computing a **cache key** for a task before running it, then checking whether a cached result already exists for that exact key. - **Hit** — if a matching result is found, that's a cache hit: the system fetches the previously stored output (compiled artifacts, test logs, exit code) and hands it back to whoever asked, without ever spawning the compiler or test runner. - **Miss** — if no match exists, that's a cache miss, and the task actually executes; its output is then stored under that key so future requests for the same inputs can hit the cache. ## What goes into the key The inputs that go into the key typically include: - the content of every source file the task reads; - the versions and content hashes of its dependencies; - the command-line flags and environment variables that affect its behavior; - the toolchain version itself (compiler binary, linter version). Because the key is derived from file content (usually via a hash like `SHA-256`) rather than file modification timestamps, this technique is called **content-addressed caching**—two files with identical bytes produce the same hash regardless of when they were last touched, which makes the cache immune to things like a `git clone` resetting timestamps or a CI checkout re-writing every file's mtime. ## Why it matters at monorepo scale Why this matters becomes obvious at monorepo scale. A repository with hundreds of packages might have a full build/test/lint pipeline that takes 30–60 minutes from a cold start. Most day-to-day changes touch only a handful of those packages. Without caching, every CI run and even every local build command re-executes every task for every package, because the build tool has no way to know that most of the graph is unchanged. With content-addressed caching, only the tasks whose actual inputs changed miss the cache; everything downstream that transitively depends on unchanged inputs also gets a cache hit, because its own inputs (which include the hashes of its upstream dependencies' outputs) are unchanged too. The practical effect is that a single-line change to one leaf package can produce a build that finishes in seconds instead of tens of minutes, because only that package and its direct consumers actually re-run. ## The trade-off The trade-off is bookkeeping overhead and correctness risk. Computing hashes for every input of every task costs CPU time and I/O, though this is normally far cheaper than re-running the task itself. The bigger risk is **under-specification**: if the cache key omits an input that actually affects the output — a stray environment variable, an implicit read of a config file not declared as a dependency, a non-deterministic code generator — the build system will confidently return a cache hit for a task whose real behavior has changed. This produces a 'stale' or 'wrong' result that looks identical to a correct one until someone notices the shipped artifact doesn't match the source. Teams debug this by identifying the untracked input and adding it to the task's declared inputs, or by making the task's declared inputs stricter (sandboxing filesystem/network access) so nothing can sneak in unnoticed. ## Failure modes in production In production this shows up in a few recognizable ways. 1. **First**, 'why didn't my change take effect' bugs, where a developer changes a file that (unknown to the build graph) is read by a task but not declared as an input, so the task keeps serving a stale cached build. 2. **Second**, flaky CI where a task is legitimately non-deterministic (e.g. embeds a timestamp or random UUID in its output) and so never gets a stable cache key, defeating caching for that node and everything downstream of it. 3. **Third**, security and trust concerns for shared/remote caches, where a malicious or buggy CI job could poison the shared cache with an incorrect artifact that then gets served to every other developer and pipeline that computes the same key. ## Where it shows up A concrete example: `Bazel` popularized this model for large monorepos at Google, using strict sandboxing to guarantee hermeticity so cache keys can be trusted; `Nx` and `Turborepo` bring a lighter-weight version of the same idea to JS/TS monorepos, hashing each project's source files plus its dependency graph to decide whether to replay a cached build versus actually invoking `webpack`, `tsc`, or `jest`. In all three, the fundamental unit of reasoning is the same: a task node in the dependency graph, a hash of everything that could affect its output, and a lookup against a store of previous results keyed by that hash.