Why is build provenance from a long-lived, reused build machine worth less than from a single-use builder?
answer
- who produced it, not just what it says
- state survives between jobs
- an earlier job can poison a later one
- residue never appears in the statement
- provisioned solely for this build, then destroyed
basics
~20 sA reused machine keeps state from earlier builds - caches, installed tools, leftover credentials, running processes - so an earlier job could have altered this build or what was recorded about it. A single-use environment removes that whole class of doubt.
solid answer
~40 sProvenance is a signed record saying 'this builder ran this build definition and produced this artifact'. That claim is only as strong as the platform's ability to stop one build from influencing another. On a machine that is reused, an earlier job can leave a poisoned entry in a dependency cache, a wrapper script earlier on `PATH`, a background process, or credentials on disk - and none of that residue appears anywhere in the provenance. So every field in the statement can verify while the artifact is not what the recorded definition would honestly have produced. That is why the hardened rung of the SLSA build track asks for an environment provisioned solely for one build and discarded afterwards, rather than only for a signed document. Ephemerality is what makes the record describe reality.
go deeper
Be ready to say what provenance claims and why the machine matters at all. Know the phrase 'ephemeral and isolated' and that it means an environment created for one build and thrown away afterwards.
Explain the concrete carriers of state between builds - caches, installed tools, background processes, credentials on disk - and why none of them show up in the provenance, so the record can verify while the artifact is wrong.
Show judgment about a real fleet: which builds actually get fresh environments, what your cleanup misses, whether release builds are separated from everything else, and what you would stop claiming until they are.
Own the cost argument. Fresh capacity per build is a platform spend, so be able to say what you buy by moving release builds to single-use environments while leaving pull-request feedback on warm capacity, and how you keep the claim honest.
## What a provenance statement actually claims Build provenance is a signed record produced by a build platform. It identifies an artifact by its digest and says how it came to be: which builder ran, from which source revision, through which entry point, with which parameters. A consumer verifies the signature, checks the builder identity against policy, and checks that the recorded source and entry point are the ones it expects. Notice what is *not* in that list. The statement describes the **build definition**, not the **build environment's history**. Nothing in a well-formed provenance file tells you whether the machine that executed it had just finished running somebody else's code. ## What 'ephemeral and isolated' means The SLSA build track's hardened rung asks the platform to guarantee that each build ran in an environment provisioned solely for that build, free of influence from other builds - including previous ones - and not reused afterwards. It is a property the *platform* enforces for every tenant, not something an individual pipeline can grant itself. ## What actually crosses a reused machine - **Dependency and tool caches.** A later build silently consumes a cache entry an earlier build wrote. The lockfile still matches; the bytes on disk do not. - **Installed toolchains and anything on `PATH`.** A wrapper planted where the compiler or package manager is looked up intercepts every subsequent build. - **Resident processes.** A daemon left running can watch the filesystem, scrape environment variables from later jobs, or patch an output after the tests have passed. - **Credentials on disk.** Cloud CLI credential caches, registry logins and helper config files written outside the workspace. - **Container images and layer caches**, and the state of the build agent process itself. The common thread: all of it is invisible to the provenance. The statement records intent; the residue changes the outcome. ## Why 'we clean the workspace' is not equivalent Workspace cleanup is a deny-list that humans maintain: someone has to enumerate everything a hostile job might leave and remove it, forever, as the toolchain changes. Ephemerality is the same guarantee by construction - whatever an earlier build touched is gone because the environment is gone. The asymmetry matters: the attacker picks where to hide, and you have to guess correctly every time. ## Trust boundary, not team boundary A common objection is that all the builds on the shared machine belong to one trusted team. Trust in colleagues is not isolation. Every one of those builds executes third-party dependencies, plugins and test frameworks that nobody reviewed. One compromised transitive dependency in the least important job gets a foothold that outlives that job and reaches the release build. The platform's security level is set by the worst code that runs on it, not by the best-behaved pipeline. ## Where the signing key has to live The same reasoning extends to the key the platform signs provenance with. If user-defined build steps can read it, any build can mint a statement naming any builder and any artifact, and verification downstream becomes theatre. Hardened platforms keep that key in a component the build cannot address, and sign after the build's steps have finished. ## What ephemerality does not buy It does not stop a malicious commit, a vulnerable or backdoored dependency, or a compromised platform operator. It closes exactly one class: builds influencing other builds through the environment they share. Being precise about that scope is what separates a candidate who has read the requirement from one who has recited it. ## Practical shape Provision a fresh container or virtual machine per build from a known image, keep no writable state that survives the run, and perform the signing outside the build's reach. Where fresh capacity for everything is genuinely too expensive, the honest compromise is to separate: release builds on single-use environments, fast feedback builds on warm capacity - and claim only what the release path actually satisfies.
- Does wiping the workspace between builds give you the same property as an ephemeral environment?No. Cleaning the checkout directory removes the most visible contamination, but the host still carries tool caches, installed packages, running daemons, environment state, container images and anything an earlier job planted outside the workspace. The property being asked for is an environment provisioned solely for this build and discarded after it, so residue is gone by construction rather than by a cleanup script somebody has to keep exhaustive.
- If every build on that shared machine belongs to one trusted team, does reuse still matter?Yes. Trust in colleagues is not isolation. Every build pulls dependencies, plugins and test frameworks whose code nobody reviewed, so one compromised package in the least important job gets a foothold that outlives it and reaches the release build. The guarantee is about what the platform enforces, not about who owns the pipelines.
- Where does the provenance signing key have to live for any of this to mean something?Outside the build. If build steps can read the key the platform signs with, any build on the platform can mint provenance naming any builder and any artifact, and the chain collapses back to self-assertion. A hardened platform holds that key in a component the build cannot address and signs once the build's own steps are finished.
A signed lab certificate is only as good as the lab. If the bench was never cleaned between samples, the paperwork is perfectly honest about the procedure and still tells you nothing reliable about your sample.
saying these in an interview costs you the question
- Thinks signing the provenance compensates for a dirty build machine
- Treats machine reuse as only a flakiness and caching problem
- Assumes deleting the workspace equals an ephemeral environment
- Calls builds isolated because trusted teams own them
- Claims ephemerality also stops malicious commits or bad dependencies