skip to content

Why do mature infrastructure-as-code pipelines execute a saved preview artifact from the review step instead of recomputing the diff at apply time, and what does that approach cost?

level: seniorimportance: nice to knowfreq 42%

answer

  1. review what will run, not what might
  2. the approved diff is a document
  3. recompute means unreviewed
  4. the artifact ages and it leaks
  5. fail closed when it is missing

basics

~20 s

Saving the reviewed diff and executing exactly that artifact guarantees the change a human approved is the change that runs. Recomputing at apply time silently applies a diff nobody saw. The costs are artifact plumbing between jobs, a sensitive file to protect, staleness failures, and a hard version pin.

solid answer

~50 s

The property you want is that the reviewed diff is the applied diff. If the apply job recomputes the change, it can legitimately produce something different — the branch moved, a variable changed, a lookup resolved differently, someone else applied in between — and it will do so without telling anyone that the approval no longer matches. Saving the computed diff as an artifact and executing only that closes the loop: the approval is attached to a specific document, and the tool will refuse the artifact if the recorded baseline has moved on. The costs are real. The artifact must travel between pipeline jobs, it contains resolved values including secrets in readable form, it is bound to the exact tool and provider versions that produced it, and it goes stale, so you need a clean re-plan path when it is rejected.

go deeper

for a junior

Know that the change you approve should be the change that runs, and that a pipeline which runs the tool a second time at apply is computing a fresh set of actions.

for a middle

Explain the concrete ways two computations diverge — moved branch, mutable variables, image lookups, differing versions — and why the tool refuses a saved artifact when the recorded baseline has moved.

for a senior

Show that you have built this: artifact passed between jobs, apply that fails closed without it, pinned versions on both jobs, restricted storage and short retention because the artifact carries resolved secrets.

for a principal

Decide whether approval integrity is worth its friction for each class of change, and set the standard estate-wide: which pipelines must ship an artifact, which may compare-and-fail instead, and how a rejected artifact gets re-approved without people routing around the gate.

## The property being protected A review gate is only worth its cost if the thing reviewed is the thing executed. Call it *approval integrity*. It is easy to lose without noticing, because the common pipeline shape loses it by default: a job on the pull request computes and posts the diff, a human approves, and a job on merge runs the tool again to apply. The second job computes its own diff. Nobody looked at that one. Most of the time the two diffs are identical, which is exactly why the gap survives in real pipelines for years. The failure is rare and severe. ## What can differ between the two computations - The merge commit is not the commit that was reviewed — other pull requests landed. - Input variables come from a mutable source: a variable file on the default branch, a secret store, a parameter that another team edits. - A lookup resolves differently: "the newest image matching this pattern" returns a new image that was published this morning. - The apply job runs a different tool or integration version than the preview job, and a version change can alter what a diff contains. - Another apply landed in between, so the second diff is computed against a different reality. ## What the artifact buys you The saved diff is a frozen document: the exact set of actions, computed against a specific baseline of the tool's record. Executing it means the tool performs those actions and no others. If the record moved on since — because another run wrote to it — the tool detects the mismatch and refuses. That refusal is a feature: it converts a silent divergence into a stop, and the correct response is to re-plan and re-review rather than to force it. Some platforms achieve the same thing differently, keeping the proposed change server-side as a stored object with an identifier: review references the object, execution references the same object, and no file travels between jobs. The tradeoff is whether you would rather have a file in your CI system to protect, or trust the platform to hold it. ## What it costs **Plumbing.** The artifact has to move from the job that produced it to the job that consumes it, which means artifact upload and download, retention settings, and a pipeline that fails closed when the artifact is missing rather than quietly falling back to recomputing. That fallback is the most common way teams think they have approval integrity and do not. **Secrecy.** A computed diff contains resolved values — the actual strings being written, including credentials, connection details and keys that appear as resource attributes. Some tools mask values marked sensitive, but the underlying artifact generally holds the real data. Treat it with the same care as the record itself: restricted access, short retention, never posted verbatim into a public pull-request comment. A common split is a redacted human-readable summary in the review, and the full artifact stored where only the pipeline can read it. **Version binding.** The artifact was produced by a specific tool version and specific provider integrations. The apply job must use the same ones, which in practice means pinned versions and a dependency lock committed to the repository, not floating constraints resolved at job start. **Staleness.** Any real pipeline will sometimes reject an artifact because the world moved. You need a clear, low-friction path for re-planning and re-approving, or people will start looking for a way around the gate — which is worse than not having it. ## What good looks like - The apply job takes the artifact as its only input and has no path that recomputes a diff. - The job fails when the artifact is absent, unreadable, or older than a threshold. - Pipeline concurrency is one per environment, so two runs cannot both hold approved artifacts against the same resources. - The artifact lives in restricted storage with retention measured in hours or days, not months. - The review surface shows a readable summary; the artifact itself is machine input. ## Interview framing Lead with the property — reviewed diff equals applied diff — and describe the default pipeline shape that quietly breaks it. Then be honest about the costs, especially that the artifact is sensitive and that a silent fallback to re-planning gives you the appearance of the control without the control.

  • If you cannot pass an artifact between pipeline jobs, what is the next best control?
    Recompute the diff at apply time and compare it against the approved one, failing the run on any difference. It gives you detection instead of prevention: the applied diff is still one nobody read, but a divergence stops the pipeline rather than proceeding. Pair it with strict version pinning and immutable inputs, since floating versions and mutable variable sources are the usual reasons the two diffs differ at all.
  • Why is a saved preview artifact treated as a sensitive file?
    Because it contains the resolved attribute values the tool is about to write — passwords, tokens, connection strings and keys appear in readable form even when the review output masks them. It is also a complete description of an intended production change, which is useful reconnaissance on its own. Restrict access to the pipeline, keep retention short, and never attach the raw artifact to a publicly visible review comment.
  • What is the failure mode of a pipeline that falls back to recomputing when the artifact is missing?
    You get the appearance of approval integrity with none of the guarantee, and you will not notice, because the fallback path succeeds. Any artifact expiry, storage hiccup or misconfigured job silently converts an approved change into an unreviewed one. The apply job should treat a missing artifact as a hard error that requires a fresh preview and a fresh approval.

saying these in an interview costs you the question

  • Treats recomputing at apply time as equivalent to applying the reviewed diff
  • Publishes the raw preview artifact where anyone can read it
  • Assumes a saved artifact stays valid for as long as you like
  • Lets the apply job silently re-plan when the artifact is missing
  • Ignores that the artifact is bound to specific tool and integration versions

context