skip to content

A customer audits a 14-month-old release's provenance and scan record, but CI keeps 90 days. Now what?

level: seniorimportance: should knowfreq 44%

answer

  1. prove what survived, admit what did not
  2. never backdate a record
  3. a scan today is a different statement
  4. evidence stored with the artifact digest
  5. window set by contract and support life

basics

~20 s

State precisely what you can still prove and what is gone, and never reconstruct the missing records. Then fix the real defect: evidence retention is its own control, sized by the contract and the release's support life.

solid answer

~50 s

Two answers, and the interviewer wants both. In the moment: gather what survived — the released artifact and its digest, anything stored alongside it, source history, review and ticket records — and state plainly what each proves and what is no longer provable. If you re-scan the preserved artifact, label it as a scan performed today against today's advisory data; it answers 'what do we know now', not 'what did the release gate see then'. Reconstructing or backdating the missing record is fabricating audit evidence and is the one unrecoverable mistake here. Then fix the systemic defect: evidence retention is a control in its own right, separate from the control it evidences. Evidence must live with the released artifact, bound to its digest, not in a build system whose outputs roll off on an operations schedule sized for disk rather than for the contract.

go deeper

for a junior

Be ready to say that a control you cannot evidence later is hard to defend in an audit, and that records about a release must outlive the build job that produced them.

for a middle

Explain why evidence should be stored against the released artifact's digest as signed, self-describing statements, and why a scan run today is a different claim from the one made at build time.

for a senior

Show that you would answer the audit honestly with a stated gap and a remediation date, refuse to reconstruct records, and size the retention window from contract terms and release support life.

for a principal

Own the split between telemetry that rolls off and evidence the company stands behind for years, including who funds the storage and how retention interacts with data-minimisation duties.

## Why this scenario is the classic one Companies pass audits on controls they genuinely operate and fail them on evidence they did not keep. The pipeline did scan the release. It did produce provenance. Fourteen months later none of that is retrievable, because the build system's retention was set by whoever was worried about disk, not by whoever signed the contract. The asset at stake here is audit truth — the ability to make a non-repudiable statement about something that happened in the past — and it is destroyed by an ordinary housekeeping default. ## What to do in the moment **Inventory what survived.** The released artifact almost certainly still exists, and its digest identifies it exactly. Anything that was stored *with* the artifact rather than in the build job — a signed inventory, a signed provenance statement, a release manifest — survives with it. Outside that: version-control history, code-review records, the change ticket, the release approval, and any report that was exported to a durable system at the time. **Say what each thing proves.** A signed provenance statement retrieved from artifact storage proves what build produced that digest from what source. A code-review record proves a human approved a change. A ticket proves an approval happened. None of them prove what the vulnerability scan reported on the day of the build, if that record is gone. **Say plainly what is gone.** Auditors are far more tolerant of a clean gap with a remediation plan than of a vague answer they later discover was assembled. The written response is: this is what we retained, this is what we can evidence, this specific record was not retained beyond 90 days, here is the change we are making and by when. **Offer a fresh scan, correctly labelled.** Re-scanning the preserved artifact is genuinely useful and you should offer it — but it is a different statement. Advisory databases change constantly, so a scan run today reports today's knowledge about that artifact. Presenting it as the build-time record is not a shortcut, it is a false audit statement, and if the artifact contains components with advisories published after the release, the two records would not even have matched. ## The systemic fix **Retention is a control, not a side effect.** Write it down as its own line in the control register, with an owner, a window, and a place where the evidence lives. Otherwise it is owned by nobody and set by a platform default. **Evidence belongs with the artifact.** The durable pattern is that everything you may later have to prove about a release is stored against that release's artifact digest — the inventory of what is inside it, the provenance of how it was built, the signature saying who vouched for it, and the record of what the release gate checked. Storing evidence in the build system couples the lifetime of your proof to the lifetime of a CI job, which is measured in weeks. **Prefer evidence that is self-describing and tamper-evident.** A signed statement about a specific digest can be verified by anyone, later, without trusting the storage it came from. Console output pasted into a ticket, or a screenshot of a dashboard, cannot be attributed, cannot be checked, and quietly rots. **Size the window from obligations, not from disk.** Take the longest of: the audit window in the customer contract, the supported life of the release — customers run old versions for years and will ask about the one they are running — and any regulatory retention duty that applies to your sector. Then add margin, because the request always arrives after the window you guessed. **Prune what you do not need.** Retention costs money and, for records containing personal data, creates its own liability, so keep the evidence and not the entire build log estate. The distinction is between logs, which are operational telemetry, and evidence, which is a small set of signed statements you intend to stand behind. ## How to answer this in an interview The weak answer treats it as a storage problem and promises to raise the retention setting. The strong answer separates the two halves — the honest response to this customer now, and the recognition that having a control and being able to evidence it years later are different capabilities with different owners — and it refuses, out loud, to manufacture the missing record.

  • What retention window do you set, and what do you set it against?
    The longest of three obligations: the audit window written into the customer contract, the supported life of the release — customers run old versions for years and will ask about the one they are running — and any sector regulatory duty. Add margin, and record the window as an owned control rather than a platform setting.
  • The evidence exists, but only as build-job console output. Is that acceptable?
    Weakly. Console text is mutable, unattributable and not machine-checkable, and it disappears with the job. What an auditor can act on is a signed statement bound to the artifact's digest, verifiable later without trusting the system it came from. Console output is a starting point for a remediation plan, not an evidence strategy.
  • Is keeping every build log forever the safe default?
    No. Retention costs money, and logs that contain personal data create their own liability, so indefinite retention trades one problem for two. Separate operational telemetry, which can roll off, from evidence — a small set of signed statements bound to released artifacts — and retain only the second for the long window.

It is the difference between having done the safety checks and having kept the signed checklists: the second is what an inspector can act on, and only one of them survives a filing-cabinet clear-out.

saying these in an interview costs you the question

  • Re-runs a scan today and presents it as the build-time record
  • Reconstructs plausible logs from the current pipeline configuration
  • Treats retention as a CI storage setting rather than a control
  • Keeps evidence only inside the build system that produced it
  • Offers a dashboard screenshot as durable audit evidence

context