Failing tests of a generative feature save real customer text as evidence. How should that evidence be redacted, kept and read?

seniorshould knowfreq 38%

answer

The store inherits the data's sensitivity
Redact on the way in, not on read
Per field, not a blanket scrub
Strip too much and it explains nothing
A window, and a named set of readers

basics

~20 s

Apply per-field redaction in the writer, so the unredacted form never reaches the store; keep enough shape that the record still explains the failure; set and enforce a retention window; and default read access to the people investigating that feature.

solid answer

~50 s

Redact in the writer, not on read and not at the end of a window: an unredacted record is exposed for as long as it exists, and copies of it end up in messages and attachments. Use per-field rules rather than a blanket scrub. Replace identifying values with consistent placeholders so the same value still appears in the same places, keep length and structure of free text, and keep the product's own artefacts — instruction text, version, model identifier, settings — in full.

Then set a retention window and actually delete, and default the readership closed. Any rule's test: is the failure still explainable from what survives?

Once a suite captures what a failing attempt was given and what came back, its evidence store inherits the sensitivity of the data the test ran on — and it usually sits wherever build output goes, readable by anyone with pipeline access. Redact in the writer, not on read and not at the end of a window: an unredacted record is exposed for as long as it exists, and copies of it end up in messages and attachments. Use per-field rules rather than a blanket scrub. Replace identifying values with consistent placeholders so the same value still appears in the same places, keep length and structure of free text, and keep the product's own artefacts — instruction text, version, model identifier, settings — in full. Then set a retention window and actually delete, and default the readership closed. Any rule's test: is the failure still explainable from what survives?

code

pseudocode · 14 lines

pseudocode

FIELD_RULES = {
    input.customerText:  placeholder(stable_for_same_value, keep_length),
    input.accountNumber: placeholder(stable_for_same_value),
    suppliedMaterial:    same_rules_as(input),
    rawOutput:           same_rules_as(input),
    instruction*:        keep_in_full,
    modelIdentifier:     keep_in_full,
    generationSettings:  keep_in_full
}

on write_record(record):
    safe = apply(FIELD_RULES, record)      // before it reaches the store
    store.put(safe, retainUntil: now + EVIDENCE_WINDOW,
              readableBy: investigators_of(record.testCase))

go deeper

for a junior

Know that the evidence a failing test saves can contain the same customer material the test ran on, and that it therefore needs the same care as the source rather than the care a build log usually gets.

for a middle

Explain why redaction belongs in the writer rather than the reader, and give per-field rules: consistent placeholders for identifying values, shape preserved for free text, the product's own artefacts kept in full.

for a senior

Show the tension you are managing — safe enough to store, complete enough to explain the failure — plus a real retention window, a closed-by-default readership, and a case that tests the redaction path itself.

for a principal

Own the standard centrally rather than per suite: what is captured by default, how long it lives, who may read it, and the accepted residual risk of holding realistic material at all versus the cost of failures nobody can explain.

What the evidence store actually holds

The moment a suite starts capturing what a failing attempt was given and what came back, its evidence store inherits whatever was in the data. If a test case runs against real or realistic customer material, the stored record now contains that material — the input, the supporting content the product assembled, and the returned text that may quote it straight back. It sits in whatever place build output goes, and in most teams that place was designed for logs and screenshots and is readable by everyone with pipeline access.

Two facts follow, and neither is intuitive:

The evidence store takes the classification of the most sensitive thing written into it. Not the classification anyone assigned to the build system when it was set up.
Nobody is auditing it. The store fills up automatically, from a mechanism that exists so failures are explainable, run by people who are thinking about the failure rather than about what the record contains.

Redact on the way in

The decision that matters is when. Redacting when someone opens a record leaves the original in the store, which is the whole exposure; redacting at the end of a retention window leaves it there for the whole window. Apply the field rules in the writer, so the unredacted form never reaches the store at all. A record written through a redacting writer is safe wherever it later gets copied — into a chat message, into an attachment on a defect, onto somebody's laptop — and those copies are how this material actually escapes.

The rules themselves are per field rather than global. A blanket scrub of anything that looks personal reliably destroys the record; letting fields through by default reliably leaks. Enumerate the fields the capture writes and give each one a rule.

Keep it useful: the tension

Redaction and investigability pull against each other, and the honest answer is a per-field trade rather than a slogan.

Part of the record	Sensible treatment	Why
Identifying values in the input	Replace with a placeholder, consistently for the same source value	The record still shows the same value appearing in two places
Free customer text	Replace, but keep length, language and structure	The failure often depends on shape, not content
The instruction text and its version	Keep in full	It is the product's own artefact and carries no customer data
Model identifier and generation settings	Keep in full	Nothing sensitive, and useless if partial
The returned text	Apply the same field rules as the input	It commonly repeats input material verbatim

The test of a redaction rule is simple: could someone still explain the failure from what survives? A record stripped to the point where nobody can see what went wrong has not been made safe, it has been made pointless, and the team will quietly start capturing the raw form on the side.

How long, and who reads it

Set a window and enforce deletion. Failure evidence is useful for as long as the failure is live plus a margin. Storage being cheap is not a reason to keep it, and in many regimes a store of personal data with no stated limit is itself the problem.
Default the readership closed. The people investigating that feature, not everyone who can open a build. Where the material is regulated, the same access rules that govern the source data apply to the copy, because a copy is what this is.
Say all of this once, in the capture mechanism. If each team decides per suite, some will decide nothing.

Test the redaction itself

The rule that gets bypassed silently is the one nobody checks. Write a case that pushes a known identifying value through the capture path and asserts that the stored record contains the placeholder and not the original — and that the parts you rely on for investigation are still there. Without that, a new field added to the capture next quarter will simply not be covered by any rule, and nothing will say so. The check costs one case and it is the only thing standing between a well-intentioned policy and a store full of exceptions to it.

## What the evidence store actually holds The moment a suite starts capturing what a failing attempt was given and what came back, its evidence store inherits whatever was in the data. If a test case runs against real or realistic customer material, the stored record now contains that material — the input, the supporting content the product assembled, and the returned text that may quote it straight back. It sits in whatever place build output goes, and in most teams that place was designed for logs and screenshots and is readable by everyone with pipeline access. Two facts follow, and neither is intuitive: - **The evidence store takes the classification of the most sensitive thing written into it.** Not the classification anyone assigned to the build system when it was set up. - **Nobody is auditing it.** The store fills up automatically, from a mechanism that exists so failures are explainable, run by people who are thinking about the failure rather than about what the record contains. ## Redact on the way in The decision that matters is *when*. Redacting when someone opens a record leaves the original in the store, which is the whole exposure; redacting at the end of a retention window leaves it there for the whole window. **Apply the field rules in the writer, so the unredacted form never reaches the store at all.** A record written through a redacting writer is safe wherever it later gets copied — into a chat message, into an attachment on a defect, onto somebody's laptop — and those copies are how this material actually escapes. The rules themselves are per field rather than global. A blanket scrub of anything that looks personal reliably destroys the record; letting fields through by default reliably leaks. Enumerate the fields the capture writes and give each one a rule. ## Keep it useful: the tension Redaction and investigability pull against each other, and the honest answer is a per-field trade rather than a slogan. | Part of the record | Sensible treatment | Why | |---|---|---| | Identifying values in the input | Replace with a placeholder, consistently for the same source value | The record still shows the same value appearing in two places | | Free customer text | Replace, but keep length, language and structure | The failure often depends on shape, not content | | The instruction text and its version | Keep in full | It is the product's own artefact and carries no customer data | | Model identifier and generation settings | Keep in full | Nothing sensitive, and useless if partial | | The returned text | Apply the same field rules as the input | It commonly repeats input material verbatim | The test of a redaction rule is simple: **could someone still explain the failure from what survives?** A record stripped to the point where nobody can see what went wrong has not been made safe, it has been made pointless, and the team will quietly start capturing the raw form on the side. ## How long, and who reads it - **Set a window and enforce deletion.** Failure evidence is useful for as long as the failure is live plus a margin. Storage being cheap is not a reason to keep it, and in many regimes a store of personal data with no stated limit is itself the problem. - **Default the readership closed.** The people investigating that feature, not everyone who can open a build. Where the material is regulated, the same access rules that govern the source data apply to the copy, because a copy is what this is. - **Say all of this once, in the capture mechanism.** If each team decides per suite, some will decide nothing. ## Test the redaction itself The rule that gets bypassed silently is the one nobody checks. Write a case that pushes a known identifying value through the capture path and asserts that the stored record contains the placeholder and not the original — and that the parts you rely on for investigation are still there. Without that, a new field added to the capture next quarter will simply not be covered by any rule, and nothing will say so. The check costs one case and it is the only thing standing between a well-intentioned policy and a store full of exceptions to it.

How do you know the redaction rules are actually being applied?
Test them. Push a known identifying value through the capture path and assert that the stored record holds the placeholder and not the original, and that the parts you rely on to investigate survived. Otherwise a field added to the capture next quarter is covered by no rule at all and nothing says so.
Someone argues test evidence needs no controls because it is not production data.
The control follows the data, not the environment. If a case ran against real or realistic customer material, the record now holds that material, and a build store readable by everyone with pipeline access is a wider audience than the source system ever had. Where the material is regulated, the copy carries the same obligations as the original.

saying these in an interview costs you the question

Treats stored test evidence as harmless because it is not production
Redacts when a record is opened rather than when it is written
Keeps failing-attempt evidence indefinitely because storage is cheap
Strips so much that the record no longer explains the failure
Leaves evidence readable by everyone with build access

Failing tests of a generative feature save real customer text as evidence. How should that evidence be redacted, kept and read?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

What the evidence store actually holds

Redact on the way in

Keep it useful: the tension

How long, and who reads it

Test the redaction itself

▾common follow-ups2 questions

▾red flags5 flags

What the evidence store actually holds

Redact on the way in

Keep it useful: the tension

How long, and who reads it

Test the redaction itself

answer

go deeper