Release exit criteria are unmet on ship day. How do you assemble the recommendation you hand to the decision-maker?
answer
- Evidence and options, never a verdict
- Unmet by how much, not merely unmet
- Say what nobody knows, and why
- Every condition: what, who, when, and else
- The accountable owner signs the decision
basics
~10 sHand over evidence and options, not a verdict: what was covered, which criteria are unmet and by how much, what is unverified, and realistic choices. The accountable owner signs.
solid answer
~50 sThe recommendation is an evidence package, not a vote. State what was covered and to what depth, which exit criteria are unmet and by exactly how much, what remains unverified and why, and what the known failures would look like in production rather than in a defect list. Then offer a small set of real options — ship as is, ship with named mitigations such as a limited rollout or a disabled path, hold for a specific fix, or ship a reduced scope — each with its cost and what it buys. For a **conditional release**, every condition needs an owner, a deadline and a definition of done; a condition with no owner is a wish. Whoever is accountable for the business outcome signs; the test owner signs for the accuracy of the evidence. Never quietly rewrite an unmet criterion so it reads as met.
go deeper
You will not own this call, but know that unmet criteria do not automatically stop a release — they trigger a documented decision. Learn to report what you did not get to as clearly as what passed.
Be able to state an unmet criterion precisely: its original wording, the actual value, and what is unverified behind it. Practise translating an open defect into who hits it, how often, and whether there is a workaround.
Show that you can convert evidence into options with costs — ship, ship with mitigations, hold, or reduce scope — and that you keep the evidence signature separate from the business decision signature.
Own the mechanism and its credibility. Argue why criteria are changed between cycles and never during one, how conditions get owners and expiry, and what it costs the organisation when a not-yet is overruled twice in a row without consequence.
## The question behind the question When exit criteria are unmet on the planned date, the interesting failure is rarely technical. It is that the conversation collapses into two bad postures: **"we cannot sign off"**, which abdicates by refusing to convert evidence into a usable position, and **"we sign off"**, which claims a business decision that does not belong to the test function. The job is the third thing — to make the decision *decidable by the person who owns it*. ## What the package contains **1. What was covered, and how deeply.** Not a case count. Which areas were exercised, at what level, and with what kind of checking — a smoke pass over an area is not the same claim as a full scenario pass, and the recommendation should not let the two blur. **2. Which criteria are unmet, and by how much.** A criterion missed by a hair and a criterion missed by half are different facts, and rounding them into "criteria not met" throws away the only information that makes the decision tractable. Quote each unmet criterion in its original wording beside its actual value. **3. What is unverified, and why.** Every blocked or unrun area, with the reason: environment, time, or a deliberate decision. This is the section people compress and it is the section the decision actually turns on, because it is the only honest statement of what nobody knows. **4. What the failures mean in production.** Translate open defects out of tracker language into consequences: who hits this, how often, what they see, whether a workaround exists, whether it is reversible. A decision-maker cannot weigh "one open high-severity defect"; they can weigh "signers on a large account see a rejection they cannot self-correct." **5. Options with costs.** Usually four: ship as is; ship with mitigations (limited rollout, a path disabled, a rehearsed rollback); hold for named fixes with an estimate; or ship a reduced scope — each with its cost. ## The conditional release, done properly Most of these decisions end in a conditional release, and this is where the discipline either exists or does not. A condition is only real if it carries four things: **what**, stated so completion is observable; **who**, a named owner and not a team; **when**, a date; and **what happens if it is missed** — which is the part that gets left out and is the reason conditions rot. A condition without an expiry becomes permanent. The pattern: a release ships on a promise that the missing verification happens next cycle, and the next cycle is fully committed to new scope. Six weeks later it has not happened, nobody is uncomfortable, and everyone downstream treats the release as fully tested. The fix is unglamorous: conditions go on the same list as scope, with the same owners and the same review, and a missed condition is an event that someone has to acknowledge. Sign-off follows accountability. Whoever owns the business outcome signs for the decision; the test owner signs only for the accuracy and completeness of the evidence. Keeping those two signatures distinct is what stops a test function from being handed responsibility for a commercial call it cannot make, and equally stops it from hiding behind process when a decision is needed. ## The pressure that deforms the artefact Three deformations recur. **Rewriting the criterion.** The pass-rate bar moves from 95 % to 90 % on ship morning and the release is now compliant. If a bar was genuinely wrong, change it before the cycle, on the record, with a reason. Setting it against the result is a redefinition, and it teaches everyone that criteria are decorative. **The single number.** A pass-rate percentage as the whole recommendation. It hides the denominator, hides which areas the failures fall in, and hides the unverified remainder entirely. **The hedge.** Pages of caveats with no position. If you genuinely cannot recommend, say what evidence you would need and how long it would take to get. ## A worked example An 11-person team on a document e-signing flow reaches ship day with the pass-rate criterion at 78.7 % against a 95 % bar, 39 blocked cases behind one unavailable collaborator, and one open defect: an off-by-one at the signer-limit boundary, where an envelope containing exactly the maximum permitted signers is rejected. Measured against usage, that boundary is hit by roughly 0.4 % of envelopes but by three of the eleven largest accounts, and it has no workaround the signer can apply themselves. The recommendation is not "do not ship". It is: the boundary defect is a two-line fix with a one-day confirmation cycle; the 39 blocked cases sit entirely in a bulk-import area that this release does not change; the unverified remainder is therefore narrower than the raw numbers suggest. Options: hold one day for the boundary fix; or ship with the maximum-signer path capped one below the limit and a named owner to remove the cap after confirmation, expiring in nine days. Both are honest, both are actionable, and the person who owns the launch picks. ## What separates a lead here None of that content is technically hard. What is hard is holding the line that unmet is unmet, converting evidence into options rather than complaints, and establishing before the pressure arrives that conditions carry owners and expiry dates. The credibility of every future *not yet* is decided here.
- Someone proposes lowering the pass-rate bar so the release becomes compliant. How do you respond?Separate the two questions. If the bar was genuinely miscalibrated, change it deliberately — before the next cycle, on the record, with the reasoning — because a bar nobody can ever meet is a real problem. Changing it while looking at this result is not a decision but a redefinition, and it costs every future criterion its force. The release can still ship; it ships as a criterion consciously waived, which is an entirely defensible thing to record.
- How do you stop the conditions on a conditional release from quietly expiring unmet?Give each one a named individual owner, a date and a definition of done, then put them on the same backlog and the same review as delivery scope rather than in the release note nobody reopens. A missed condition should generate an event somebody has to acknowledge. Without that, the next cycle's committed scope absorbs the capacity and the release is treated downstream as fully verified within a couple of months.
A surveyor's report before a purchase: it does not tell you whether to buy, it tells you exactly what you are buying and what each defect will cost to live with.
saying these in an interview costs you the question
- Refuses to give a position, only a list of caveats
- Claims the ship decision as the test function's to make
- Reports a pass-rate percentage as the whole recommendation
- Lowers an unmet criterion on ship day so it reads as met
- Attaches conditions with no owner, date or definition of done
- Omits what was left unverified because it is uncomfortable