What is the difference between confirmation testing and regression testing after a defect fix?
answer
- Two different questions after one fix
- One activity has a defect report
- The other has previously passing behaviour
- Re-testing is the other name
- Green fix, broken neighbour
basics
~20 sConfirmation testing re-runs the case that exposed the defect, to prove the fix works. Regression testing re-runs other cases around the change, to prove the fix broke nothing that previously passed. Both are needed after a fix.
solid answer
~40 sConfirmation testing, also called re-testing, re-executes the exact steps from the defect report on a build that contains the fix and checks the behaviour the report says is correct. Regression testing asks a different question — did this change break behaviour that was already working — so it re-runs other cases around the changed area, and its yardstick is the previously accepted behaviour, not the defect report. A green confirmation run says nothing about collateral damage, which is why both run after a fix. Confirmation exists only after a fix; regression testing applies after **any** change, including a feature, a dependency update or a configuration edit. Once the fix is confirmed, the case that exposed the defect is normally added to the regression pack, so the same defect is covered from then on.
code
pseudocode · 13 lineson defect_fix_landed(defect, change):
# 1. Confirmation: the defect report is the yardstick
result = run(defect.reproduction_steps, build = change.build)
assert result == defect.expected_behaviour
# 2. Regression: previously passing behaviour is the yardstick
neighbours = cases_touching(change.components)
+ cases_asserting(change.downstream_artefacts)
for case in neighbours:
assert run(case, build = change.build) == case.last_passing_result
# 3. The pack grows by one
regression_pack.add(case_from(defect.reproduction_steps))go deeper
Be ready to define both terms in one sentence each and say which yardstick decides pass or fail. Know that re-testing is another name for confirmation testing, and that a fix needs both activities before it is called done.
Explain how you scope the regression around a fix rather than running everything: the edited component, its callers, shared data, and the records the change writes. Expect to be asked why a green confirmation run proves nothing about collateral damage.
Show the judgement about what a confirmation pass is worth: the right build, comparable data and configuration, the symptom rather than the fix as the expectation, and every reproduction path in the report. Be able to describe a real fix that repaired the reported behaviour and damaged a neighbour.
Own the policy: what a fix must carry before it is accepted, whether the escaped case is added to the pack, which regression scope is mandatory for a fix versus deferred to a slower run, and how you keep that from becoming a ritual nobody reads.
## Two questions, two activities **Confirmation testing** — also called re-testing — answers one question: *is the reported defect gone?* You re-execute the steps from the defect report, on a build that actually contains the fix, under conditions comparable to the ones where the defect was seen, and compare the result against the behaviour the report says is correct. The yardstick that decides pass or fail here is the defect report itself. **Regression testing** answers a different question: *did this change break behaviour that was already working?* You re-execute other cases — around the changed area, and across paths that share code, data or state with it — and compare against behaviour that was passing before the change. The yardstick is the previously accepted behaviour, not the defect report. Neither activity produces the other's evidence. That is the whole point of the distinction, and it is why an interviewer asks it: candidates who blur the two tend to ship fixes that repair one symptom and cause another. ## A worked example A fare calculator for a public-transport network applies a **daily cap**: however many journeys a rider makes in a day, the day is charged at most the cap. A defect report says a rider who made 9 journeys in one day was charged 14.60 instead of the 9.80 cap. *Confirmation.* Take a build whose identifier contains the fix, load the same tariff table version, replay the 9-journey itinerary from the report, and assert the day totals 9.80. It does. Confirmation passes. *Regression around the change.* The fix moved the capping adjustment inside the per-journey loop instead of applying it once per day. The **total** is still correct, because each pass recomputes the same final figure. But every capped day now writes one adjustment entry per journey into the refund ledger — nine entries where there should be one. That is a duplicated side effect, and the confirmation case can never see it, because the confirmation case asserts the fare total and the fare total is right. Only a case asserting the ledger entry count for a capped day catches it. That is the usual shape of regression damage: the fixed behaviour is fine, and the thing standing next to it is not. ## Scoping the regression around a fix "Regression testing after a fix" does not mean "run everything". A defensible scope follows the change: the component that was edited, its callers and callees, cases sharing the same data structures, and — as the example shows — the downstream artefacts the change writes to. A time-boxed subset immediately, with the full pack on its slower schedule, is the normal arrangement. ## How the pack grows from this A defect escaped because nothing covered it, or because what covered it asserted the wrong thing. So after the fix is confirmed, the case that exposed the defect is written up and added to the regression pack. That is the honest way a pack grows: it accretes cases from real escapes rather than from a wish list. If the defect was found by exploratory work or reported from production, write the case from the report before the defect is closed, while the reproduction steps are still known to work. ## Common failure modes - **Confirming on the wrong build.** Check that the build under test actually contains the fix before believing a pass. - **Confirming with different data or configuration.** A pass that depends on the data on your machine proves nothing about the reported conditions. - **Writing the confirmation case against the fix instead of the symptom.** Then it passes by construction and tells you only that the code does what it does. - **Accepting "the developer says it works"** in place of an executed check. - **Assuming the pack already covers the fixed behaviour.** It usually does not — that gap is why the defect shipped. - **Running the whole pack for every fix**, so feedback arrives hours later and nobody reads it. ## A partial fix, and re-opening Confirmation is also where partial fixes surface. If the report contains several reproduction paths and only one is repaired, the defect is not confirmed; it is re-opened with the paths that still fail spelled out. Re-testing only the single narrowest step from the report is how a defect gets closed three times. ## Vocabulary to keep straight Confirmation testing and re-testing are the same activity under two names. Regression testing means re-running previously passing checks after a change — it is an activity, not a particular suite, and it does not automatically mean the whole nightly pack. The same case can serve as a confirmation case today and as a regression case for the next twelve months.
- A fix is confirmed green but a shared downstream record is now written twice. Why did confirmation miss it?Because confirmation only checks the behaviour named in the defect report, and that behaviour is correct — the total, the message, the returned value. A duplicated side effect lives outside the report's expectations, in a record, an event or a downstream artefact nobody asserted. It is found only by cases that assert on those artefacts, selected because the change touched the code that writes them. This is the standard argument for regression scope following the change's write paths, not just its inputs and outputs.
- Should the case that exposed the defect always go into the regression pack?Usually yes, because a defect that escaped is evidence of a real gap and its reproduction steps are already known to work. Add it in the form the pack can run repeatedly — deterministic data, no manual setup — rather than pasting the report verbatim. The judgement is about the form and the level the case is written at, not about whether the gap deserves cover; a gap that produced a real defect has earned it.
- Does regression testing only happen after defect fixes?No. Regression testing applies after any change that could disturb working behaviour: a new feature, a refactor, a dependency upgrade, a configuration or data change, an environment migration. Confirmation testing is the narrower activity — it exists only where there is a defect report to confirm against. In practice most regression runs are triggered by ordinary changes, and only a minority follow a fix.
Confirmation testing is checking that the leak you called the plumber about has stopped. Regression testing is checking the rooms next door for the water they let in while fixing it.
saying these in an interview costs you the question
- Says regression testing means re-running the failing case after the fix
- Treats a green confirmation run as proof nothing else broke
- Confirms a fix on a build that does not contain it
- Skips confirmation because the pack supposedly covers that area
- Thinks regression testing only happens after defect fixes