In the JUnit-XML result file Maven Surefire writes, a re-run test case can appear with `<flakyFailure>` children, or with a `<failure>` followed by `<rerunFailure>` children. What does each of those two shapes say about how the case ended?

juniormust knowfreq 44%

answer

four elements, not two
the suffix only names the error kind
the prefix decides the build colour
ask whether a later attempt passed

basics

~10 s

<flakyFailure> records a failed attempt of a case that eventually passed, so the build stays green. A <failure> plus <rerunFailure> records a case that failed on every attempt, so the build goes red.

solid answer

~40 s

Maven Surefire has four repetition elements - <flakyFailure>, <flakyError>, <rerunFailure> and <rerunError> - and the pair you get is chosen by the case's final outcome, not by the flavour of the problem. If a case failed at least once and then passed, every failed attempt is written as <flakyFailure> (or <flakyError> when that attempt ended in an error rather than an assertion failure); the case is counted in the suite's @flakes rather than its @failures, and @time is the last, successful run. If the case never passed, the first attempt is the ordinary <failure> (or <error>) and every later attempt a <rerunFailure> (or <rerunError>); the case is red and @time is the first, failing run. The prefix answers did it ever pass; the suffix answers how that attempt broke.

Maven Surefire has four repetition elements - `<flakyFailure>`, `<flakyError>`, `<rerunFailure>` and `<rerunError>` - and the pair you get is chosen by the case's **final** outcome, not by the flavour of the problem. If a case failed at least once and then passed, every failed attempt is written as `<flakyFailure>` (or `<flakyError>` when that attempt ended in an error rather than an assertion failure); the case is counted in the suite's `@flakes` rather than its `@failures`, and `@time` is the last, successful run. If the case never passed, the first attempt is the ordinary `<failure>` (or `<error>`) and every later attempt a `<rerunFailure>` (or `<rerunError>`); the case is red and `@time` is the first, failing run. The prefix answers *did it ever pass*; the suffix answers *how that attempt broke*.

code

xml · 16 lines

xml

<testsuite name="com.example.CartTest" tests="2" errors="0" skipped="0" failures="1" flakes="1">
  <testcase name="addsItem" classname="com.example.CartTest" time="0.41">
    <flakyFailure message="expected 1 but was 0" type="java.lang.AssertionError">
      <stackTrace>java.lang.AssertionError: expected 1 but was 0
    at com.example.CartTest.addsItem(CartTest.java:22)</stackTrace>
    </flakyFailure>
  </testcase>
  <testcase name="clearsItem" classname="com.example.CartTest" time="0.12">
    <failure message="expected 0 but was 3" type="java.lang.AssertionError">java.lang.AssertionError: expected 0 but was 3
    at com.example.CartTest.clearsItem(CartTest.java:31)</failure>
    <rerunFailure message="expected 0 but was 3" type="java.lang.AssertionError">
      <stackTrace>java.lang.AssertionError: expected 0 but was 3
    at com.example.CartTest.clearsItem(CartTest.java:31)</stackTrace>
    </rerunFailure>
  </testcase>
</testsuite>

go deeper

for a junior

Be ready to say which element appears when a case eventually passed and which appears when it never did, and to name all four rather than two.

for a middle

Explain that the prefix is chosen from the final outcome while the suffix mirrors the ordinary <failure> and <error> distinction, and say what the suite's @flakes count and the case's @time then hold.

for a senior

Show where this bites in production: a consumer keying off the suffix reports a green build as broken, and @time means the last run for a recovered case but the first for one that never passed.

for a principal

Own the position that a consolidated result store should record did-it-ever-pass as a field of its own, rather than re-deriving it from element names each writer chooses differently.

Maven Surefire writes one JUnit-XML file per test class, named TEST-<sourceName>.xml, and the outcome of a test case is recorded by the children of its <testcase> element. When a case is allowed to run more than once, the schema surefire-test-report.xsd hands it four extra element names rather than two, and the choice between them is the single fact about this file that is most often read backwards.

The four elements, and the two questions they answer

The four repetition elements are <rerunFailure>, <rerunError>, <flakyFailure> and <flakyError>. Each is declared minOccurs="0" maxOccurs="unbounded", so one <testcase> may carry any number of them, and each one records exactly one attempt.

Every name glues two independent decisions together:

The prefix - flaky or rerun - answers "did the case ever pass?" flaky means yes: some later attempt succeeded. rerun means no: the case failed on every attempt it was given.
The suffix - Failure or Error - answers "how did that one attempt break?" It mirrors the ordinary <failure> / <error> distinction the format already uses for a case that ran once.

So the split is on the final outcome, not on the flavour of the problem. A case that threw an unexpected exception on its first run and then passed is written with <flakyError>; a case that failed an assertion on every run is written with <failure> and then <rerunFailure>. Reading the suffix to decide whether the build should be red is the classic mistake - the suffix never says anything about the verdict.

What each shape looks like on disk

the case	what `<testcase>` holds	build	what `@time` holds
failed, then passed on a later attempt	one `<flakyFailure>` or `<flakyError>` per failed attempt	green	the last, successful run
failed on every attempt	one `<failure>` or `<error>`, then one `<rerunFailure>` or `<rerunError>` per later attempt	red	the first, failing run
passed first time	no outcome child at all	green	its only run

Two consequences of that table catch people out:

A case that recovered carries no <failure> element at all. The plain <failure> is written only for the first attempt of a case that never passed. A tool that scans for <failure> to find problems will not see the recovered cases, because their failed attempts sit under a different element name entirely.
The final, passing attempt is written nowhere. A pass leaves no outcome child, so three <flakyFailure> children describe four runs: three failures and one success that has no element of its own.

The @time row matters as much as the element names. Because the writer starts the <testcase> element from the successful entry for a recovered case and from the first entry for one that never passed, @time is not a total and it does not even describe the same attempt across the two shapes.

The suite-level counter

<testsuite> carries an optional @flakes attribute beside its required @name, @tests, @errors, @skipped and @failures. @flakes counts the cases that recovered - precisely those written with <flakyFailure> or <flakyError> children. Those cases are not counted in @failures, because they ended green. A suite can therefore report failures="0" next to flakes="3", and anything reading only @failures will call that build clean without noticing that three cases needed a second chance.

What the schema deliberately does not have

There is no rerunSkipped and no flakySkipped. The enumeration Surefire uses to choose a tag, ReportEntryType, has four rows, and only two of them carry rerun and flaky names:

FAILURE maps to failure / flakyFailure / rerunFailure
ERROR maps to error / flakyError / rerunError
SKIPPED maps to skipped, and to empty strings for the other two
SUCCESS maps to empty strings throughout

The empty strings on SUCCESS are the mechanism behind "a pass writes nothing at all", and the empty strings on SKIPPED are why a skip that happens on a later attempt has nowhere of its own to go - the writer emits an XML comment in that position instead of an element.

Why the distinction is worth the trouble

The point of the split is that the file is read by something that has to reach a verdict: a consolidation job, a dashboard, a merge step across shards. That reader needs "did this case end green?", and the four element names are the only place the file says so. Get the prefix backwards and every conclusion inverts - builds that are hard red get reported as merely flaky, and builds where a case quietly needed three goes get reported as clean.

Two habits keep it straight. First, describe a case by its final outcome before naming an element; the element name follows from the outcome, never the other way round. Second, when you test a consumer of these files, build a fixture holding all four repetition elements plus a plain pass and a plain failure, and assert the verdict for each. A fixture containing only <failure> will never catch a reader that has the prefix the wrong way round.

Maven Surefire writes one JUnit-XML file per test class, named `TEST-<sourceName>.xml`, and the outcome of a test case is recorded by the children of its `<testcase>` element. When a case is allowed to run more than once, the schema `surefire-test-report.xsd` hands it **four** extra element names rather than two, and the choice between them is the single fact about this file that is most often read backwards. ## The four elements, and the two questions they answer The four repetition elements are `<rerunFailure>`, `<rerunError>`, `<flakyFailure>` and `<flakyError>`. Each is declared `minOccurs="0" maxOccurs="unbounded"`, so one `<testcase>` may carry any number of them, and each one records exactly one attempt. Every name glues two independent decisions together: - **The prefix - `flaky` or `rerun` - answers "did the case ever pass?"** `flaky` means yes: some later attempt succeeded. `rerun` means no: the case failed on every attempt it was given. - **The suffix - `Failure` or `Error` - answers "how did that one attempt break?"** It mirrors the ordinary `<failure>` / `<error>` distinction the format already uses for a case that ran once. So the split is on the **final outcome, not on the flavour of the problem**. A case that threw an unexpected exception on its first run and then passed is written with `<flakyError>`; a case that failed an assertion on every run is written with `<failure>` and then `<rerunFailure>`. Reading the suffix to decide whether the build should be red is the classic mistake - the suffix never says anything about the verdict. ## What each shape looks like on disk | the case | what `<testcase>` holds | build | what `@time` holds | |---|---|---|---| | failed, then passed on a later attempt | one `<flakyFailure>` or `<flakyError>` per failed attempt | green | the last, successful run | | failed on every attempt | one `<failure>` or `<error>`, then one `<rerunFailure>` or `<rerunError>` per later attempt | red | the first, failing run | | passed first time | no outcome child at all | green | its only run | Two consequences of that table catch people out: 1. **A case that recovered carries no `<failure>` element at all.** The plain `<failure>` is written only for the *first* attempt of a case that never passed. A tool that scans for `<failure>` to find problems will not see the recovered cases, because their failed attempts sit under a different element name entirely. 2. **The final, passing attempt is written nowhere.** A pass leaves no outcome child, so three `<flakyFailure>` children describe four runs: three failures and one success that has no element of its own. The `@time` row matters as much as the element names. Because the writer starts the `<testcase>` element from the successful entry for a recovered case and from the first entry for one that never passed, `@time` is not a total and it does not even describe the same attempt across the two shapes. ## The suite-level counter `<testsuite>` carries an optional `@flakes` attribute beside its required `@name`, `@tests`, `@errors`, `@skipped` and `@failures`. `@flakes` counts the **cases that recovered** - precisely those written with `<flakyFailure>` or `<flakyError>` children. Those cases are *not* counted in `@failures`, because they ended green. A suite can therefore report `failures="0"` next to `flakes="3"`, and anything reading only `@failures` will call that build clean without noticing that three cases needed a second chance. ## What the schema deliberately does not have There is **no `rerunSkipped` and no `flakySkipped`**. The enumeration Surefire uses to choose a tag, `ReportEntryType`, has four rows, and only two of them carry rerun and flaky names: - `FAILURE` maps to `failure` / `flakyFailure` / `rerunFailure` - `ERROR` maps to `error` / `flakyError` / `rerunError` - `SKIPPED` maps to `skipped`, and to empty strings for the other two - `SUCCESS` maps to empty strings throughout The empty strings on `SUCCESS` are the mechanism behind "a pass writes nothing at all", and the empty strings on `SKIPPED` are why a skip that happens on a later attempt has nowhere of its own to go - the writer emits an XML comment in that position instead of an element. ## Why the distinction is worth the trouble The point of the split is that the file is read by something that has to reach a verdict: a consolidation job, a dashboard, a merge step across shards. That reader needs "did this case end green?", and the four element names are the only place the file says so. Get the prefix backwards and every conclusion inverts - builds that are hard red get reported as merely flaky, and builds where a case quietly needed three goes get reported as clean. Two habits keep it straight. First, describe a case by its *final* outcome before naming an element; the element name follows from the outcome, never the other way round. Second, when you test a consumer of these files, build a fixture holding all four repetition elements plus a plain pass and a plain failure, and assert the verdict for each. A fixture containing only `<failure>` will never catch a reader that has the prefix the wrong way round.

The schema declares all four repetition elements unbounded. When would one `<testcase>` legitimately carry several of them?
When the case was re-run more than once. Each failed attempt gets its own element with its own `<stackTrace>`, so three failed attempts before a pass produce three `<flakyFailure>` children, and three attempts that all failed produce one `<failure>` plus two `<rerunFailure>` children. Counting those elements is how the file records how many attempts there were.
Is there a `rerunSkipped` or `flakySkipped` element for a case skipped on a later attempt?
No. The schema declares only the four, mirroring the failure and error outcomes. `<skipped>` may appear at most once inside a `<testcase>` and has no repeated form, so a skip on a later attempt has nowhere of its own to go - Surefire writes an XML comment in that position instead of an element.

Think of a driving test kept as one candidate file. Every attempt is written down, but the label on the file says whether a licence was finally issued - and it is that label, not the number of attempts, that anything downstream acts on.

saying these in an interview costs you the question

Says the flaky and rerun elements differ by error kind, not by outcome
Assumes a case with <flakyFailure> children turned the build red
Thinks the four elements are two elements with alternative spellings
Believes <rerunFailure> means the case eventually passed
Expects a rerunSkipped element to exist alongside the other four

In the JUnit-XML result file Maven Surefire writes, a re-run test case can appear with `<flakyFailure>` children, or with a `<failure>` followed by `<rerunFailure>` children. What does each of those two shapes say about how the case ended?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

The four elements, and the two questions they answer

What each shape looks like on disk

The suite-level counter

What the schema deliberately does not have

Why the distinction is worth the trouble

▾common follow-ups2 questions

▾analogy~20 s

▾red flags5 flags

The four elements, and the two questions they answer

What each shape looks like on disk

The suite-level counter

What the schema deliberately does not have

Why the distinction is worth the trouble

answer

go deeper