A build with no re-runs configured still writes `flakes="0"` on every `<testsuite>` in the JUnit-XML Maven Surefire produces. What can a reader conclude from that attribute being present, and what only from its value?

middlenice to knowfreq 22%

answer

optional in the schema, not in the writer
the attribute is always there
zero is written, never omitted
presence identifies the writer, not the setting

basics

~10 s

Presence says almost nothing: Maven Surefire writes @flakes on every <testsuite> unconditionally, using 0 when nothing flaked. Only a non-zero value reports anything - the count of cases that failed and then passed.

solid answer

~40 s

@flakes is optional in surefire-test-report.xsd, but Surefire's StatelessXmlReporter emits it outside any conditional, so it appears on every <testsuite> that writer produces - with the value 0 in a build where nothing flaked, including the common build where surefire.rerunFailingTestsCount is left at its default of 0 and nothing was ever re-run. That makes presence a fingerprint of the writer, not evidence that re-runs were enabled. Only the value carries information: a non-zero @flakes counts the cases that failed at least once and then passed, the ones written with <flakyFailure> or <flakyError>. Contrast @group, @time and @timestamp, which Surefire genuinely does write conditionally, and JUnit 5's LegacyXmlReportGeneratingListener, which emits no @flakes at all.

`@flakes` is optional in `surefire-test-report.xsd`, but Surefire's `StatelessXmlReporter` emits it outside any conditional, so it appears on every `<testsuite>` that writer produces - with the value `0` in a build where nothing flaked, including the common build where `surefire.rerunFailingTestsCount` is left at its default of `0` and nothing was ever re-run. That makes presence a fingerprint of the writer, not evidence that re-runs were enabled. Only the value carries information: a non-zero `@flakes` counts the cases that failed at least once and then passed, the ones written with `<flakyFailure>` or `<flakyError>`. Contrast `@group`, `@time` and `@timestamp`, which Surefire genuinely does write conditionally, and JUnit 5's `LegacyXmlReportGeneratingListener`, which emits no `@flakes` at all.

code

xml · 5 lines

xml

<testsuite name="com.example.CartTest" tests="3" errors="0" skipped="0" failures="0" flakes="0">
  <testcase name="addsItem" classname="com.example.CartTest" time="0.09"/>
  <testcase name="clearsItem" classname="com.example.CartTest" time="0.04"/>
  <testcase name="totalsItems" classname="com.example.CartTest" time="0.07"/>
</testsuite>

go deeper

for a junior

Know that @flakes sits on <testsuite> and counts cases that failed and then passed, and that a value of 0 is written out rather than left off.

for a middle

Explain the gap between an attribute the schema marks optional and one the writer emits unconditionally, and name which of <testsuite>'s optional attributes Surefire genuinely does write conditionally.

for a senior

Show how you would stop a consolidation job inferring configuration from a file: presence of @flakes identifies the writer, and only the value reports anything about the run.

for a principal

Take a position on whether a shared result contract should require writers to emit every counter always, so absent and zero can never be confused downstream.

@flakes is one of the optional attributes surefire-test-report.xsd declares on <testsuite>, beside the required @name, @tests, @errors, @skipped and @failures and the other optional ones, @version, @time, @timestamp and @group. "Optional in the schema" is a statement about what a file may legally omit. It is not a statement about what the writer does, and for @flakes the two come apart.

Optional in the schema, unconditional in the writer

Surefire's StatelessXmlReporter counts the cases that recovered and then emits the attribute outside any conditional. Every <testsuite> element it writes carries @flakes, and in a build where nothing flaked the value written is 0 - not an omitted attribute, not an empty string.

Contrast that with the other optional attributes on the same element. @group, @time and @timestamp are genuinely conditional in the writer: they appear when there is something to say and are left out otherwise. @flakes is the exception, and it is the one people guess wrong about, because "optional" invites the assumption that its presence is itself a signal.

What presence and value each tell you

Presence tells you about the writer, not about the run. A <testsuite> with @flakes on it was produced by something that always emits the attribute. It does not tell you that re-runs were enabled, that any case was re-run, or that the schema was consulted. surefire.rerunFailingTestsCount defaults to 0, so the overwhelmingly common build - the one that never re-ran anything - still ships a file full of flakes="0".
Absence tells you it was a different writer. JUnit 5's LegacyXmlReportGeneratingListener emits no @flakes at all, and none of the four repetition elements either. So in a directory of files gathered from several sources, the presence or absence of @flakes is a usable fingerprint of which tool wrote each file - and that is genuinely all it is.
Only the value carries information about the run. A non-zero @flakes counts the cases that failed at least once and then passed: exactly the cases written with <flakyFailure> or <flakyError> children. Those cases are not in @failures, because they ended green.

The two mistakes this invites

Inferring configuration from the file. "The attribute is there, so re-runs must be on" is wrong in the common case, and produces confident nonsense in a dashboard that claims to report which projects have re-runs enabled. The file records outcomes; it does not record settings.
Treating zero and absent as the same thing. They are not, and a consumer that normalises a missing attribute to 0 merges Surefire files and other writers' files into one set of numbers where "no flakes recorded" and "flakes not measurable" look identical. If your consolidation has to be honest about coverage, keep those two states apart on the way in.

There is a smaller trap on the counting side. @flakes counts cases under both suffixes, so a reader that scans only for <flakyFailure> elements can find fewer cases than the attribute claims: the difference is cases whose failed attempts ended in errors and were written as <flakyError>.

Using it well

If you need a suite-level flake number and you are reading Surefire's own output, @flakes is the cheapest correct source - it is one attribute rather than a walk over every <testcase>, and it already counts cases rather than attempts. Two conditions apply:

Check the value, never the presence. Presence is constant across that writer's entire output.
Confirm the file was actually written by Surefire before trusting the number, because a file from another writer contributes nothing rather than contributing a zero.

And if what you want is attempts rather than cases, @flakes is the wrong number entirely - that has to come from counting the repetition elements inside each <testcase>.

`@flakes` is one of the optional attributes `surefire-test-report.xsd` declares on `<testsuite>`, beside the required `@name`, `@tests`, `@errors`, `@skipped` and `@failures` and the other optional ones, `@version`, `@time`, `@timestamp` and `@group`. "Optional in the schema" is a statement about what a *file* may legally omit. It is not a statement about what the *writer* does, and for `@flakes` the two come apart. ## Optional in the schema, unconditional in the writer Surefire's `StatelessXmlReporter` counts the cases that recovered and then emits the attribute outside any conditional. Every `<testsuite>` element it writes carries `@flakes`, and in a build where nothing flaked the value written is `0` - not an omitted attribute, not an empty string. Contrast that with the other optional attributes on the same element. `@group`, `@time` and `@timestamp` are genuinely conditional in the writer: they appear when there is something to say and are left out otherwise. `@flakes` is the exception, and it is the one people guess wrong about, because "optional" invites the assumption that its presence is itself a signal. ## What presence and value each tell you - **Presence tells you about the writer, not about the run.** A `<testsuite>` with `@flakes` on it was produced by something that always emits the attribute. It does not tell you that re-runs were enabled, that any case was re-run, or that the schema was consulted. `surefire.rerunFailingTestsCount` defaults to `0`, so the overwhelmingly common build - the one that never re-ran anything - still ships a file full of `flakes="0"`. - **Absence tells you it was a different writer.** JUnit 5's `LegacyXmlReportGeneratingListener` emits no `@flakes` at all, and none of the four repetition elements either. So in a directory of files gathered from several sources, the presence or absence of `@flakes` is a usable fingerprint of which tool wrote each file - and that is genuinely all it is. - **Only the value carries information about the run.** A non-zero `@flakes` counts the **cases** that failed at least once and then passed: exactly the cases written with `<flakyFailure>` or `<flakyError>` children. Those cases are not in `@failures`, because they ended green. ## The two mistakes this invites 1. **Inferring configuration from the file.** "The attribute is there, so re-runs must be on" is wrong in the common case, and produces confident nonsense in a dashboard that claims to report which projects have re-runs enabled. The file records outcomes; it does not record settings. 2. **Treating zero and absent as the same thing.** They are not, and a consumer that normalises a missing attribute to `0` merges Surefire files and other writers' files into one set of numbers where "no flakes recorded" and "flakes not measurable" look identical. If your consolidation has to be honest about coverage, keep those two states apart on the way in. There is a smaller trap on the counting side. `@flakes` counts cases under **both** suffixes, so a reader that scans only for `<flakyFailure>` elements can find fewer cases than the attribute claims: the difference is cases whose failed attempts ended in errors and were written as `<flakyError>`. ## Using it well If you need a suite-level flake number and you are reading Surefire's own output, `@flakes` is the cheapest correct source - it is one attribute rather than a walk over every `<testcase>`, and it already counts cases rather than attempts. Two conditions apply: - **Check the value, never the presence.** Presence is constant across that writer's entire output. - **Confirm the file was actually written by Surefire** before trusting the number, because a file from another writer contributes nothing rather than contributing a zero. And if what you want is attempts rather than cases, `@flakes` is the wrong number entirely - that has to come from counting the repetition elements inside each `<testcase>`.

A `<testsuite>` reports `flakes="2"` and `failures="0"`. Are those two cases counted anywhere else in the suite's totals?
They are counted in `@tests`, like every case, but not in `@failures` - a case that ended green is not a failure. They appear as `<testcase>` elements carrying `<flakyFailure>` or `<flakyError>` children, and `@flakes` is the only suite-level number that says they happened at all.
Could `@flakes` be non-zero on a suite whose `<testcase>` elements carry no `<flakyFailure>` children?
Yes, if the failed attempts ended in errors rather than assertion failures, in which case they were written as `<flakyError>`. `@flakes` counts recovered cases under either suffix, so a reader that scans only for `<flakyFailure>` will find fewer cases than the attribute claims.

saying these in an interview costs you the question

Treats the presence of @flakes as proof re-runs were enabled
Assumes @flakes is omitted when no case flaked
Reads flakes=0 as the writer not supporting the attribute
Expects every JUnit-XML writer to emit an @flakes attribute
Normalises a missing @flakes to zero when merging writers

A build with no re-runs configured still writes `flakes="0"` on every `<testsuite>` in the JUnit-XML Maven Surefire produces. What can a reader conclude from that attribute being present, and what only from its value?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

Optional in the schema, unconditional in the writer

What presence and value each tell you

The two mistakes this invites

Using it well

▾common follow-ups2 questions

▾red flags5 flags

Optional in the schema, unconditional in the writer

What presence and value each tell you

The two mistakes this invites

Using it well

answer

go deeper