In the Maven Surefire report schema, `<failure>` may appear any number of times inside one `<testcase>` while `<error>` and `<skipped>` may appear at most once. What does that let a file express, and where does it break down?

middlenice to knowfreq 42%

answer

count the children, not the outcomes
one repeats, two are capped
maxOccurs unbounded on <failure>
two <error> elements break the schema

basics

~20 s

<failure> is declared unbounded while <error> and <skipped> are capped at one, so a case may carry several assertion failures but only a single error or skip. A writer emitting two <error> elements produces a file the schema rejects.

solid answer

~40 s

The schema declares <failure> with minOccurs=0 and maxOccurs=unbounded, and <error> and <skipped> with a maximum of one. All three are also nillable=true over string content, so an empty <skipped/> or <failure/> is valid and records the outcome with no message and no trace. The unbounded declaration is genuinely used: JUnit 5's LegacyXmlReportGeneratingListener collects results for the case and its enclosing containers and writes one element per recorded throwable, so two assertion failures give two <failure> elements. The same mechanism with two non-assertion throwables gives two <error> elements, which exceeds the cap -- the file no longer validates, and readers that assume one element typically take the first and drop the rest.

The schema declares `<failure>` with `minOccurs=0` and `maxOccurs=unbounded`, and `<error>` and `<skipped>` with a maximum of one. All three are also `nillable=true` over string content, so an empty `<skipped/>` or `<failure/>` is valid and records the outcome with no message and no trace. The unbounded declaration is genuinely used: JUnit 5's `LegacyXmlReportGeneratingListener` collects results for the case and its enclosing containers and writes one element per recorded throwable, so two assertion failures give two `<failure>` elements. The same mechanism with two non-assertion throwables gives two `<error>` elements, which exceeds the cap -- the file no longer validates, and readers that assume one element typically take the first and drop the rest.

code

xml · 11 lines

xml

<testsuite name="com.example.OrderTest" tests="2" errors="0" skipped="1" failures="1" time="0.180">
  <testcase name="totalsLineItems" classname="com.example.OrderTest" time="0.120">
    <failure message="total was 0" type="java.lang.AssertionError">java.lang.AssertionError: total was 0
	at com.example.OrderTest.totalsLineItems(OrderTest.java:31)</failure>
    <failure message="invariant check failed" type="java.lang.AssertionError">java.lang.AssertionError: invariant check failed
	at com.example.OrderTest.checkInvariants(OrderTest.java:88)</failure>
  </testcase>
  <testcase name="appliesTax" classname="com.example.OrderTest" time="0.000">
    <skipped/>
  </testcase>
</testsuite>

go deeper

for a junior

Know that <failure> may appear more than once on a case while <error> and <skipped> may not, and that an outcome element with no attributes or text is still valid.

for a middle

Explain why the asymmetry exists -- one case can genuinely produce several assertion failures -- and what a writer must do instead when it holds several non-assertion throwables.

for a senior

Show what goes wrong in the field: an invalid file that everything still parses, and a reader that takes the first outcome element so the extra evidence disappears without an error.

for a principal

Be ready to decide what your tooling treats as authoritative when a file exceeds the schema's cardinality, and whether you validate at ingest or accept whatever parses.

What the schema actually declares

Inside <testcase>, the Maven Surefire report schema surefire-test-report.xsd does not treat the three ordinary outcome elements alike. Their declarations differ in exactly the way that matters to anyone writing or reading these files:

element	maxOccurs	attributes	nillable
`<failure>`	unbounded	`@message`, `@type`	yes
`<error>`	1	`@message`, `@type`	yes
`<skipped>`	1	`@message`	yes

Two facts fall out of that table. First, one case may carry many <failure> elements but at most one <error> and at most one <skipped>. Second, all three are declared nillable="true" and their content is an extension of xs:string, so an element written with no attributes and no text -- <skipped/>, <failure/> -- is perfectly valid. It records that the outcome happened and nothing else about it.

Why `<failure>` is the one that repeats

A single test case can produce more than one assertion failure, and the format was built to let a writer say so rather than pick one and discard the rest. Grouped assertions, a failing check in the test body followed by another in teardown, or a failure recorded against an enclosing container as well as the method -- any of these can yield two independent assertion failures attached to one case.

JUnit 5's LegacyXmlReportGeneratingListener does exactly this. It gathers the recorded results for a case and for its enclosing containers, groups them by classification, and writes one element per recorded throwable. Two assertion failures produce two <failure> elements on one <testcase>, which the schema permits.

Where the cap bites

The same mechanism has no cap of its own, so two non-assertion throwables produce two <error> elements -- one more than maxOccurs="1" allows. The file is then not valid against surefire-test-report.xsd, but nothing stops it being written and almost nothing stops it being read: most consumers of JUnit-XML parse it with an ordinary XML parser and never validate against the schema at all.

The practical failure is quieter than an invalid file. Readers written against the schema's cardinality assume a single element and take the first one they find. Allure 2's JunitXmlPlugin does precisely that when it fills a result's message and trace: it looks for <failure>, <error> and <skipped> in turn, takes the first element present, copies its @message into the message and its text into the trace, and stops. Everything after that first element is discarded.

So the asymmetry produces three distinct behaviours worth keeping straight:

Two <failure> elements -- valid, and a schema-aware reader may still show only the first.
Two <error> elements -- invalid, usually parsed anyway, and again usually only the first is shown.
A <failure> and an <error> on one case -- valid by cardinality, and a reader that decides status by checking <failure> first will call it a failure and never mention the error.

The nil case

nillable="true" on all three, combined with string content that may be empty, means the format allows an outcome to be asserted with no evidence at all. That is not hypothetical: a case skipped without a stated reason is written as an empty <skipped/>, and a failed case with no throwable recorded is written as an empty <failure/> or <error/>.

For a reader this is the difference between "no message" and "no element". A missing <skipped> means the case was not skipped; an empty <skipped/> means it was, and the writer had nothing to say about why. Code that tests only for the element's text content and treats empty text as absent collapses those two distinct states into one, and reports skipped cases as passes.

What to do with all this

When writing a converter into JUnit-XML, emit at most one <error> and one <skipped> per case; if you hold several non-assertion throwables, merge their traces into a single element rather than producing an invalid file.
Repeat <failure> freely when the case genuinely produced several assertion failures -- that is what the unbounded declaration is for -- but do not assume a reader will show more than the first.
When writing a reader, decide deliberately whether you take the first outcome element or merge them all, and document it; taking the first silently is the common behaviour and the common surprise.
Check for element presence, not for non-empty content: a nil outcome element is a real, valid, meaningful state.

## What the schema actually declares Inside `<testcase>`, the Maven Surefire report schema `surefire-test-report.xsd` does not treat the three ordinary outcome elements alike. Their declarations differ in exactly the way that matters to anyone writing or reading these files: | element | minOccurs | maxOccurs | attributes | nillable | |---|---|---|---|---| | `<failure>` | 0 | unbounded | `@message`, `@type` | yes | | `<error>` | 0 | 1 | `@message`, `@type` | yes | | `<skipped>` | 0 | 1 | `@message` | yes | Two facts fall out of that table. First, **one case may carry many `<failure>` elements but at most one `<error>` and at most one `<skipped>`.** Second, all three are declared `nillable="true"` and their content is an extension of `xs:string`, so an element written with no attributes and no text -- `<skipped/>`, `<failure/>` -- is perfectly valid. It records that the outcome happened and nothing else about it. ## Why `<failure>` is the one that repeats A single test case can produce more than one assertion failure, and the format was built to let a writer say so rather than pick one and discard the rest. Grouped assertions, a failing check in the test body followed by another in teardown, or a failure recorded against an enclosing container as well as the method -- any of these can yield two independent assertion failures attached to one case. JUnit 5's `LegacyXmlReportGeneratingListener` does exactly this. It gathers the recorded results for a case **and for its enclosing containers**, groups them by classification, and writes one element per recorded throwable. Two assertion failures produce two `<failure>` elements on one `<testcase>`, which the schema permits. ## Where the cap bites The same mechanism has no cap of its own, so two non-assertion throwables produce **two `<error>` elements** -- one more than `maxOccurs="1"` allows. The file is then not valid against `surefire-test-report.xsd`, but nothing stops it being written and almost nothing stops it being read: most consumers of JUnit-XML parse it with an ordinary XML parser and never validate against the schema at all. The practical failure is quieter than an invalid file. Readers written against the schema's cardinality assume a single element and take the first one they find. Allure 2's `JunitXmlPlugin` does precisely that when it fills a result's message and trace: it looks for `<failure>`, `<error>` and `<skipped>` in turn, takes the **first** element present, copies its `@message` into the message and its text into the trace, and stops. Everything after that first element is discarded. So the asymmetry produces three distinct behaviours worth keeping straight: 1. **Two `<failure>` elements** -- valid, and a schema-aware reader may still show only the first. 2. **Two `<error>` elements** -- invalid, usually parsed anyway, and again usually only the first is shown. 3. **A `<failure>` and an `<error>` on one case** -- valid by cardinality, and a reader that decides status by checking `<failure>` first will call it a failure and never mention the error. ## The nil case `nillable="true"` on all three, combined with string content that may be empty, means the format allows an outcome to be asserted with no evidence at all. That is not hypothetical: a case skipped without a stated reason is written as an empty `<skipped/>`, and a failed case with no throwable recorded is written as an empty `<failure/>` or `<error/>`. For a reader this is the difference between "no message" and "no element". A missing `<skipped>` means the case was not skipped; an empty `<skipped/>` means it was, and the writer had nothing to say about why. Code that tests only for the element's text content and treats empty text as absent collapses those two distinct states into one, and reports skipped cases as passes. ## What to do with all this - When writing a converter into JUnit-XML, emit at most one `<error>` and one `<skipped>` per case; if you hold several non-assertion throwables, merge their traces into a single element rather than producing an invalid file. - Repeat `<failure>` freely when the case genuinely produced several assertion failures -- that is what the unbounded declaration is for -- but do not assume a reader will show more than the first. - When writing a reader, decide deliberately whether you take the first outcome element or merge them all, and document it; taking the first silently is the common behaviour and the common surprise. - Check for element presence, not for non-empty content: a nil outcome element is a real, valid, meaningful state.

Which writer actually emits more than one `<failure>` for a single case?
JUnit 5's `LegacyXmlReportGeneratingListener`. It writes one element per recorded throwable and gathers results for the case and its enclosing containers, so an assertion in the method plus a failing check on the container produces two `<failure>` elements on one `<testcase>`.
What happens when that same mechanism produces two non-assertion throwables?
It writes two `<error>` elements, one more than `surefire-test-report.xsd` permits, so the file no longer validates. In practice most consumers parse it without validating and use only the first element, so the second error is silently dropped rather than reported as a problem.

saying these in an interview costs you the question

Claims all three outcome elements may repeat inside one <testcase>
Says an empty <skipped/> is invalid without a @message
Assumes a reader merges several <failure> elements rather than taking the first
Treats an outcome element with empty content as if it were absent

In the Maven Surefire report schema, `<failure>` may appear any number of times inside one `<testcase>` while `<error>` and `<skipped>` may appear at most once. What does that let a file express, and where does it break down?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

What the schema actually declares

Why <failure> is the one that repeats

Where the cap bites

The nil case

What to do with all this

▾common follow-ups2 questions

▾red flags4 flags

What the schema actually declares

Why <failure> is the one that repeats

Where the cap bites

The nil case

What to do with all this

answer

go deeper

Why `<failure>` is the one that repeats

Why `<failure>` is the one that repeats