In the JUnit-XML result file Maven Surefire writes, how is a passing test case recorded, and which children of `<testcase>` mark a case that did not pass?
answer
- the verdict nobody writes down
- absence is the signal
- no outcome child means green
- SUCCESS maps to an empty tag name
basics
~10 sA pass is written as a <testcase> element carrying no outcome child at all. Only <failure>, <error> and <skipped> mark a non-pass, so a reader concludes a case passed from their absence.
solid answer
~40 sThe Maven Surefire report schema gives `<testcase>` no attribute or child that says *passed*. A case that passed is simply a `<testcase>` element with none of the outcome children present -- no `<failure>`, no `<error>`, no `<skipped>`. Surefire's `ReportEntryType` enum makes that explicit: its `SUCCESS` row maps to an empty XML tag name, so there is literally no tag to write, while every other outcome names exactly one child. Readers invert the same rule; Allure 2's `JunitXmlPlugin` looks for `<failure>`, then `<error>`, then `<skipped>`, and returns `PASSED` when it finds none. Note that a pass is not necessarily childless: `<system-out>` and `<system-err>` may still be there. The rule is *no outcome child*, not *no children*.
code
xml · 11 lines<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="com.example.CartTest" tests="3" errors="0" skipped="1" failures="1" time="0.412">
<testcase name="addsItem" classname="com.example.CartTest" time="0.031"/>
<testcase name="rejectsNegativeQuantity" classname="com.example.CartTest" time="0.010">
<failure message="quantity was accepted" type="java.lang.AssertionError">java.lang.AssertionError: quantity was accepted
at com.example.CartTest.rejectsNegativeQuantity(CartTest.java:42)</failure>
</testcase>
<testcase name="appliesCoupon" classname="com.example.CartTest" time="0.000">
<skipped message="coupon service unavailable"/>
</testcase>
</testsuite>go deeper
Be able to say plainly that a passing case is a <testcase> with no outcome child, and to name the three children that mark a non-pass: <failure>, <error> and <skipped>.
Explain the mechanism, not just the rule: the writer's SUCCESS outcome maps to an empty tag name, so nothing is emitted, and a reader treats passed as the fallthrough when no outcome child matched.
Show you know what the encoding costs in practice -- a truncated or killed run yields missing cases rather than incomplete ones, so a green-looking file can hide work that never happened.
Be ready to say what your organisation treats as the contract when a format encodes its most common outcome as an absence, and where the check that everything expected actually ran has to live instead.
## The format records exceptions, not verdicts The JUnit-XML interchange file is often described as if every test case carried a verdict. It does not. In the Maven Surefire report schema `surefire-test-report.xsd`, a `<testcase>` element declares `@name` and `@time` as required and `@classname`, `@group` and `@timestamp` as optional. **None of those attributes says whether the case passed.** There is no `@status`, no `@result`, and no `<passed>` or `<success>` element anywhere in the schema. What the schema does declare, in a fixed order, is a set of *outcome children* that a case may contain. Three of them are the ordinary, non-repetition outcomes: - **`<failure>`** -- the case ended on an assertion that did not hold. - **`<error>`** -- the case ended on something that was not an assertion. - **`<skipped>`** -- the case was not executed to a verdict. A pass is the absence of all three. The writer emits the `<testcase>` element, writes its attributes, finds no outcome to record, and closes it. That is the whole mechanism. ## Where the absence comes from on the writing side Surefire keeps the mapping from an outcome to the XML tag that represents it in the enum `ReportEntryType`. Its `ERROR`, `FAILURE` and `SKIPPED` rows each name a tag -- `error`, `failure`, `skipped` -- and its `SUCCESS` row maps to an **empty string**. The serializer asks an entry for its tag only when the outcome is not `SUCCESS`; for a green case there is no tag to write, so nothing is written. The absence is not an omission or an optimisation someone bolted on -- it is the tag name itself being empty. ## What a consumer does with it Readers invert the same rule. Allure 2's `JunitXmlPlugin`, which parses these files, decides a case's status by looking for children in order: `<failure>` first, then `<error>`, then `<skipped>`. If none of the three is present -- and the case does not carry a dialect `@status` of `notrun` -- it returns `PASSED`. **Pass is the fallthrough branch**, the thing you get when nothing matched, which is exactly what the writer encoded. | What the file contains | What a reader concludes | |---|---| | `<testcase>` with `<failure>` | the case failed an assertion | | `<testcase>` with `<error>` | the case ended on a non-assertion throwable | | `<testcase>` with `<skipped>` | the case was not run to a verdict | | `<testcase>` with none of the three | the case passed | | no `<testcase>` element at all | nothing -- the reader never sees the case | ## The nuance that catches people: absent outcome, not absent children "A pass is an empty element" is *almost* right, and it is worth being exact, because a passing case can legitimately carry children. `<system-out>` and `<system-err>` are declared on `<testcase>` independently of the outcome children, and a writer may be configured to record captured output for green cases as well as red ones. So a `<testcase>` with a `<system-out>` child and nothing else is still a pass. The rule a reader must apply is therefore: 1. Look for `<failure>`, `<error>` and `<skipped>` specifically -- not for "any child". 2. Treat their combined absence as a pass. 3. Do not treat a self-closed `<testcase/>` as a special case; it is just the common shape of the same thing. ## What the design costs Encoding the majority outcome as an absence keeps the file small -- most cases in a healthy suite are green, and green costs nothing but a one-line element. The price is paid in two places. **A pass and a non-report are indistinguishable at the case level.** If a run is killed halfway, the cases that never executed do not appear as incomplete; they simply are not in the file, or the file is not written at all. Nothing inside the document distinguishes "this passed" from "this was never reached", because both are represented by the lack of a marker. Only comparing what the file contains against what was expected to run recovers that difference, and that comparison lives outside the format. **A truncated file reads as a green file.** A strict XML parser will object to an unclosed root element, but many consumers of these files are tolerant, and a document that ends early after a run of green cases looks much like a document that legitimately ended there. ## Practical takeaways - Never look for a positive pass marker in JUnit-XML; there is none to find. - When writing a converter into this format, emit no outcome child for a pass -- do not invent one, because every existing reader ignores children it does not recognise and still falls through to `PASSED`. - When writing a reader, check for the three outcome elements by name; anything else you find, including `<system-out>`, `<system-err>` and `<properties>`, is not an outcome. - Treat the counts on the enclosing suite and the per-case children as two independent statements about the same run; the file itself does not enforce that they agree.
- If a pass is recorded as an absence, how does a reader tell a passing case from one the writer never got to?It cannot, from the case alone. A run killed part-way produces fewer `<testcase>` elements, or no file, rather than cases marked incomplete -- both a pass and a never-reached case are represented by the lack of a marker. Recovering the difference means comparing what the file holds against what was expected to run.
- Can a passing `<testcase>` carry any children at all?Yes. `<system-out>` and `<system-err>` are declared on `<testcase>` independently of the outcome, and a writer may be configured to record captured output for green cases too. What a passing case never carries is `<failure>`, `<error>` or `<skipped>`.
It works like an attendance sheet that records only absences: a name with nothing beside it means the person turned up. Cheap to keep, but a torn-off corner of the page and a full day of attendance then look identical.
saying these in an interview costs you the question
- Claims a pass is written as a <passed> or <success> element
- Says every <testcase> must carry a @status attribute
- Thinks a passing case must have no children whatsoever
- Assumes a case that never ran looks different from one that passed