Result Consolidation

A pass is written as a <testcase> element carrying no outcome child at all. Only <failure>, <error> and <skipped> mark a non-pass, so a reader concludes a case passed from their absence.

<flakyFailure> records a failed attempt of a case that eventually passed, so the build stays green. A <failure> plus <rerunFailure> records a case that failed on every attempt, so the build goes red.

A <testcase> requires only @name and @time; @classname, @group and @timestamp are optional. The identity the file declares is therefore @classname plus @name - and a writer may legally omit the @classname half of that pair.

Nothing validates these files. JUnit 5's legacy writer emits hostname unconditionally, falling back to the literal <unknown host>; the Surefire schema declares no such attribute, so the file fails that schema and readers, which never validate, consume it anyway.

No single official schema governs this format. The only published one, surefire-test-report.xsd, has a single root, testsuite, and declares no testsuites wrapper; other writers emit that wrapper anyway, so readers dispatch on whichever root they find.

A -container.json file holds Allure's TestResultContainer: setup in befores, teardown in afters, and a children list naming by uuid the results it wraps. The test bodies themselves live in separate -result.json files.

The bytes live in a file of their own in the results directory, named with a random UUID, the suffix -attachment, and the file's extension. The JSON entry beside the test holds only a pointer: name, source and type.

Labels are name-and-value pairs the report groups and filters by, such as epic, feature, suite, owner and severity. Links point outside the report: each holds a name, a url and a type, which is issue, tms or custom.

allure.properties is a classpath resource the test run reads to configure Allure's writer. environment.properties is a file dropped into the results directory that only the report generator reads, to display name and value rows about the run.

The reader joins on uuid. Every container's befores and afters are folded onto each child uuid it names, so the report's test result gains beforeStages, testStage and afterStages built from those fixtures plus the case's own steps.

retriesCount is how many earlier attempts were folded into this result, so it excludes the attempt on screen; a test executed twice reads one. retriesStatusChange is true when any earlier attempt ended in a different status.

The previous report's history directory is missing. Allure 2 keeps no database: it builds trends only from a history folder already present in the results directory it reads, so you copy that folder in from the last report before generating.

Flakiness is a boolean on StatusDetails, set with setFlaky(...), not a status value. The result keeps one of Status's four values - FAILED, BROKEN, PASSED or SKIPPED - and the flag rides alongside it.

testCaseId identifies the logical test case - one value for a method however many rows it runs. historyId identifies one trend series: that case plus the parameter values it ran with. Two rows share testCaseId but not historyId.

RetryPlugin buckets every parsed result by the retry hash it derives, keeps the attempt with the newest start time as the visible one, hides the rest, and hangs them off the survivor as a retries block.

message is the shared failure text the group formed around, matchedTests counts the test items in it, index is the identifier the analyzer gave the group, and launchId scopes the cluster to a single launch.

One externalSystemIssues entry records where a tracker item lives: ticketId, btsUrl, btsProject, a direct url, plus submitDate and pluginName. It carries no ticket status. Attaching one labels the failure without changing its defect type.

TO_INVESTIGATE is ReportPortal's undecided defect group, and every new failure is filed there automatically. PRODUCT_BUG, AUTOMATION_BUG, SYSTEM_ISSUE and NO_DEFECT each record a decision already taken, by a person or by the auto-analyzer.

categories.json is a rule file placed in the Allure results directory. Each entry names one bucket and lists the conditions a result must meet to land in it: a message pattern, a trace pattern, matched statuses and a flaky flag.

removeNumbers strips digits from failure messages before they are compared. Left off, ids and counts split one problem into many clusters; turned on, failures whose only difference was a numeric code merge into one.

A report-level quality gate is a threshold checked against the whole aggregated report, not against one runner's outcome. Allure computes it in the 3.x line only, through its quality-gate command, which exits non-zero when a rule is breached.

allure.results.directory names the local folder the Allure adaptor writes its per-test result files into during the run; it defaults to allure-results. The upload step is a plain directory copy, so if it reads a different path it pushes nothing.

allure generate turns a results directory into a static report directory, by default allure-report, whose entry point is index.html. That page loads its data over HTTP, so opened as a local file it renders empty; serve the directory instead.

Allure 2 writes one JSON file per panel into the generated report's widgets directory at generate time. The file widgets/summary.json holds the report name, a five-status statistic with a derived total, and a time block for the run.

In Allure 2, duration is the run's wall-clock span, the latest stop minus the earliest start. sumDuration adds up each counted result's own duration. The duration-trend tile plots the wall-clock span, not the summed test time.