skip to content

Why does test report aggregation merge binary result data rather than the generated HTML reports, and what attribute/variant mechanism makes that work?

level: seniorimportance: should knowfreq 35%

answer

  1. binary results = source of truth
  2. HTML is just a rendering
  3. outgoing variant + attributes
  4. resolvable vs consumable
  5. render once from the union

basics

~20 s

HTML is already-rendered output that can't be cleanly merged. Gradle instead collects each project's raw binary test results via a dedicated outgoing variant selected by attributes, then renders one fresh report from all of them.

solid answer

~50 s

A `Test` task writes both binary results (`build/test-results/...`) and an HTML report. HTML is a presentation artifact — concatenating ten index.html files would not give a coherent single report. The aggregation plugin instead consumes the **binary result data**, which is the canonical source the HTML renderer reads. Each producing project exposes its results through an **outgoing variant** carrying attributes such as the category/usage marking it as test-results plus the test-suite name and type. The aggregating project's `testReportAggregation` configuration is **resolvable** and requests matching attributes, so Gradle's variant-aware resolution picks exactly the test-results variant from each dependency — no file paths, no globbing. The `AggregateTestReport` task then feeds all collected binary results into the standard report renderer to produce one combined HTML report. This keeps aggregation correct, incremental, and cache-friendly because it operates on declared inputs resolved through the dependency graph.

code

kotlin · 7 lines
kotlin
// Producer side (added by java/jvm-test-suite): a consumable variant
// tagged category=verification, verificationType=test-results.
// Consumer side (added by test-report-aggregation): a resolvable
// 'testReportAggregation' configuration requesting that variant.
dependencies {
    testReportAggregation(project(":service-a")) // resolved by attributes
}

go deeper

for a junior

It's enough to know it merges raw results, not HTML, and you don't give it file paths.

for a middle

Explain binary results are the source of truth and the plugin re-renders one report from them.

for a senior

Articulate variant-aware resolution: attribute-matched outgoing variants and a resolvable consuming configuration.

for a principal

Discuss how this model underpins reproducible, cacheable cross-project reporting and how custom verification variants could extend it.

## Binary results vs HTML A `Test` task produces two things: - **Binary result data** under `build/test-results/<suite>/` — the machine-readable record of every test's outcome, timing, and failure detail. This is the *source of truth*. - An **HTML report** under `build/reports/tests/<suite>/` — a *rendering* of that data for humans. Merging already-rendered HTML is fragile: you'd be stitching presentation, deduplicating CSS, and losing the ability to recompute totals. Merging the binary data is clean — you just render once from the union. ## Variant-aware resolution Gradle dependencies are resolved by matching **attributes** between what a consumer *requests* and what producers *offer*: - A producing project publishes an **outgoing (consumable) configuration** — a *variant* — tagged with attributes that say "this is test result data for suite `test`." - The aggregating project's `testReportAggregation` configuration is **resolvable** and carries an attribute request that matches that variant. Because matching is attribute-based, the consumer never names a file. It says "give me the test-results variant of these projects," and Gradle returns the right artifacts even across the build. ## Resolvable vs consumable Gradle splits configuration roles: - **consumable** (`canBeConsumed = true`, `canBeResolved = false`) — what a project *exposes* to others (the producer's test-results variant). - **resolvable** (`canBeResolved = true`, `canBeConsumed = false`) — what a project *reads* (the aggregator's `testReportAggregation`). This separation is why aggregation is robust: the plugin wires the resolvable side and the `java`/`jvm-test-suite` plugins wire the consumable side. ## Why it matters operationally - **Correctness**: totals are recomputed from raw data. - **Incrementality / caching**: inputs are declared artifacts resolved from the graph, so up-to-date checks and the build cache work. - **No brittle paths**: refactoring a project's layout doesn't break aggregation. ```kotlin // Conceptually, the resolvable side the plugin sets up: configurations { // 'testReportAggregation' is resolvable and requests // the test-results variant (attributes: category=verification, // verificationType=test-results, testSuiteName=...) } ```

  • What's the difference between a consumable and a resolvable configuration here?
    Consumable = the producing project exposes its test-results variant to others; resolvable = the aggregator reads/resolves those variants. One offers, the other consumes.
  • Why is this approach more cache-friendly than copying HTML?
    Inputs are declared artifacts resolved from the dependency graph, so Gradle's up-to-date checks and build cache can track them; copied HTML would be opaque, untracked output.

saying these in an interview costs you the question

  • Saying the plugin concatenates or iframes the HTML reports.
  • Claiming you must point the aggregator at result directories by path — resolution is attribute-driven.
  • Confusing resolvable and consumable roles.

context

resolving links…