skip to content

When you run `testAggregateTestReport` in CI and some subproject tests fail, what happens to the report and the build, and how do you make the report reliably available?

level: seniorimportance: should knowfreq 30%

answer

  1. failing test fails Test task
  2. default fail-fast halts scheduling
  3. --continue runs all, fails at end
  4. finalizedBy report task
  5. always upload artifact, exit non-zero

basics

~10 s

A failing test fails its Test task, which can stop the build before aggregation runs. Use --continue (or finalizers) so all tests run and the aggregate report is still produced even when something fails.

solid answer

~50 s

Aggregation depends on the result data of each subproject, so to merge results the underlying `Test` tasks generally must execute. If a test fails, that `Test` task fails; by default Gradle stops at the first failure, so later projects' tests may not run and the aggregate report may be incomplete or absent. To get a full report despite failures, run with **`--continue`** so Gradle executes every independent task and only fails the build at the end — now the aggregate report contains all projects. Alternatively, wire reporting as a **finalizer** (`finalizedBy`) so the report task runs even when tests fail. In CI you then always archive the report directory as a build artifact (e.g. `build/reports/tests/test/aggregated-results`) regardless of build outcome, and let the non-zero exit code from the failed tests fail the pipeline stage. Don't suppress failures with `ignoreFailures = true` just to get a report — that hides red builds.

code

bash · 3 lines
bash
# Run all subproject tests even if some fail, then aggregate,
# while still exiting non-zero so CI marks the stage red.
./gradlew testAggregateTestReport --continue

go deeper

for a junior

Know that a failing test can stop the build before the report is made, and --continue helps run everything.

for a middle

Explain default fail-fast scheduling and that --continue runs all tasks then fails at the end.

for a senior

Combine --continue/finalizers with unconditional artifact upload and an honest non-zero exit; reject ignoreFailures.

for a principal

Define a team CI convention for reporting on failure, balancing fast feedback (fail-fast) against complete reports (continue) per pipeline stage.

## How failures propagate The aggregate report is produced from each project's binary **test results**. Those results come from running the `Test` tasks. A failing test makes its `Test` task **fail**. By default Gradle uses **fail-fast scheduling**: on the first task failure it stops scheduling new tasks (those not already in flight). So if `:service-a:test` fails, `:service-b:test` might never run, and `testAggregateTestReport` — which depends on both — won't execute, leaving you with no combined report exactly when you need it most. ## Getting a complete report anyway ### Option 1 — `--continue` ```bash ./gradlew testAggregateTestReport --continue ``` `--continue` tells Gradle to run **every** task that isn't blocked by a failed dependency, deferring the build failure to the end. All subproject tests run, all result data exists, and aggregation can render the full report. The build still **exits non-zero** because tests failed — which is what you want. ### Option 2 — finalizers ```kotlin tasks.withType<Test>().configureEach { finalizedBy(":test-report:testAggregateTestReport") } ``` Finalizers run even if the finalized task failed, so the report is generated after tests regardless of outcome. ## CI artifact pattern Whatever the build result, **always archive** the report so engineers can open it: ```yaml # pseudo-CI - run: ./gradlew testAggregateTestReport --continue - uses: upload-artifact # runs even on failure with: path: test-report/build/reports/tests/test/aggregated-results if: always() ``` ## Anti-pattern: ignoreFailures Setting `test { ignoreFailures = true }` makes the `Test` task succeed even when tests fail, so aggregation runs — but it also makes the **build green on red tests**, masking real failures. Prefer `--continue` + always-upload, keeping the exit code honest. ## Summary - Failures fail the `Test` task and, by default, halt scheduling. - `--continue` runs everything then fails at the end → full report + honest exit code. - Finalizers (`finalizedBy`) also force report generation. - Always archive the report unconditionally in CI. - Never use `ignoreFailures` just to obtain a report.

  • Why not just set `ignoreFailures = true` to always get the report?
    That makes the Test task succeed on failing tests, turning a red build green and hiding real failures. Use `--continue` so the report is complete but the build still fails.
  • How do you ensure the report is uploaded even when the build fails?
    Archive the report directory in CI with an `if: always()`-style condition so the artifact step runs regardless of the build's exit code.
  • What does a finalizer give you over `--continue`?
    `finalizedBy` forces the report task to run after each Test task even on failure, baked into the build logic rather than relying on the invocation flag.

saying these in an interview costs you the question

  • Recommending `ignoreFailures = true` as the way to always get a report.
  • Assuming the report is always produced even when an early test task fails under default scheduling.
  • Forgetting that the build should still exit non-zero on test failure.

context

resolving links…