skip to content

How do you measure code coverage for integration tests run by the Failsafe plugin, separately from unit tests?

level: seniorimportance: should knowfreq 40%

answer

  1. Failsafe = *IT, verify phase
  2. separate destFile jacoco-it.exec
  3. separate propertyName failsafeArgLine
  4. @{} late eval
  5. prepare-agent-integration / report-integration
  6. merge goal combines exec

basics

~10 s

Add a second prepare-agent execution that writes to a separate property (e.g. failsafeArgLine) and a separate destFile (jacoco-it.exec), reference that property in Failsafe's argLine, then run a report on that exec during verify.

solid answer

~40 s

Failsafe runs integration tests in the integration-test/verify phases, separate from Surefire's unit tests. To get IT coverage you create a second JaCoCo `prepare-agent` execution with `<destFile>target/jacoco-it.exec</destFile>` and `<propertyName>failsafeArgLine</propertyName>` so it doesn't collide with the unit-test agent. Then in the failsafe-maven-plugin config set `<argLine>@{failsafeArgLine}</argLine>` (or just let Failsafe read its own argLine property). Bind a `report` execution to `verify` pointing `<dataFile>` at `jacoco-it.exec`. The reason for the separate property and destFile is that the default `argLine`/`jacoco.exec` are already used by the unit-test run; reusing them would either clobber the unit data or attach the wrong agent options. Optionally use `merge` to combine unit + IT exec into one report.

code

xml · 10 lines
xml
<execution>
  <id>prepare-it</id>
  <phase>pre-integration-test</phase>
  <goals><goal>prepare-agent-integration</goal></goals>
  <configuration>
    <destFile>${project.build.directory}/jacoco-it.exec</destFile>
    <propertyName>failsafeArgLine</propertyName>
  </configuration>
</execution>
<!-- in failsafe config: <argLine>@{failsafeArgLine}</argLine> -->

go deeper

for a junior

Knows Failsafe runs integration tests separately from unit tests.

for a middle

Can add a second prepare-agent for ITs with its own destFile.

for a senior

Isolates unit vs IT coverage via propertyName/destFile, wires @{} into Failsafe, and merges exec files for a combined number.

for a principal

Designs the unit/IT coverage strategy (separate vs merged gates) consistently across modules and pipelines.

## The two test plugins - **Surefire** runs **unit tests** (`*Test.java`) in the `test` phase. - **Failsafe** runs **integration tests** (`*IT.java`) across `pre-integration-test` -> `integration-test` -> `post-integration-test` -> `verify`. Failsafe is designed so a failing IT doesn't abort before cleanup (failures are checked in `verify`). ## Why a naive single prepare-agent isn't enough A single `prepare-agent` writes to property `argLine` and destFile `target/jacoco.exec`. That serves the unit run. The IT run is a separate forked JVM in a later phase; if you reuse the same destFile the second agent may overwrite or you can't distinguish unit vs IT coverage, and Failsafe needs its own argLine value. ## The separate-agent pattern Create a second prepare-agent bound before integration tests, isolated by `<propertyName>` and `<destFile>`: ```xml <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.12</version> <executions> <!-- unit --> <execution> <id>prepare-unit</id> <goals><goal>prepare-agent</goal></goals> <configuration> <destFile>${project.build.directory}/jacoco-ut.exec</destFile> <propertyName>surefireArgLine</propertyName> </configuration> </execution> <!-- integration --> <execution> <id>prepare-it</id> <phase>pre-integration-test</phase> <goals><goal>prepare-agent-integration</goal></goals> <configuration> <destFile>${project.build.directory}/jacoco-it.exec</destFile> <propertyName>failsafeArgLine</propertyName> </configuration> </execution> <execution> <id>report-it</id> <phase>verify</phase> <goals><goal>report-integration</goal></goals> </execution> </executions> </plugin> ``` Note `prepare-agent-integration` and `report-integration` are convenience goals that default destFile to `jacoco-it.exec`. ## Wiring the property into Failsafe ```xml <plugin> <artifactId>maven-failsafe-plugin</artifactId> <configuration> <argLine>@{failsafeArgLine}</argLine> </configuration> <executions> <execution> <goals><goal>integration-test</goal><goal>verify</goal></goals> </execution> </executions> </plugin> ``` Use `@{...}` (late evaluation) so the property exists when Failsafe forks. ## Combining unit + IT coverage Use the JaCoCo **`merge`** goal to combine `jacoco-ut.exec` + `jacoco-it.exec` into one `.exec`, then run `report` on the merged file for a single number. This matters when a line is only covered by an IT but not a unit test. ## Common pitfalls - Forgetting `@{}` so Failsafe forks before the property is set -> no IT coverage. - Reusing the default `argLine` for both -> the second prepare-agent clobbers the first. - Failsafe failures not surfacing because you only run `mvn integration-test` without `verify`.

  • Why use a distinct propertyName and destFile for the IT agent instead of the defaults?
    The default argLine/jacoco.exec already serve the unit run; reusing them clobbers unit data and gives Failsafe the wrong agent options. Separate names keep unit and IT coverage independent.
  • How would you report a single combined coverage figure across unit and IT tests?
    Use the JaCoCo merge goal to combine jacoco-ut.exec and jacoco-it.exec into one exec file, then run report on the merged file.
  • What does the @{...} syntax in Failsafe's argLine accomplish?
    Late property resolution — it defers evaluating failsafeArgLine until Failsafe forks the JVM, after prepare-agent-integration has set it.

saying these in an interview costs you the question

  • Reusing the default argLine/jacoco.exec for both unit and IT runs
  • Forgetting @{} so the property is empty at fork time
  • Assuming Surefire runs ITs
  • Running only integration-test without verify so failures and reports are skipped

context

resolving links…