skip to content

You want one test suite to run its tests across several JDK versions (a JDK matrix). How would you model that with suite targets and toolchains, and what are the trade-offs?

level: seniorimportance: should knowfreq 22%

answer

  1. one target/suite per JDK, each with its own launcher
  2. javaLauncher = launcherFor { languageVersion }
  3. compile to lowest release; run on each JDK
  4. wire all targets into check
  5. CI strategy matrix is often the cheaper alternative

basics

~20 s

Give the suite multiple targets, each with a Test task pinned to a different JavaLanguageVersion launcher. Iterate the JDK list, register a target per version, and set javaLauncher on each target's testTask so the same tests run on each JDK.

solid answer

~40 s

A suite normally has one target, but you can drive several. Conceptually you want one `Test` task per JDK, each forked on a different toolchain launcher, all running the same compiled tests. In practice you create distinct suites/targets (e.g. `testJdk17`, `testJdk21`) or program the `targets` so each gets a `javaLauncher` from `javaToolchains.launcherFor { languageVersion = JavaLanguageVersion.of(v) }`, then wire them into `check`. The benefit is genuine cross-JDK verification of behavior (GC, reflection, preview features) in one build. Trade-offs: build time multiplies by the number of JDKs; each JDK must be installed or auto-provisioned; compilation usually targets the lowest supported `release` so the bytecode is valid everywhere; and you must aggregate reports/coverage across targets. Many teams push the matrix to CI (a job per JDK) instead, keeping the local build single-JDK for speed.

code

kotlin · 20 lines
kotlin
val jdks = listOf(17, 21)
testing {
    suites {
        jdks.forEach { v ->
            register("testJdk$v", JvmTestSuite::class) {
                useJUnitJupiter()
                dependencies { implementation(project()) }
                targets {
                    all {
                        testTask.configure {
                            javaLauncher = javaToolchains.launcherFor {
                                languageVersion = JavaLanguageVersion.of(v)
                            }
                        }
                    }
                }
            }
        }
    }
}

go deeper

for a junior

Recognize that you'd need multiple Test tasks, each pinned to a different JDK toolchain.

for a middle

Describe registering a target/suite per JDK with its own javaLauncher and wiring them into check.

for a senior

Reason about compile-release vs run-JDK, report/coverage aggregation, and the build-time/provisioning trade-offs versus a CI matrix.

for a principal

Decide org policy: in-build matrix for published libraries' multi-JDK certification vs. CI-job matrix for apps; standardize via convention plugin and provisioning governance.

## The goal Run the *same* tests on multiple JDKs to catch version-specific behavior: differences in the JIT/GC, reflection and module access, deprecation removals, or preview features. The JVM Test Suite + toolchains give you the building blocks; the matrix itself you assemble. ## Modeling with targets / suites Each **target** of a suite owns one `Test` task. The cleanest, most readable approach is one suite (or one target) per JDK, each pinned to a toolchain launcher: ```kotlin val jdks = listOf(17, 21) testing { suites { jdks.forEach { v -> register("testJdk$v", JvmTestSuite::class) { useJUnitJupiter() dependencies { implementation(project()) } targets { all { testTask.configure { javaLauncher = javaToolchains.launcherFor { languageVersion = JavaLanguageVersion.of(v) } } } } } } } } tasks.named("check") { dependsOn(testing.suites.withType(JvmTestSuite::class).map { it.targets }) } ``` Each suite produces a `testJdk17` / `testJdk21` task forked on its own JDK. (The `targets { }` DSL also supports multiple targets within a single suite for the same idea; either way the key is a per-target `javaLauncher`.) ## Compile vs. run in a matrix - **Run**: set `javaLauncher` per target (above). - **Compile**: to be runnable on the *lowest* JDK in the matrix, compile the suite's classes with a matching bytecode level, e.g. `options.release = 17` on the suite's `JavaCompile` task, or compile each variant with its own `javaCompiler`. If you compile with class-file version 65 (JDK 21) and try to run on JDK 17, you get `UnsupportedClassVersionError`. ## Aggregating results Each target has its own test report and (if JaCoCo is wired) its own exec data. To get a unified view, aggregate: combine exec files into one JaCoCo report, or use the `test-report-aggregation` / `jacoco-report-aggregation` plugins, and make `check` depend on all targets so CI fails if any JDK fails. ## Trade-offs and when to use it | Concern | In-build matrix | CI-job matrix | |---|---|---| | Build time | N x slower locally | parallel across agents | | JDK availability | needs all JDKs installed/provisioned | each agent has one JDK | | Reproducibility | one Gradle invocation | depends on CI config | | Local feedback | slow | fast (single JDK) | Because an in-build matrix multiplies local build time and requires every JDK present, many teams keep the local build single-JDK and express the matrix as a **CI strategy matrix** (one job per JDK, each running the normal `check`). The in-build approach shines when you need a *single artifact/report* certifying multi-JDK support, or for libraries published for many JDKs. ## Pitfalls - Forgetting to wire the new targets into `check` — they silently never run. - Toolchain provisioning failing on agents without the JDK and without a download repository. - Mismatched compile `release` vs. run JDK -> `UnsupportedClassVersionError`. - Coverage/report fragmentation if you don't aggregate.

  • Why might you compile the matrix suite with options.release set to the lowest JDK in the matrix?
    So the produced bytecode's class-file version is accepted by every JDK in the matrix. If you compile to a newer release and run on an older JDK you get UnsupportedClassVersionError.
  • What is a common reason a newly added matrix target never runs?
    It was not wired into the check (or build) lifecycle, so nothing depends on its Test task. You must add the target's task as a dependency of check.
  • When is a CI strategy matrix preferable to an in-build matrix?
    When you want fast local feedback and parallelism: each CI agent runs the normal check on one JDK, avoiding installing every JDK locally and avoiding N-times-slower local builds.

saying these in an interview costs you the question

  • Assuming a single suite automatically runs on all installed JDKs without configuring targets.
  • Compiling to a newer release than the lowest target JDK and expecting it to run.
  • Forgetting to aggregate per-target reports/coverage, leaving fragmented results.

context

resolving links…