You own the CI strategy for a Spring Boot service deployed as a native image. How do you use `nativeTest` cost-effectively?
answer
- layered CI: JVM per-commit, nativeTest nightly/release gate
- full native compile = slow + RAM-heavy
- exclude Mockito/bytecode tests via tags
- consolidate contexts to cut AOT build cost
- failures → RuntimeHints; skip nativeTest if JVM-only deploy
basics
~20 sRun fast JVM tests on every commit for feedback; run nativeTest on a subset (excluding mock-heavy tests) as a scheduled/pre-release gate because each run does a slow, memory-heavy native compilation. Native tests exist to validate metadata, not to replace the JVM suite.
solid answer
~40 sTreat `nativeTest` as a specialized gate, not the default. JVM `test` gives fast per-commit feedback; `nativeTest` triggers a full GraalVM compilation — minutes and gigabytes of RAM — so run it nightly and/or as a required check before a native release, not on every push. Curate what runs natively: exclude tests that depend on Mockito or other bytecode-generating tools (they don't work closed-world) and consolidate `@SpringBootTest` configurations, since each distinct context multiplies AOT generation and build time. Keep a representative slice that exercises the reflection/resource/proxy paths your app and its libraries use, so missing-hint regressions are caught before production. Feed failures back into `RuntimeHintsRegistrar` fixes or upgraded library metadata. Also weigh CI runner cost/parallelism and cache the native toolchain. If you don't actually ship native, skip nativeTest entirely.
code
kotlin · 11 lines// Keep mock-heavy tests out of the native run while retaining them on the JVM.
// Tag them:
// @Tag("native-unsupported") // on Mockito/@MockBean-based tests
// build.gradle.kts — exclude that tag from the native test task
tasks.named<Test>("nativeTest") {
useJUnitPlatform { excludeTags("native-unsupported") }
}
// JVM `test` still runs everything for fast per-commit feedback.
// nativeTest runs nightly / as a pre-release required check.go deeper
Know native tests are slow and not run on every commit.
Layer JVM per-commit vs native scheduled, and know Mockito tests must be excluded.
Design tagging/exclusion, context consolidation, and a hint-fix feedback loop.
Own the cost/benefit tradeoff, runner sizing, release-gating, and the decision to include or skip native tests based on deployment reality.
## Framing: nativeTest is a gate, not a suite The purpose of `nativeTest` is narrow: prove your tests pass under GraalVM's **closed-world** model, catching **missing reachability metadata** (reflection/proxy/resource/serialization hints) and **AOT test-context** problems before they hit the production native binary. It is *not* your primary correctness suite — the JVM `test` task already covers logic, and runs in seconds. So the strategy is layered. ## Cost characteristics that drive the design - **Full native compilation per run**: minutes of wall-clock, high memory (often multiple GB). This dwarfs JVM test time. - **Distinct test contexts multiply cost**: Spring's `processTestAot`/`TestContextAotGenerator` generates an `ApplicationContextInitializer` per unique `MergedContextConfiguration`; more contexts → more generated code → longer builds and bigger binary. - **Mockito & bytecode mockers don't work** natively, so a naive "run everything" will fail for reasons unrelated to production. ## A pragmatic pipeline 1. **Per-commit / PR**: `./gradlew test` (JVM) — fast feedback, blocks merges. Optionally `processAot`/`nativeCompile` smoke to catch build breaks without running the full native suite. 2. **Nightly (or pre-release)**: `./gradlew nativeTest` on a **curated subset** — tag or exclude mock-heavy tests, keep tests that exercise real reflection/resource/proxy paths (JSON mapping, JPA entities, HTTP clients, i18n resources). This is where missing-hint regressions are caught. 3. **Release gate**: a required `nativeTest` (or at least `nativeCompile` + a targeted native smoke test) before publishing a native image, so no release ships with an untested closed-world gap. ## Tactics to control cost - **Consolidate contexts**: share identical `@SpringBootTest` configuration so contexts cache/reuse; avoid gratuitous per-test property/profile variations. - **Exclude the unsupported**: use JUnit tags (e.g., `@Tag("native-unsupported")`) or task test filters to skip Mockito-based tests in the native run while keeping them in the JVM run. - **Cache / warm the toolchain**: cache GraalVM, dependencies, and any reachability-metadata; use capable runners (native builds are CPU/RAM hungry). - **Parallelize** across modules where possible, and fail fast. ## Feedback loop on failures A native test failure usually means a **missing hint**, not broken behavior. Resolve by: - adding a `RuntimeHintsRegistrar` (wired via `@ImportRuntimeHints`) or `@RegisterReflectionForBinding` for binding types, - upgrading a library to a version that ships GraalVM reachability metadata, or - supplying explicit `reflect-config.json` / resource-config metadata. Then re-run the native gate. Over time these hints accumulate and native runs stabilize. ## When to skip entirely If the service is only ever deployed on the JVM, `nativeTest` buys nothing — drop it and save the CI budget. Introduce it only when native deployment is real or planned. ## Organizational note Because native builds are heavy, be deliberate about **runner cost** (self-hosted vs. cloud, memory sizing). Document that native failures are typically metadata tasks so on-call engineers don't chase phantom logic bugs. The overall message: JVM suite for correctness and speed; `nativeTest` as a scoped, scheduled safety net for the closed-world runtime.
- How would you keep native test build times from growing as the suite grows?Consolidate `@SpringBootTest` configurations so distinct contexts (and their AOT-generated initializers) stay few, run only a representative native subset, exclude mock-heavy tests, cache the toolchain, and use adequately sized runners. The dominant cost is native compilation and per-distinct-context AOT generation.
- When would you argue against adding nativeTest to CI at all?When the service is deployed only on the JVM. Native tests validate closed-world behavior that never runs in production, so they add slow, expensive builds with no payoff. Add them only when native deployment is real or on the roadmap.
- Running nativeTest on every commit as the default gate
- Trying to run Mockito-heavy suites natively and blaming the app
- Ignoring that distinct test contexts drive native build cost
- Adding nativeTest to a JVM-only deployment for no benefit
- Treating native failures as logic bugs rather than missing metadata