skip to content

You own the CI strategy for a Spring Boot service deployed as a native image. How do you use `nativeTest` cost-effectively?

level: principalimportance: nice to knowfreq 28%

answer

  1. layered CI: JVM per-commit, nativeTest nightly/release gate
  2. full native compile = slow + RAM-heavy
  3. exclude Mockito/bytecode tests via tags
  4. consolidate contexts to cut AOT build cost
  5. failures → RuntimeHints; skip nativeTest if JVM-only deploy

basics

~20 s

Run fast JVM tests on every commit for feedback; run nativeTest on a subset (excluding mock-heavy tests) as a scheduled/pre-release gate because each run does a slow, memory-heavy native compilation. Native tests exist to validate metadata, not to replace the JVM suite.

solid answer

~40 s

Treat `nativeTest` as a specialized gate, not the default. JVM `test` gives fast per-commit feedback; `nativeTest` triggers a full GraalVM compilation — minutes and gigabytes of RAM — so run it nightly and/or as a required check before a native release, not on every push. Curate what runs natively: exclude tests that depend on Mockito or other bytecode-generating tools (they don't work closed-world) and consolidate `@SpringBootTest` configurations, since each distinct context multiplies AOT generation and build time. Keep a representative slice that exercises the reflection/resource/proxy paths your app and its libraries use, so missing-hint regressions are caught before production. Feed failures back into `RuntimeHintsRegistrar` fixes or upgraded library metadata. Also weigh CI runner cost/parallelism and cache the native toolchain. If you don't actually ship native, skip nativeTest entirely.

code

kotlin · 11 lines
kotlin
// Keep mock-heavy tests out of the native run while retaining them on the JVM.
// Tag them:
//   @Tag("native-unsupported")  // on Mockito/@MockBean-based tests

// build.gradle.kts — exclude that tag from the native test task
tasks.named<Test>("nativeTest") {
    useJUnitPlatform { excludeTags("native-unsupported") }
}

// JVM `test` still runs everything for fast per-commit feedback.
// nativeTest runs nightly / as a pre-release required check.

go deeper

for a junior

Know native tests are slow and not run on every commit.

for a middle

Layer JVM per-commit vs native scheduled, and know Mockito tests must be excluded.

for a senior

Design tagging/exclusion, context consolidation, and a hint-fix feedback loop.

for a principal

Own the cost/benefit tradeoff, runner sizing, release-gating, and the decision to include or skip native tests based on deployment reality.

## Framing: nativeTest is a gate, not a suite The purpose of `nativeTest` is narrow: prove your tests pass under GraalVM's **closed-world** model, catching **missing reachability metadata** (reflection/proxy/resource/serialization hints) and **AOT test-context** problems before they hit the production native binary. It is *not* your primary correctness suite — the JVM `test` task already covers logic, and runs in seconds. So the strategy is layered. ## Cost characteristics that drive the design - **Full native compilation per run**: minutes of wall-clock, high memory (often multiple GB). This dwarfs JVM test time. - **Distinct test contexts multiply cost**: Spring's `processTestAot`/`TestContextAotGenerator` generates an `ApplicationContextInitializer` per unique `MergedContextConfiguration`; more contexts → more generated code → longer builds and bigger binary. - **Mockito & bytecode mockers don't work** natively, so a naive "run everything" will fail for reasons unrelated to production. ## A pragmatic pipeline 1. **Per-commit / PR**: `./gradlew test` (JVM) — fast feedback, blocks merges. Optionally `processAot`/`nativeCompile` smoke to catch build breaks without running the full native suite. 2. **Nightly (or pre-release)**: `./gradlew nativeTest` on a **curated subset** — tag or exclude mock-heavy tests, keep tests that exercise real reflection/resource/proxy paths (JSON mapping, JPA entities, HTTP clients, i18n resources). This is where missing-hint regressions are caught. 3. **Release gate**: a required `nativeTest` (or at least `nativeCompile` + a targeted native smoke test) before publishing a native image, so no release ships with an untested closed-world gap. ## Tactics to control cost - **Consolidate contexts**: share identical `@SpringBootTest` configuration so contexts cache/reuse; avoid gratuitous per-test property/profile variations. - **Exclude the unsupported**: use JUnit tags (e.g., `@Tag("native-unsupported")`) or task test filters to skip Mockito-based tests in the native run while keeping them in the JVM run. - **Cache / warm the toolchain**: cache GraalVM, dependencies, and any reachability-metadata; use capable runners (native builds are CPU/RAM hungry). - **Parallelize** across modules where possible, and fail fast. ## Feedback loop on failures A native test failure usually means a **missing hint**, not broken behavior. Resolve by: - adding a `RuntimeHintsRegistrar` (wired via `@ImportRuntimeHints`) or `@RegisterReflectionForBinding` for binding types, - upgrading a library to a version that ships GraalVM reachability metadata, or - supplying explicit `reflect-config.json` / resource-config metadata. Then re-run the native gate. Over time these hints accumulate and native runs stabilize. ## When to skip entirely If the service is only ever deployed on the JVM, `nativeTest` buys nothing — drop it and save the CI budget. Introduce it only when native deployment is real or planned. ## Organizational note Because native builds are heavy, be deliberate about **runner cost** (self-hosted vs. cloud, memory sizing). Document that native failures are typically metadata tasks so on-call engineers don't chase phantom logic bugs. The overall message: JVM suite for correctness and speed; `nativeTest` as a scoped, scheduled safety net for the closed-world runtime.

  • How would you keep native test build times from growing as the suite grows?
    Consolidate `@SpringBootTest` configurations so distinct contexts (and their AOT-generated initializers) stay few, run only a representative native subset, exclude mock-heavy tests, cache the toolchain, and use adequately sized runners. The dominant cost is native compilation and per-distinct-context AOT generation.
  • When would you argue against adding nativeTest to CI at all?
    When the service is deployed only on the JVM. Native tests validate closed-world behavior that never runs in production, so they add slow, expensive builds with no payoff. Add them only when native deployment is real or on the roadmap.

saying these in an interview costs you the question

  • Running nativeTest on every commit as the default gate
  • Trying to run Mockito-heavy suites natively and blaming the app
  • Ignoring that distinct test contexts drive native build cost
  • Adding nativeTest to a JVM-only deployment for no benefit
  • Treating native failures as logic bugs rather than missing metadata

context

resolving links…