skip to content

A Kotlin test class uses @TestInstance(PER_CLASS) and passes locally but fails intermittently in CI. What are the likely causes and how do you diagnose them?

level: seniorimportance: nice to knowfreq 28%

answer

  1. PER_CLASS = one instance = shared mutable fields
  2. Order-dependent tests fail when order changes in CI
  3. Reproduce with MethodOrderer.Random
  4. Reset fields in @BeforeEach; prefer val
  5. Parallel execution races on the shared instance

basics

~10 s

PER_CLASS reuses one instance, so leftover state from one test can affect another. If tests run in a different order in CI, hidden dependencies surface. Reset shared state and pin or remove order assumptions.

solid answer

~40 s

With `PER_CLASS`, JUnit reuses a single instance, so mutable instance fields persist across tests. Intermittent CI failures usually mean **order-dependent state leakage**: a test mutates a field another test silently relies on, and JUnit's default method order (deterministic but unspecified, MethodOrderer.MethodName-like only if configured) can differ across JDK/compiler versions, or parallel execution interleaves tests sharing the one instance. Diagnose by: forcing a fixed order with `@TestMethodOrder(MethodOrderer.Random::class)` to surface the flake locally, adding a `@BeforeEach` that resets every mutable field, checking whether `junit.jupiter.execution.parallel.enabled=true` is racing on shared state, and ensuring `@BeforeAll`-created resources are idempotent/cleaned in `@AfterAll`. The fix is usually: make fields immutable, reset them in `@BeforeEach`, or drop PER_CLASS in favour of a static companion if isolation matters more than ergonomics.

code

kotlin · 9 lines
kotlin
import org.junit.jupiter.api.*

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class OrderSafeTest {
    private val items = mutableListOf<String>()
    @BeforeEach fun clean() { items.clear() }   // reset every shared field
    @Test fun `starts empty`() { assert(items.isEmpty()) }
    @Test fun `adds one`() { items += "a"; assertEquals(1, items.size) }
}

go deeper

for a junior

Recognizes that reused state between tests can cause failures.

for a middle

Adds @BeforeEach resets and knows PER_CLASS shares one instance.

for a senior

Systematically diagnoses with random ordering, audits mutable fields, and checks parallel settings.

for a principal

Establishes anti-flake conventions (immutable fixtures, isolation policy, CI ordering) across the codebase.

## Why PER_CLASS invites flakiness `@TestInstance(TestInstance.Lifecycle.PER_CLASS)` makes JUnit construct **one** instance for the whole class. Every `@Test` shares that instance's fields. If test A leaves a field mutated and test B assumes a clean value, B passes only when A ran first (or didn't run). This is a classic **order-dependent test**. ## Why it shows up in CI but not locally - **Execution order differs.** JUnit's default order is deterministic but *unspecified*; it can vary across JDK versions, Kotlin compiler versions, or class-file layout. CI's toolchain may differ from your laptop. - **Parallel execution.** If `junit.jupiter.execution.parallel.enabled=true`, multiple tests can run concurrently against the **same shared instance**, causing data races on mutable fields. - **Resource bleed.** A `@BeforeAll`-created resource (server, temp dir) accumulates state across tests because it's set up only once. ## Diagnosis checklist ```kotlin import org.junit.jupiter.api.* @TestInstance(TestInstance.Lifecycle.PER_CLASS) @TestMethodOrder(MethodOrderer.Random::class) // surface order dependence class FlakyTest { private var counter = 0 @BeforeEach fun reset() { counter = 0 } // the usual fix @Test fun `increments once`() { counter++; assertEquals(1, counter) } } ``` 1. Run with `MethodOrderer.Random` (or repeat with `@RepeatedTest`) to reproduce the flake locally. 2. Audit every mutable instance field; add `@BeforeEach` resets. 3. Check `junit-platform.properties` for parallel execution; if on, the shared instance is unsafe — isolate state or use `@ResourceLock`. 4. Verify `@AfterAll` actually tears down `@BeforeAll` resources. ## Fixes, in order of preference - **Make fields immutable** (`val`) — best. - **Reset mutable fields in `@BeforeEach`.** - **Use a fresh local in each test** instead of a shared field. - **Drop PER_CLASS** for a `companion object` + `@JvmStatic @BeforeAll`, restoring per-test instance isolation. ## Key APIs/terms `@TestInstance`, `Lifecycle.PER_CLASS`, `@BeforeEach`, `@AfterAll`, `@TestMethodOrder`, `MethodOrderer.Random`, `junit.jupiter.execution.parallel.enabled`, `@ResourceLock`.

  • How would you force the flake to appear locally?
    Run tests in random order (@TestMethodOrder(MethodOrderer.Random::class)) and/or repeat them, so hidden order dependencies surface.
  • Why does parallel execution worsen PER_CLASS?
    Multiple tests run concurrently against the single shared instance, creating data races on its mutable fields unless guarded by @ResourceLock.

It's like a shared whiteboard: it works while one person uses it in a known order, but breaks when others scribble in a different sequence.

saying these in an interview costs you the question

  • Blaming CI environment without considering shared instance state
  • Assuming JUnit guarantees a specific default test order
  • Adding sleeps instead of resetting shared state
  • Ignoring parallel-execution settings as a cause

context

resolving links…