skip to content

When porting tests between TestNG and JUnit Jupiter, which behaviours change silently even though every test compiles and the build stays green?

level: seniorimportance: should knowfreq 35%

answer

  1. instances, skips, labels, threads
  2. one shared instance vs one per method
  3. skip turns into run
  4. suite-file excludes do not travel
  5. exception expectations tighten

basics

~20 s

Instance lifecycle (TestNG shares one instance per class, Jupiter makes one per method), dependency skips, suite-file group excludes and parallel settings, assert argument order, and the scope of exception expectations all change without a compile error.

solid answer

~50 s

Five things change under a green build. First, instances: TestNG runs a class's methods on one shared instance by default, while JUnit Jupiter creates a new instance per method, so state passed through fields disappears one way and leaks the other. Second, dependencies: TestNG skips a dependent test when its prerequisite fails; Jupiter has no such gate, so skips become failures or misleading passes. Third, selection: groups excluded in `testng.xml` run again unless a tag filter replaces the exclude, and classes the suite file never listed may be discovered. Fourth, parallelism: the suite file's `parallel` setting does not travel; Jupiter is sequential until configuration enables it. Fifth, assertions: TestNG's `Assert.assertEquals` takes actual first and Jupiter's takes expected first, so verdicts hold but messages mislabel values, and `expectedExceptions` covers the whole method where `assertThrows` covers one lambda.

code

java · 19 lines
java
import org.testng.annotations.Test;

import static org.testng.Assert.assertNotNull;

public class CartFlowTest {

  private String cartId; // set by one method, read by the next

  @Test
  public void createCart() {
    cartId = CartClient.create();
    assertNotNull(cartId);
  }

  @Test(dependsOnMethods = "createCart")
  public void addItem() {
    CartClient.add(cartId, "sku-1");
  }
}

go deeper

for a junior

Recall the default instance difference: TestNG shares one instance per class, JUnit Jupiter creates one per test method.

for a middle

Explain how dependency skips, suite-file excludes and the parallel setting fail to carry over, and what each looks like in the results.

for a senior

Show the checklist: diff executed test lists, compare skip counts, grep for field-shared state and dependencies, and mutation-check a sample before trusting a green port.

for a principal

Frame which behaviours a port should deliberately change, such as independence and tighter exception scope, versus which it must preserve, and budget verification accordingly.

## Why porting is risky even when everything is green A migration between **TestNG** and **JUnit Jupiter** (the JUnit 5 and JUnit 6 model) is usually judged by two signals: the code compiles and the build is green. Both frameworks share enough vocabulary that annotation-by-annotation translation compiles easily. The danger is in behaviour that the translation does not carry, where tests keep passing while checking less, running in a different order or scope, or running things that were deliberately excluded. ## 1. Test instance lifecycle - **TestNG** by default creates **one instance of a test class** and runs every `@Test` method on it, so an instance field set in one method is visible in the next. - **JUnit Jupiter** by default creates a **new instance per test method**, so instance fields start fresh each time. TestNG to Jupiter: a test that read a field written by an earlier method now sees `null` or a default. It fails if you are lucky; it passes vacuously if it only checks the field when present. Jupiter to TestNG: per-test state such as a list initialised in a field declaration now accumulates across methods, which breaks independence and, under parallel methods, thread safety. ## 2. Dependency skips `@Test(dependsOnMethods = ...)` in TestNG reports a dependent as **SKIPPED** when its prerequisite fails. Jupiter has no dependency annotation; `@TestMethodOrder` only orders. After the port: - a broken first step yields a cascade of **failures** instead of skips, which makes triage noisier; - if the order is lost too, a dependent can run before its prerequisite and pass or fail by accident. ## 3. What gets selected - A `testng.xml` that **excludes** a group (say `slow` or `quarantine`) leaves those tests out. Adding `@Tag("slow")` in Jupiter does nothing until some filter excludes the tag, so those tests **start running**. - A suite file lists the classes it runs; Jupiter's discovery scans for test classes under the build's selection rules rather than a suite file, so classes nobody listed, including stale ones, may start running. - Values from suite-file `@Parameters` have no Jupiter annotation; if a port substitutes hard-coded defaults, every environment silently tests the same target. ## 4. Parallelism and threads The suite file's `parallel` attribute and thread count do not travel. JUnit Jupiter runs sequentially by default and enables parallel execution through configuration parameters. Porting TestNG to Jupiter tends to make the suite slower but safer; porting Jupiter to TestNG with a parallel suite file can expose shared-state races, made worse by TestNG's shared instance under `parallel="methods"`. ## 5. Assertions and exception expectations | Concern | TestNG | JUnit Jupiter | |---|---|---| | `assertEquals` argument order | actual, then expected | expected, then actual | | Exception expectation | `expectedExceptions` on the whole method | `assertThrows` around one lambda | | Message regex | `expectedExceptionsMessageRegExp` (default `.*`) | assert on the returned exception | Swapped `assertEquals` arguments do not change the verdict, since `equals()` is meant to be symmetric, but the failure message labels the expected value as actual and the reverse, which misleads whoever triages it. TestNG's `AssertJUnit` keeps the expected-first order and is a common bridge when moving JUnit code into TestNG. On exceptions, a TestNG method with `expectedExceptions` passes if *any* line of the method body throws that type, including its arrange lines; moving to `assertThrows` tightens the test, and a test that "passed" may now fail honestly. ## 6. Hook scope false friends TestNG's `@BeforeTest` runs once per `<test>` block of the suite file. Mapping it to Jupiter's `@BeforeEach` makes it run before every method; mapping TestNG's `@BeforeClass` to `@BeforeAll` requires `static` unless the per-class lifecycle is used, which tempts people to switch the lifecycle and inherit the sharing problems from section 1. ## A checklist that catches these 1. **Compare test counts and skip counts** per class before and after; skips turning into failures or tests appearing are both signals. 2. **Grep for state shared through fields** and for `dependsOnMethods` before translating. 3. **Replicate every suite-file include and exclude** as an explicit filter, then diff the selected test list. 4. **Decide the parallel setting explicitly** rather than inheriting a default. 5. **Mutation-check a sample**: break the code under test and confirm the ported tests go red. The common thread: compile errors cover only syntax. Each behaviour above lives in a default or in the suite file, which is why a green build is weak evidence that a port preserved what the tests check.

  • Why not switch JUnit Jupiter to the per-class instance lifecycle to make a TestNG port behave the same?
    It restores field sharing, but it also makes every method in the class share state, which is the coupling the port might have removed. It still does not restore TestNG's dependency skips, and with Jupiter's parallel execution enabled the shared instance needs thread-safe fields. Use it deliberately for genuinely ordered flows, not as a blanket compatibility switch.
  • How do you prove a ported suite still selects the same tests as the old testng.xml?
    Produce the list of executed test identifiers from both runs, per class and method, and diff them. Differences reveal excluded groups that now run, unlisted classes that discovery picked up, and data-driven rows that changed in number. Pair that with skip and failure counts, since a dependency skip turning into a failure is invisible in a pass count.

  • TestNG and JUnit Jupiter both create a new instance per test method by default
  • A green build after a port proves the tests still check the same things
  • Adding @Tag("slow") in JUnit Jupiter excludes those tests from the run by itself
  • Moving from expectedExceptions to assertThrows can only make a test more lenient
  • JUnit Jupiter honours the parallel setting from an existing testng.xml