A TestNG class holds one SoftAssert as a field, and later test methods fail with messages from checks they never made; why, and how do you fix it?
answer
- how many instances run the methods
- what assertAll() leaves behind
- an unsynchronised map underneath
- one collector per test method
basics
~20 sTestNG runs all of a class's test methods on one instance by default, so a SoftAssert field is shared, and assertAll() never clears what it recorded. Later methods rethrow earlier failures. Create a new SoftAssert inside each test method.
solid answer
~50 sBy default TestNG creates one instance of a test class and runs every `@Test` method on it, so a `SoftAssert` field is a single shared collector. `assertAll()` throws when anything is recorded but **does not clear** the list, so once one method records a failure, every later method that calls `assertAll()` on that field fails with the old message, even when its own checks pass. Running the 'failing' method alone turns it green, which is the tell. Under `parallel="methods"` it gets worse: methods of the same instance run on different threads and write into the same unsynchronised map, so failures can be lost or land in the wrong test. The fix is a local `SoftAssert softly = new SoftAssert()` in each test method, `assertAll()` as its last line, and the instance passed explicitly to any helper that records into it.
code
java · 37 linesimport org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;
public class CartTest {
// Broken: one collector shared by every method on this instance
private final SoftAssert shared = new SoftAssert();
@Test
public void emptyCartShowsZeroTotal() {
shared.assertEquals(cartTotal(), "0.00", "total");
shared.assertAll();
}
@Test
public void cartShowsCurrency() {
// Its own check passes, but assertAll() rethrows anything recorded earlier
shared.assertEquals(currency(), "EUR", "currency");
shared.assertAll();
}
// Fixed: a fresh collector that lives only inside this method
@Test
public void cartShowsCurrencyIsolated() {
SoftAssert softly = new SoftAssert();
softly.assertEquals(currency(), "EUR", "currency");
softly.assertAll();
}
private String cartTotal() {
return "0.00";
}
private String currency() {
return "EUR";
}
}go deeper
Recall that a SoftAssert belongs inside the test method that uses it, and that every such method ends with assertAll().
Explain that TestNG runs a class's methods on one instance and that assertAll() never clears its recorded failures, so a field collector carries failures forward.
Diagnose the haunted report from its tells, separate victim from culprit when assertAll() is missing, and explain what changes under parallel="methods".
Set a team rule, enforced in review or by a static check, that soft-assert collectors are method-local and passed explicitly to helpers, and explain its cost to page-object design.
## The symptom A test class keeps `private final SoftAssert softly = new SoftAssert();` as a field so that every test method and a few page helpers can record into it. Then the reports start to look haunted: - A method fails with a message about a field it never checks. - The same method passes when it is run on its own. - Fixing the method named in the report changes nothing, and the "failure" moves when methods are added or reordered. - Sometimes a method with real failures goes green. Each of these follows from two facts about how TestNG and `SoftAssert` work. ## Why it happens: one instance, one collector 1. **TestNG reuses the test instance.** By default TestNG creates one instance of each test class and runs all of that class's `@Test` methods on it. (A `@Factory` can supply several instances, but each instance still hosts all its methods.) Any instance field, including a `SoftAssert`, is therefore shared by every method of the class. JUnit Jupiter's default is a fresh instance per test method, which is why the same habit can look harmless to someone coming from JUnit. 2. **`assertAll()` does not reset.** A `SoftAssert` stores each failed check in an internal map. `assertAll()` reads that map and throws one `AssertionError` listing everything in it, but nothing removes the entries afterwards. 3. **So failures carry forward.** Method A records a failure and its `assertAll()` throws: A fails, correctly. Method B runs next on the same instance; its own checks pass, but its `assertAll()` finds A's failure still recorded and throws it again. B fails with A's message, and so does every later method in the class. 4. **And a missing `assertAll()` hides the culprit.** If A forgets to call `assertAll()`, A passes, and its failures surface in whichever method next calls `assertAll()`. The green method is the broken one; the red one is the victim. ## Parallel runs make it worse What shares a thread depends on the suite file's `parallel` mode, and the risk differs by mode: | `parallel` setting | Methods of one instance | Effect on a shared `SoftAssert` field | |---|---|---| | none (sequential) | One after another | Failures carry forward from method to method | | `methods` | In separate threads | Concurrent writes into an unsynchronised map; failures can be lost, duplicated in reports or attributed to the wrong method | | `classes` | All in the same thread | No concurrent writes, but failures still carry forward | | `instances` | All in the same thread | Same as `classes` for that instance | | `tests` | All in the same thread within one `<test>` | Same: carried forward, not written concurrently | The underlying collection is a plain `LinkedHashMap`, and `SoftAssert` adds no locking, so the `methods` row is a genuine data race, not just a reporting oddity. ## The fix - **Make it local.** Declare `SoftAssert softly = new SoftAssert();` as the first line of each test method that needs it and call `softly.assertAll()` as the last. A local variable cannot be seen by another method or thread, so the problem disappears in every mode. - **Pass it explicitly.** If a page object or helper needs to record soft checks, give it the instance as a parameter rather than having it reach for a field. - **Do not reach for `static`.** A static field is shared by every instance of the class, which widens the leak instead of closing it. - **Treat `@BeforeMethod` reassignment as a half-fix.** Creating a new `SoftAssert` in a `@BeforeMethod` method works for a sequential run, but under `parallel="methods"` one method's setup can replace the field while another method is still recording into it. ## How to spot it in review - A `SoftAssert` declared anywhere other than inside a test method. - A test method that records soft checks but has no `assertAll()` on every path. - Failure messages in the report that mention values the failing method never touches. - A failure that disappears when the method is run alone and reappears in the full class. Interviewers use this scenario to check two things at once: that the candidate knows TestNG shares one instance across a class's methods, and that they read `assertAll()` as a report of everything recorded so far, not a fresh verdict for the current method.
- Is creating the SoftAssert in a @BeforeMethod method a good enough fix?For a sequential run, yes: each method starts with a fresh collector. Under parallel="methods", no: the methods share one instance on different threads, so one method's @BeforeMethod can replace the field while another method is still recording into it, and failures land in the wrong test. A local variable in the test method is safe in every mode.
- One method that forgot assertAll() is green and the next method fails with its messages. Which method is broken?The green one. It recorded failures into the shared field without ever throwing them, so its own verdict is wrong, and the next method's assertAll() reports them under the wrong name. Fix both defects: give each method its own SoftAssert and make assertAll() its last statement.
- TestNG creates a new test-class instance for every test method, so fields are never shared.
- assertAll() clears the recorded failures, so reusing one SoftAssert is safe.
- Making the SoftAssert field static stops failures leaking between methods.
- The method whose assertAll() threw is always the one with the bug.
- SoftAssert synchronises internally, so one instance is safe across parallel threads.