A TestNG @Factory builds one instance per browser and locale pair, but results show identical method names; how do you tell instances apart?
answer
- result name comes from the instance
- an interface with one method
- toString() as a fallback
- constructor arguments in fields
basics
~20 sImplement org.testng.ITest so getTestName() returns, say, the browser and locale; TestNG uses it as each result's name. Without ITest, overriding toString() makes TestNG name results as the method name followed by "on" and that string.
solid answer
~40 sTestNG computes each result's name from the test instance. If the instance implements `org.testng.ITest`, the result is named by `getTestName()`; if that returns null, TestNG falls back to the method name. If the instance does not implement `ITest` but overrides `toString()`, the result is named "`methodName` on `toString()`". Otherwise only the method name is used, which is why instances of one class look identical. For a factory, the clean pattern is to keep the constructor arguments in final fields and build the name from them, which is stable and safe under parallel runs because each instance owns its fields. The results also record the factory's constructor arguments, available through `ITestResult.getFactoryParameters()`.
code
java · 29 linesimport org.testng.ITest;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
public class CheckoutTest implements ITest {
private final String browser;
private final String locale;
@Factory(dataProvider = "pairs")
public CheckoutTest(String browser, String locale) {
this.browser = browser;
this.locale = locale;
}
@DataProvider
public static Object[][] pairs() {
return new Object[][] {{"chrome", "en-US"}, {"firefox", "de-DE"}};
}
@Override
public String getTestName() {
return browser + "/" + locale;
}
@Test
public void cartShowsLocalisedPrice() {
}
}go deeper
Recall that implementing ITest and returning a name from getTestName() is how TestNG labels results with something other than the method name.
Explain the naming order: ITest name, then a class test name, then an overridden toString() as 'method on ...', then the bare method name.
Prefer names built from final constructor fields, guard against null, and avoid per-row ITest tricks under parallel providers in favour of recorded parameters.
Set a team convention for how instances and rows are labelled, so failures are attributable in every report without per-class naming code.
## The problem A `@Factory` that creates one instance of a checkout class per browser and locale pair produces many results with the same class and method names. When one fails, the HTML report or the CI test tab says `cartShowsLocalisedPrice` failed, but not for which pair. TestNG has a specific rule for naming results from instances, and knowing it turns this into a small change. ## How TestNG names a result When it creates a result, TestNG looks at the test instance and applies these rules in order: 1. **The instance implements `org.testng.ITest`**: the result's name is whatever `getTestName()` returns. If it returns `null`, TestNG falls back to the method name (and, at higher verbosity, prints a warning). 2. **The class carries a test name** from TestNG's own class-level settings: that name is used. 3. **The instance overrides `toString()`**: the result is named `methodName on <toString()>`. 4. **Otherwise**: the result is named after the method only. Rule 4 is the default, and it is why factory instances blur together. ## Option A: implement ITest `ITest` has a single method, `String getTestName()`. Its javadoc says a class implementing it receives special treatment, such as having the test name displayed in the HTML reports. - **Build the name from constructor arguments** stored in `final` fields: `browser + "/" + locale`. - **Every method of the instance gets that name** as its result name; the method itself is still known from the result's method, but a name like `chrome/en-US` alone no longer says which check ran, so some teams include more context in the reporting layer. - **Never return null by accident**: a null name quietly falls back to the bare method name. ## Option B: override toString() If you do not want to implement an interface, override `toString()` to return the pair. TestNG then names results `cartShowsLocalisedPrice on chrome/en-US`, keeping the method name and adding the instance. Only an overridden `toString()` counts; the default `Object.toString()` is ignored. | Approach | Result name | Keeps method name in the name? | |---|---|---| | neither | `cartShowsLocalisedPrice` | yes, but instances are indistinguishable | | `toString()` overridden | `cartShowsLocalisedPrice on chrome/en-US` | yes | | `ITest` implemented | `chrome/en-US` | no | ## The data recorded anyway Even without either option, TestNG keeps the inputs: `ITestResult.getParameters()` holds a data provider row's arguments, and `ITestResult.getFactoryParameters()` holds the arguments the factory passed to the constructor. A listener or a custom report can use them, and the built-in HTML report shows parameters, but a name is what people read first in a failure list. ## Why per-instance names are safe and per-row names are not With a factory, the name is derived from fields fixed at construction, so it is **stable and thread-safe**: each instance owns its own fields, whatever parallel mode the suite file selects. Naming each **data provider row** with `ITest` is harder, because all rows share one instance and `getTestName()` has no row argument. The common workaround stores the current row's name in a per-thread field set before each invocation; it works but is easy to get wrong when rows run in parallel, and a stale value names the wrong row. If per-row names matter, the recorded parameters are often the safer source. ## Choosing between the two - Pick **`toString()`** when the report should keep reading by method, with the instance as a qualifier; it also helps debugging, since loggers and debuggers show the same string. - Pick **`ITest`** when the instance label is what readers scan for, for example a matrix of tenants where the tenant is the first thing anyone asks about. - Whichever you choose, make the label **unique per instance**: two instances built with the same browser and locale would again be indistinguishable, which usually means the factory's rows contain a duplicate. - Keep labels short and free of volatile values such as timestamps, so the same instance gets the same name from one run to the next and failure history lines up. ## Interview summary - Default: method name only, so factory instances collide. - `ITest.getTestName()` replaces the name; `toString()` appends " on ...". - Build names from final constructor fields; never return null. - Factory and provider arguments are recorded on each result regardless.
- What happens if a TestNG ITest implementation returns null from getTestName()?TestNG falls back to the method name for that result, and at higher verbosity prints a warning that the ITest implementation returned null. The run continues, but the instance becomes indistinguishable again, so a name built from possibly-unset fields should be guarded.
- Without ITest or toString(), can a TestNG listener still tell which factory instance a result came from?Yes. Each `ITestResult` records the constructor arguments the factory used, through `getFactoryParameters()`, and the test instance itself through `getInstance()`. A listener can build its own label from those, even though the default result name is just the method name.
- TestNG always prints constructor arguments in the result name automatically
- Any toString() counts, including the one inherited from Object
- Implementing ITest keeps the method name and appends the custom name
- Returning null from getTestName() makes the test fail
- ITest names for factory instances are unsafe under parallel runs