skip to content

As the lead of a large TestNG Selenium suite, which parallel mode and thread-count would you make the team default in testng.xml, and how would you defend it?

level: principalimportance: should knowfreq 38%

answer

  1. which unit owns the browser
  2. classes keeps setup with its tests
  3. tests for the browser matrix
  4. size to the environment, not the CPU

basics

~10 s

There is no universal answer, but a defensible default for a TestNG Selenium suite is parallel="classes", or parallel="tests" for a per-browser matrix, with thread-count sized to available browser capacity and measured, not guessed.

solid answer

~50 s

I would tie the mode to the unit our classes own state at. If each class holds one browser opened in `@BeforeClass`, `parallel="classes"` keeps that class's setup and tests on one thread while classes run side by side. For a per-browser matrix, `parallel="tests"` runs each browser's `<test>` element on its own thread, which also protects classes that are not thread-safe. I would not default to `methods`, because TestNG shares one instance per class across concurrent methods; a class must prove it is safe before opting in with a reviewed `<test>`-level override. `thread-count` I would size to what the browser nodes and test environment sustain, raise it stepwise while watching wall-clock time and the failure rate, and write the number down with its reason. Every file declares the DOCTYPE so a typo fails instead of quietly going serial.

go deeper

for a junior

Know that the suite file's parallel and thread-count attributes control concurrency and that none and 5 are the defaults.

for a middle

Be able to say what each mode keeps on one thread and why classes or tests suit a browser suite better than methods.

for a senior

Show you can predict which classes break under each mode and how the per-<test> pooling rule limits real concurrency.

for a principal

Defend a default from the unit that owns state, size thread-count by measured capacity, and set guard rails for overrides, legacy values and the DOCTYPE.

## Why this is a lead's decision TestNG makes parallelism a **property of the suite file**: one `parallel` attribute and one `thread-count` on `<suite>`, overridable per `<test>`. In a large suite those two values decide wall-clock time, how much load hits the test environment, and which hidden assumptions in the test classes get exposed. Left to individual engineers, the shared suite files drift into a mix of modes nobody can reason about. A lead sets the default, writes down why, and decides when an override is allowed. There is **no single right answer**; there is a defensible one for a given suite. ## The options, judged for a Selenium suite In TestNG 7.12 the default is `parallel="none"` with `thread-count="5"`. The real candidates: | Mode | What it keeps on one thread | Fits when | Main risk | |---|---|---|---| | `none` | everything | the suite is small or not yet parallel-safe | long runs | | `classes` | each class's methods | each class owns its browser | uneven class sizes leave threads idle | | `tests` | each `<test>` element | one `<test>` per browser or area | only as parallel as the number of `<test>` elements | | `methods` | nothing | classes hold no mutable instance state | one shared instance per class | | `instances` | each object's methods | a `@Factory` builds one instance per configuration | depends on factory design | ## A defensible default For a typical Selenium suite I would argue for this shape: 1. **`parallel="classes"` as the suite default.** In a page-object style suite a class often opens one browser in `@BeforeClass` and closes it in `@AfterClass`; classes mode keeps that hook and all its tests on one thread, so the driver field is never touched by two threads at once. 2. **`parallel="tests"` for the cross-browser matrix.** One `<test>` per browser, each carrying its `browser` parameter, with the suite `thread-count` equal to the number of browsers run together. Everything inside one browser's `<test>` stays serial, which the documentation names as the way to run classes that are not thread-safe. 3. **`methods` only by explicit opt-in.** TestNG creates one instance per class by default, so concurrent methods share its fields. A class can move to `methods` through a `<test>`-level override after review shows it holds no mutable instance state. 4. **`instances` where a `@Factory` builds one object per configuration**, since then the object, not the class, owns the browser. ## Sizing thread-count `thread-count` is a **capacity decision**, not a CPU decision. Browser tests spend their time waiting on remote browsers and the application under test, so the limit is downstream: - the number of browser sessions the nodes can host at once; - what the shared test environment, its database and its rate limits tolerate; - the per-`<test>` pooling rule: under `classes`, `methods` and `instances`, each `<test>` element has its own pool and `<test>` elements run one after another, so the suite value is the real ceiling per `<test>`; - under `tests`, the suite pool runs `<test>` elements themselves. Raise it stepwise, for example 2, 4, 8, and stop when wall-clock time stops falling or the failure rate rises. Record the chosen value with the measurement behind it, so the next person changing it knows what it was protecting. ## Guard rails to write into the team rule - **Every suite file declares the TestNG DOCTYPE**, so a misspelt mode such as `parallel="method"` fails at parse time instead of being read as `none`. - **No legacy `parallel="true"`**; TestNG maps it to `methods` with a warning, which hides the real choice. - **`<test>`-level overrides of `parallel` or `thread-count` need review**, because they silently change one area's concurrency. - **Revisit the default when the suite's shape changes**: many small classes favour `classes`; a few huge classes may justify splitting them before reaching for `methods`. ## How I would defend it The argument is that the default **follows the unit that owns state** in our codebase, so the choice is explainable from the code rather than from habit. It trades some peak speed (versus `methods`) for predictable threads, and the remaining speed comes from `thread-count`, sized by measurement against real capacity. The opposite position, `methods` everywhere for maximum throughput, is legitimate for a suite designed for it from the start; I would accept it if the classes were built stateless and the environment could carry the load.

  • Why not simply default every TestNG suite to parallel="methods" for maximum throughput?
    Because TestNG creates one instance per class by default, `methods` lets several threads use that instance's fields at once. In a Selenium suite with a driver field that means cross-talk between tests. It is a fine default only for classes designed without mutable instance state; otherwise it trades rare speed gains for intermittent failures.
  • Your matrix has four browsers in four <test> elements under parallel="tests". What should the suite thread-count be?
    At most four, since under `tests` the suite pool runs `<test>` elements and there are only four to run; a larger pool adds nothing. Lower it if the browser nodes or the environment cannot host four sessions at once. Inside each `<test>` the methods still run serially unless that `<test>` overrides `parallel`.

  • Sizes thread-count to the CPU cores of the machine running TestNG
  • Defaults to parallel="methods" without checking for shared instance fields
  • Claims one parallel mode is correct for every TestNG suite
  • Lets each engineer set parallel per <test> with no review
  • Raises thread-count without measuring wall-clock time or failure rate

Assumes TestNG 7.12, where the suite defaults are parallel none and thread-count 5.