In a TestNG suite file (testng.xml), how do the suite, test, classes, methods and packages elements nest, and what do include and exclude select?
answer
- one suite, many tests
- a test element is not a method
- classes or packages inside each test
- method names matched as regular expressions
basics
~10 sA testng.xml holds one <suite> containing one or more <test> elements; each <test> lists <classes> (optionally narrowed by <methods>) or <packages>. Inside <methods>, include and exclude pick test methods by regular expression.
solid answer
~40 sThe root is `<suite name=...>`. It contains one or more `<test name=...>` elements, and a `<test>` is a named group of classes, not a single test method: `@BeforeTest` and `@AfterTest` run around it. Each `<test>` lists `<classes>` with `<class name="fully.qualified.Name"/>` entries, or `<packages>` with `<package name=...>` entries, where TestNG scans the package and keeps only classes carrying TestNG annotations (a name ending in `.*` takes subpackages too). Under a `<class>`, a `<methods>` block with `<include name=...>` and `<exclude name=...>` selects methods, and those names are regular expressions, so `.*Login.*` matches every method containing `Login`. `<parameter>` elements can sit at suite, test, class or methods level, and the suite and test elements carry run attributes such as `parallel` and `thread-count`.
code
xml · 13 lines<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="smoke">
<test name="login">
<classes>
<class name="com.shop.LoginTest">
<methods>
<include name=".*valid.*"/>
<exclude name=".*Captcha.*"/>
</methods>
</class>
</classes>
</test>
</suite>go deeper
Recall the nesting: one suite, many test elements, classes or packages inside each, and methods narrowed by include and exclude. Say plainly that a test element groups classes.
Explain that include and exclude names are regular expressions, that packages keep only annotated classes, and that preserve-order defaults to true so file order is run order.
Point out how method-name patterns rot silently on rename, and why a suite should declare the DOCTYPE so the DTD can reject a bad attribute value at parse time.
Frame the file as a team artifact: decide whether runs are curated in testng.xml or selected by groups in code, and who owns changes to the shared suite files.
## What a TestNG suite file is A **suite file**, conventionally named `testng.xml`, is an XML document that tells TestNG **what to run and how to run it**. Instead of discovering test classes by itself, TestNG reads the file, builds an in-memory model (`XmlSuite`, `XmlTest`, `XmlClass`, `XmlInclude`) and executes exactly what the file names. The file is checked against TestNG's DTD when it declares one at the top: ```xml <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> ``` Without that line TestNG still parses the file but logs a warning recommending it, and the DTD's checks on attribute values are not applied. ## The element hierarchy The nesting has a fixed shape: 1. **`<suite>`** - the root. One file is one suite. It carries the suite name and the run-wide attributes: `parallel`, `thread-count`, `data-provider-thread-count`, `configfailurepolicy`, `preserve-order` and others. 2. **`<test>`** - a named, ordered group of classes inside the suite. A suite may hold many. This is the element `@BeforeTest` and `@AfterTest` wrap, which is why "test" is ambiguous on this framework: a `<test>` element is not a `@Test` method. 3. **`<classes>` / `<class>`** - the test classes in that `<test>`, each by fully qualified name. 4. **`<methods>`** - optional, under a `<class>`; narrows which methods of that class run. 5. **`<packages>` / `<package>`** - an alternative to listing classes: TestNG scans the package and keeps only classes that have TestNG annotations. A `<test>` can also carry a `<groups>` block whose `<run>` element includes or excludes groups; how group names and their patterns match belongs with TestNG groups rather than with the file layout. | Element | Lives under | Selects or sets | |---|---|---| | `<suite>` | file root | the whole run and its defaults | | `<test>` | `<suite>` | a named batch of classes, with optional overrides | | `<class>` | `<classes>` | one test class | | `<include>` / `<exclude>` | `<methods>` | methods by regular expression | | `<package>` | `<packages>` | every annotated class in a package | | `<parameter>` | suite, test, class, methods | a named string value for `@Parameters` | ## Include and exclude Inside `<methods>`, the `name` attribute of `<include>` and `<exclude>` is a **regular expression**, not a wildcard. `.*smoke.*` matches any method whose name contains `smoke`; a bare `*` is not the "anything" pattern. Key points: - with no `<methods>` block, every test method of the class is eligible; - `<include>` restricts the class to matching methods, `<exclude>` removes matching methods; - a `<package>` can carry its own `<include>` and `<exclude>` to narrow which classes are picked up; - a package name ending in `.*` includes subpackages as well. The TestNG documentation warns about leaning on method-name selection: renaming a Java method silently stops a pattern from matching, so a test can drop out of the run with no error. ## A small example ```xml <!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd"> <suite name="checkout-regression"> <parameter name="env" value="staging"/> <test name="cart"> <classes> <class name="com.shop.CartTest"> <methods> <include name=".*add.*"/> <exclude name=".*legacy.*"/> </methods> </class> </classes> </test> <test name="search"> <packages> <package name="com.shop.search.*"/> </packages> </test> </suite> ``` Here the suite has two `<test>` elements. In `cart`, only methods of `CartTest` whose names contain `add` and not `legacy` run. In `search`, every annotated class in `com.shop.search` and its subpackages runs. ## Order and other elements By default `preserve-order` is `true`, so classes and included methods run in the order the file lists them, subject to priorities and dependencies declared in code. Setting `preserve-order="false"` on a `<test>` lets TestNG choose the order. A suite can also pull in other suite files through `<suite-files>`; a `<parameter>` placed inside a `<suite-file>` element is ignored, with a log message saying so. ## Mistakes that cost a run - **Reading `<test>` as a method**: `@BeforeTest` then looks like it runs per method, when it runs once around the whole `<test>` element. - **Wildcard habits**: `*Login*` is not a working pattern; write `.*Login.*`. - **Fully qualified names**: a `<class name>` must be the fully qualified class name; the simple name of a class that lives in a package cannot be loaded. - **Duplicated class lists**: the same class listed in two `<test>` elements runs twice, once per element, which is intended for a per-browser matrix and wasteful anywhere else. ## Contrast with JUnit Jupiter JUnit Jupiter has no XML suite descriptor; grouping a set of classes there is done with an annotated suite class on the JUnit Platform. In TestNG the XML file is the native, first-class way to define a run, which is why interviewers expect a TestNG candidate to read and write one fluently.
- In a TestNG suite file, why does a method stop running after it is renamed, with no error?`<include>` and `<exclude>` names are regular expressions matched against method names at run time. If a rename means the include pattern no longer matches, the method is simply not selected: TestNG does not fail the run, and its warning that the expression matched no method is logged only above the default verbosity. Patterns tied to naming conventions, or groups declared in code, survive refactoring better than literal method names.
- What does a TestNG <package> element pick up, and how do you reach subpackages?TestNG scans the named package and keeps only classes that carry TestNG annotations, so helper classes in the same package are ignored. A package name ending in `.*` includes subpackages too, and the `<package>` element can carry `<include>` and `<exclude>` children to narrow which classes it keeps.
- Treats a <test> element as one @Test method rather than a group of classes
- Writes * as the match-anything pattern in include, instead of the regular expression .*
- Thinks a <package> entry runs every class in the package, annotated or not
- Assumes the classes in a <test> run in a random order by default
- Believes a testng.xml can hold several <suite> roots