skip to content

Groups & Dependencies

Groups tag methods for selection from suite XML or the command line, and dependsOnMethods orders tests and skips dependents when one fails. Interviewers use it to test ordering judgment.

on this pageshow

explore

questions

5

In TestNG, how do you tag test methods with groups and then run only one group, such as smoke, from testng.xml or the command line?

level: juniorimportance: must knowfreq 62%

answer

  1. a label on the annotation
  2. class labels plus method labels
  3. groups, run, include, exclude
  4. regular expressions, not shell wildcards
  5. exclusion preempts inclusion

basics

~20 s

Add groups = {"smoke"} to @Test, then select it with <groups><run><include name="smoke"/></run></groups> in testng.xml or -groups smoke on TestNG's command line. Group names match as regular expressions, and an excluded group beats an included one.

solid answer

~40 s

A TestNG group is a string label on `@Test(groups = {"smoke", "regression"})`. Put `@Test(groups = ...)` on the class and its public methods inherit the label, with method-level groups added on top. The suite file picks what runs: inside a `<test>` element, `<groups><run>` holds `<include name="smoke"/>` and `<exclude name="broken"/>`, and groups named under `<suite>` accumulate with each `<test>`'s own. TestNG's command-line runner takes the same choice as `-groups` and `-excludegroups`, each a comma-separated list. Names are matched as Java regular expressions, so `windows.*` works and a bare `*` does not. A method in both an included and an excluded group is left out, which is why a `broken` group is a clean way to park tests. Switching sets needs no recompile, only a different suite file or flag.

code

java · 13 lines
java
import org.testng.annotations.Test;

@Test(groups = "checkout")
public class CheckoutTests {

  @Test(groups = {"smoke"})
  public void cartTotalIsShown() { }

  @Test(groups = {"regression", "broken"})
  public void couponStacking() { }

  public void taxRounding() { }
}

go deeper

for a junior

Recall the groups attribute on @Test, the groups/run/include/exclude block in testng.xml, and that the names are regular expressions.

for a middle

Explain the precedence: exclusion beats inclusion, an include makes untagged methods drop out, suite and test groups accumulate, class and method groups combine.

for a senior

Show how you would lay out labels so the smoke and regression slices stay predictable, and how a broken group quarantines failures without deleting them.

for a principal

Talk about naming discipline: a small, documented label vocabulary with hierarchical names, so pattern selection stays safe as the suite grows.

## What a TestNG group is A **group** in TestNG is nothing more than a string label attached to a test method through the `groups` attribute of `@Test`. A method can carry any number of labels, and the labels change nothing until a run filters on them or a dependency names them. That is the point: you tag once in code and decide at run time which slice of the suite to execute, without recompiling. - **Method level:** `@Test(groups = {"smoke", "regression"})` puts one method in two groups. - **Class level:** `@Test(groups = "checkout")` on the class gives that label to every public method of the class, and a method-level `groups` list adds to it rather than replacing it. - **No group:** a method with no label is still a test; whether it runs depends on the filter, as the table below shows. ## Selecting groups in the suite file The usual place to choose groups is the suite file, `testng.xml`. Inside a `<test>` element, a `<groups>` block holds a `<run>` element, and `<run>` holds `<include>` and `<exclude>` entries: ```xml <suite name="Nightly"> <test name="Smoke"> <groups> <run> <include name="smoke"/> <exclude name="broken"/> </run> </groups> <classes> <class name="com.shop.tests.CheckoutTests"/> </classes> </test> </suite> ``` Groups can also be declared under `<suite>`; they apply to every `<test>` below it and **accumulate** with the `<test>`'s own, so including `a` at suite level and `b` in a `<test>` runs both. A `<define>` element builds a **group of groups** (a meta-group), for example an `all` group that includes `smoke` and `regression`, so the run block can name one label. ## Regular expressions, not wildcards Every name in `<include>`, `<exclude>`, and the command-line flags is treated as a **Java regular expression**. That has two consequences worth saying out loud in an interview: 1. `windows.*` matches `windows.checkintest` and `windows.functest`, because `.*` means "any characters". 2. A shell-style `*` on its own is not "everything"; the regular expression for anything is `.*`. Naming groups hierarchically (`api.smoke`, `ui.smoke`) pays off precisely because a pattern can then select a family of them. ## How include and exclude combine TestNG's selector evaluates exclusions after inclusions and lets them win: a method that matches an excluded group is dropped even if another of its groups is included. | Filter configured | Method in included group | Method in excluded group | Method with no group | |---|---|---|---| | include only | runs | n/a | does not run | | exclude only | n/a | does not run | runs | | include and exclude | runs unless also excluded | does not run | does not run | The last column surprises people: once any `<include>` is present, an unlabelled method falls outside the selection. With only an `<exclude>`, everything not excluded still runs. This is also why a `broken` group works as a parking lot: add the label to a failing method, keep `<exclude name="broken"/>` in the suite, and the rest of the run stays clean while the broken tests remain visible in code. ## Selecting from the command line TestNG's own runner class, `org.testng.TestNG`, accepts `-groups` and `-excludegroups`, each a comma-separated list of names. They are convenient for a one-off run against a class with `-testclass`. In a pipeline the common shape is two suite files, or one suite with two `<test>` elements: a fast `smoke` slice on every change and the full `regression` slice nightly. How a build tool forwards these choices is that tool's configuration, not TestNG's. ## A worked nightly split A common layout for a pipeline that splits a fast slice from a full one: 1. Every test that guards a critical path carries `smoke` in addition to its feature label. 2. Every test carries `regression`, often set at class level so nobody forgets it. 3. The per-commit job's suite includes `smoke` and excludes `broken`. 4. The nightly job's suite includes `regression` and excludes `broken`. Because `smoke` tests are also `regression` tests, the nightly run is a superset of the per-commit run, and a quarantined test drops out of both slices through one label. ## Common mistakes - Expecting `*` to behave like a shell glob. - Assuming an include of `smoke` still runs unlabelled helper tests. - Believing a method in both an included and an excluded group runs because the include came first. - Using groups as an ordering device; groups select, they do not order. Ordering is the job of dependencies. JUnit Jupiter's nearest counterpart is `@Tag` with tag expressions; TestNG groups differ in being matched by regular expression and being definable in the suite file itself, including meta-groups.

  • In TestNG, a method is tagged both "smoke" and "flaky"; the suite includes smoke and excludes flaky. Does it run?
    No. TestNG's selector lets exclusions preempt inclusions: once a method matches any excluded group it is dropped, whichever of its other groups is included. That makes an exclude list a reliable way to quarantine a label across every slice that would otherwise pick the method up.
  • What does a <define> element inside <groups> give you in a TestNG suite file?
    A meta-group: a named group made of other groups, for example all = smoke + regression. The run block can then include that one name, and changing what all means is a one-line edit in the suite file rather than a retag of test code.

  • A bare * in an include selects every group.
  • With an include of smoke, untagged tests still run.
  • If a method is both included and excluded, the include wins.
  • Changing which groups run requires recompiling the tests.
  • Class-level groups are replaced by method-level groups, not combined.
open as a page

In TestNG, what does dependsOnMethods do, and what happens to dependent tests when the method they depend on fails?

level: middleimportance: must knowfreq 66%

basics

~20 s

dependsOnMethods makes a TestNG test run only after the named methods, and only if they all succeeded. If one fails, the dependent is reported SKIP, as is everything downstream, unless the dependent sets alwaysRun = true.

open as a page

In TestNG, what does an @BeforeGroups method do, when exactly does it run, and why might it never run in a group-filtered suite?

level: middleimportance: should knowfreq 42%

basics

~20 s

@BeforeGroups runs setup once per suite-file <test>, shortly before the first test method of the named groups. It never fires if no member of those groups is selected, or if its class is not part of that <test>.

open as a page

After a TestNG suite starts excluding a group, the run errors with 'depends on nonexistent group'; why does dependsOnGroups do that, and how would you fix it, including with ignoreMissingDependencies?

level: seniorimportance: should knowfreq 36%

basics

~20 s

dependsOnGroups resolves against the methods selected for the run, so excluding the depended-on group leaves nothing to match and TestNG throws a TestNGException. Restore the selection, or set ignoreMissingDependencies = true on the dependent to run without it.

open as a page

As the lead of a large TestNG suite, would you allow dependsOnMethods at all, and what rule would you set for when it is acceptable?

level: principalimportance: should knowfreq 38%

basics

~20 s

Allow dependsOnMethods only as a reviewed exception: for steps of one workflow in one class, where a later step is meaningless after an earlier failure. Setup goes in configuration methods, and every other test must run and pass on its own.

open as a page