skip to content

In TestNG, how does a @DataProvider method feed rows of arguments into a @Test method, and which return types may it use?

level: juniorimportance: must knowfreq 66%

answer

  1. a named supplier method
  2. linked by the dataProvider attribute
  3. one row, one invocation
  4. two-dimensional array versus iterator

basics

~10 s

A @DataProvider method returns rows of arguments, and every @Test method naming it in its dataProvider attribute runs once per row. Accepted return types are Object[][], Iterator<Object[]>, Object[] and Iterator<Object>.

solid answer

~40 s

You annotate a method with `@DataProvider`; its name defaults to the method name unless you set `name`. A test opts in with `@Test(dataProvider = "logins")`, and TestNG invokes that test once per row, assigning each `Object[]` to the method's parameter list. Four return shapes are accepted: `Object[][]` and `Iterator<Object[]>` for multi-argument rows, `Object[]` and `Iterator<Object>` when the test takes one argument. The array forms build every row before the first invocation; the iterator forms let TestNG pull rows one at a time, which matters when a large data set is read from a file. Unless you say otherwise, TestNG looks for the provider in the test class and its superclasses, and if the provider throws, the test is reported as skipped rather than failed by default.

code

java · 36 lines
java
import java.util.Iterator;
import java.util.List;
import org.testng.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class LoginTest {

  @DataProvider(name = "logins")
  public Object[][] logins() {
    return new Object[][] {
      {"alice", "correct-pass", true},
      {"alice", "wrong-pass", false},
    };
  }

  @DataProvider
  public Iterator<Object[]> lazyLogins() {
    List<Object[]> rows = List.of(new Object[] {"bob", "secret", true});
    return rows.iterator();
  }

  @Test(dataProvider = "logins")
  public void login(String user, String password, boolean expected) {
    Assert.assertEquals(authenticate(user, password), expected);
  }

  @Test(dataProvider = "lazyLogins")
  public void lazyLogin(String user, String password, boolean expected) {
    Assert.assertEquals(authenticate(user, password), expected);
  }

  private boolean authenticate(String user, String password) {
    return password.equals("correct-pass") || password.equals("secret");
  }
}

go deeper

for a junior

Recall the pairing: @DataProvider on the supplier, @Test(dataProvider = ...) on the consumer, one invocation per row, and the name defaulting to the method name.

for a middle

Explain the four return shapes, why an iterator lets rows be built lazily, and what status TestNG reports when the provider throws, returns null or returns nothing.

for a senior

Show that you watch report status: provider exceptions become skips by default, so a broken data file can hide unless propagateFailureAsTestFailure is set.

for a principal

Weigh literal rows in code against rows streamed from external files: readability and review of test data versus volume, and who owns keeping that data valid.

## What a data provider is In TestNG, a **data provider** is an ordinary method annotated with `@DataProvider` whose return value is a set of **rows**. Each row is the full argument list for one invocation of a test method. A test method that wants those rows names the provider through the `dataProvider` attribute of its `@Test` annotation, and TestNG then calls the test method once per row. The result is one test body exercised over many inputs, with each invocation reported separately and carrying its own parameters. The provider's name is taken from the annotation's `name` attribute; when that is left empty (its default), the **method name** is used. Two providers with the same name in the same class are an error that TestNG reports when it resolves the provider. ## Wiring a test to a provider - Annotate the supplier: `@DataProvider(name = "logins")` on a method returning rows. - Point the test at it: `@Test(dataProvider = "logins")`. - Match the shapes: each row must have as many values as the test method has parameters, with compatible types. - Lookup scope: by default TestNG searches the **test class and its superclasses**. A provider in another class is reached with the `dataProviderClass` attribute. - One provider can feed several test methods; TestNG calls it separately for each test method that uses it. ## The four return shapes | Return type | Row shape | When rows are built | |---|---|---| | `Object[][]` | one `Object[]` per invocation, spread over the parameters | all at once, before the first invocation | | `Iterator<Object[]>` | one `Object[]` per invocation | lazily, one `next()` per invocation in a sequential run | | `Object[]` | one value per invocation, for a single-parameter test | all at once | | `Iterator<Object>` | one value per invocation, for a single-parameter test | lazily | The element type is not limited to `Object`: a provider may return `MyRow[][]` or `Iterator<MyRow>`, and TestNG inspects the iterator's generic type to decide whether each element is a whole row (an array) or a single argument. ## Array versus iterator The difference between `Object[][]` and `Iterator<Object[]>` is **when the data exists**. An array provider must materialise every row before TestNG starts invoking the test, which is fine for ten rows and wasteful for a few hundred rows parsed from a CSV file or fetched from a database. An iterator provider hands TestNG a cursor; in a sequential run TestNG asks for the next row only when it is about to invoke the test again, so a file can be streamed rather than loaded. Two cautions apply to iterators: 1. A row must not be `null`; TestNG rejects a null row with an error. 2. The laziness is a property of how TestNG consumes the iterator, so a provider marked `parallel = true` changes the picture: rows are queued for a thread pool rather than pulled on demand. ## When the provider misbehaves The report status depends on what went wrong: - **The provider throws an exception**: the test method is reported as **skipped**, not failed, unless the provider sets `propagateFailureAsTestFailure = true` (added in 7.6.0), in which case it is reported as a failure. - **The provider returns `null`**: TestNG raises its own error ("returned a null value") and reports a failure. - **The provider returns no rows**: TestNG logs a warning that the method will be skipped since its data provider returned an empty array or iterator. - **A row does not fit the parameters**: TestNG reports a data provider mismatch for that invocation. The skip-by-default rule surprises people: a broken data file can turn a whole data-driven test into skips that a dashboard counting only failures does not flag. ## Reading the results Each row produces its **own result**: one row can pass while the next fails, and the result carries the arguments it was invoked with, which TestNG's HTML report displays next to the method. Practical consequences: - A failure points at a specific row, so keep rows small and their values recognisable (a user name rather than an opaque id). - The result count of the class grows with the row count, which is expected rather than a sign of duplicated tests. - Rerunning only failed invocations is possible because TestNG remembers which row positions failed. ## Contrast with JUnit Jupiter JUnit Jupiter reaches the same goal through its own parameterized-test annotations and argument sources, with different rules; none of that machinery applies to a TestNG `@DataProvider`, whose contract is simply a named method returning one of the four shapes above.

  • What is reported when a TestNG data provider throws while building its rows?
    By default the test method that uses it is reported as skipped, with the provider's exception as the cause. Setting `propagateFailureAsTestFailure = true` on the `@DataProvider` (available since 7.6.0) makes TestNG report it as a failure instead, which keeps a broken data source from hiding among skips.
  • When is an Iterator<Object[]> provider worth its extra code over Object[][]?
    When the rows are many or expensive to build, for example parsed from a large file or a database. In a sequential run TestNG pulls one row per invocation, so the provider can stream rather than load everything first. For a handful of literal rows, `Object[][]` is simpler and easier to read.

  • A data provider must always return Object[][]; iterators are not allowed
  • The provider's name must always be set explicitly in the annotation
  • An exception in the data provider is always reported as a test failure
  • A provider is called once per class and its rows are shared by all tests
  • Each row is passed to the test method as a single Object[] parameter