skip to content

In TestNG's `@Test`, how do `timeOut` and `invocationTimeOut` differ, and what happens to a test that exceeds either limit?

level: middleimportance: should knowfreq 40%

answer

  1. milliseconds, not seconds
  2. one call versus the whole batch
  3. needs invocationCount
  4. interrupt, not kill

basics

~20 s

timeOut caps one invocation and invocationTimeOut caps the combined time of all invocationCount runs, both in milliseconds. Breaching either marks the test FAILED with a ThreadTimeoutException; TestNG interrupts the method's thread but cannot kill it.

solid answer

~40 s

`timeOut` is the maximum milliseconds a single invocation may take; the default `0` means no limit of its own (a suite file can supply a default). `invocationTimeOut` is the maximum milliseconds for *all* invocations of a method together, and the Javadoc says it is ignored when `invocationCount` is not specified. When a limit is exceeded TestNG records the test as `FAILURE` with `org.testng.internal.thread.ThreadTimeoutException`, not as a skip. To enforce the limit TestNG runs the method under a timer and interrupts its thread at expiry; code that ignores interruption, such as a socket read with no timeout, keeps running. So guard a slow payment call with `timeOut`, and also give the HTTP client its own timeout.

code

java · 14 lines
java
import org.testng.annotations.Test;

public class PaymentGatewayTest {

  @Test(timeOut = 3_000)
  public void authorizeRespondsWithinThreeSeconds() throws Exception {
    PaymentStub.authorize("order-42");
  }

  @Test(invocationCount = 10, invocationTimeOut = 15_000)
  public void tenAuthorizationsFitFifteenSeconds() throws Exception {
    PaymentStub.authorize("order-43");
  }
}

go deeper

for a junior

Know both attribute names, that the unit is milliseconds, and that a breach is reported as a failure.

for a middle

Explain per-invocation versus whole-batch budgets, why invocationTimeOut depends on invocationCount, and how TestNG times the method on another thread or with a monitor.

for a senior

Diagnose a timed-out test that still holds a connection: interruption is cooperative, so add client-side timeouts and do not trust timeOut to free resources.

for a principal

Decide where time limits live: in the clients and the suite-wide default, with per-method timeOut kept for genuinely slow paths rather than sprinkled everywhere.

## The two attributes TestNG's `@Test` annotation offers two time limits, both declared as `long` values in **milliseconds** with a default of `0`: - **`timeOut`**: *"The maximum number of milliseconds this test should take. If it hasn't returned after this time, it will be marked as a FAIL."* It applies to each invocation of the method. - **`invocationTimeOut`**: *"The maximum number of milliseconds that the total number of invocations on this test method should take."* The Javadoc adds that it *"will be ignored if the attribute invocationCount is not specified"*. It exists for methods repeated with `invocationCount`, where you care about the total budget rather than any one call. | | `timeOut` | `invocationTimeOut` | |---|---|---| | What it measures | One invocation | All invocations together | | Unit | Milliseconds | Milliseconds | | Default | `0` (no limit of its own) | `0` (no limit) | | Needs `invocationCount` | No | Yes, per its Javadoc | | On breach | `FAILURE` with `ThreadTimeoutException` | `FAILURE` with `ThreadTimeoutException` | A unit mistake is the classic slip: `timeOut = 5` is five *milliseconds*, which fails almost any real call. JUnit Jupiter's `@Timeout` defaults to seconds, so people moving between the two runners get this wrong in both directions. ## How TestNG enforces a limit A Java thread cannot be safely killed, so TestNG runs a timed method in a way that lets it stop *waiting*: 1. In a typical sequential run, the method is submitted to a single-thread executor and TestNG waits on it for the limit. If it has not finished, TestNG calls `shutdownNow()`, which **interrupts** the worker thread, records the failure and moves on. 2. When the method already runs on a TestNG worker thread (a parallel run other than `parallel="tests"`), TestNG starts a small monitor thread that interrupts the method's thread when the limit passes. If the method ignores the interrupt, it runs to completion and only then is it marked as failed. Either way the outcome is a **failure**, never a skip, and the thrown `ThreadTimeoutException` names the method and the limit. The 7.12 changelog records a fix so that timeouts raised while reusing an existing executor propagate the stack trace into the `ThreadTimeoutException`, which helps show where the method was blocked. ## The consequence: interrupt is a request Interruption only stops code that responds to it: `Thread.sleep`, `Object.wait`, interruptible channels and most blocking queues throw `InterruptedException`. A classic socket read with no read timeout, a busy loop, or a library that swallows the interrupt does not stop. That means: - the test is reported failed, but the thread may keep holding a connection or a browser session; - in the monitor-thread mode the suite can still wait for the stuck call; - the reliable fix is a timeout **inside** the code under test, such as a connect and read timeout on the payment client, with `timeOut` as the safety net. ## Combining them, and a suite-wide default In 7.12's invoker the limit actually applied is `timeOut` when it is non-zero and `invocationTimeOut` otherwise, so setting both does not give you two independent limits; choose the one that matches what you mean. When `timeOut` is `0` on the method, TestNG falls back to a `time-out` value set on the enclosing `<test>` or `<suite>` element of the suite file, if one exists. ## Example ```java import org.testng.annotations.Test; public class PaymentGatewayTest { @Test(timeOut = 3_000) public void authorizeRespondsWithinThreeSeconds() throws Exception { PaymentStub.authorize("order-42"); } @Test(invocationCount = 10, invocationTimeOut = 15_000) public void tenAuthorizationsFitFifteenSeconds() throws Exception { PaymentStub.authorize("order-43"); } } ``` The first method fails if any single call takes more than three seconds; the second fails if the ten calls together take more than fifteen, even when each call alone is fast enough. ## What interviewers listen for - Milliseconds, and the default of zero. - Per-invocation versus total budget, and the dependency on `invocationCount`. - The status: a timeout is a **failure**. - The honest limit of the mechanism: TestNG interrupts, it does not kill, so real protection needs client-side timeouts too. - A sensible size: a limit a few times the normal duration catches a hang without failing on an ordinary slow run in a busy CI agent.

  • Why can a TestNG test that timed out keep holding resources after it is reported failed?
    TestNG enforces `timeOut` by interrupting the method's thread, and interruption is cooperative. A blocking socket read without its own timeout, or code that swallows `InterruptedException`, keeps running and keeps its connection or session. Give the client under test real connect and read timeouts and treat `timeOut` as the safety net.
  • In TestNG, what does a timeOut of 0 on @Test mean?
    It is the default and means the method sets no limit of its own. TestNG then falls back to a `time-out` value on the enclosing `<test>` or `<suite>` element of the suite file if one is set; with none, the method may run indefinitely.
  • If a TestNG method sets both timeOut and invocationTimeOut, are both limits enforced?
    No. In 7.12 the invoker applies `timeOut` when it is non-zero and falls back to `invocationTimeOut` only when `timeOut` is zero. Pick the attribute that states what you mean: one call's limit or the batch's total budget.

A referee can blow the whistle when time is up and record the loss, but cannot drag a player off the pitch who refuses to stop playing.

  • TestNG's timeOut value is measured in seconds.
  • A test that exceeds timeOut is reported as SKIPPED.
  • invocationTimeOut limits each single invocation separately.
  • TestNG forcibly kills the thread of a timed-out method.
  • Setting both timeOut and invocationTimeOut enforces two independent limits.

Assumes TestNG 7.12; the units, defaults and failure status are long-standing, and a 7.12 fix propagates the stack trace into timeouts raised on an existing executor.