skip to content

With PHPUnit 13, how do you run one test method against several inline inputs using the #[TestWith] attribute?

level: juniorimportance: should knowfreq 34%

answer

  1. one attribute per data set
  2. array becomes the method's arguments
  3. optional second argument names the set
  4. #[TestWithJson] takes a JSON string
  5. never mix with #[DataProvider]

basics

~10 s

Stack one #[TestWith([...])] attribute per case on the test method. Each array is unpacked into the method's parameters, and PHPUnit runs the method once per attribute, reporting every case as its own test.

solid answer

~40 s

`#[TestWith]` from `PHPUnit\Framework\Attributes` is repeatable: each instance holds one array whose values become the test method's arguments, and each becomes a separate test labelled `with data set #0`, `#1` and so on. An optional second argument names the case - `#[TestWith(['Ab1!', false], 'too short')]` - so a failure reads `with data set "too short"`. `#[TestWithJson('["Ab1!", false]')]` is the JSON-string variant. It suits a handful of literal cases; when data must be computed or shared, switch to a static `#[DataProvider]` method. Never put both on one method: PHPUnit 13 warns and uses only the provider's data. The old `@testWith` docblock is not read at all.

code

php · 19 lines
php
<?php
declare(strict_types=1);

namespace App\Tests\Security;

use App\Security\PasswordStrength;
use PHPUnit\Framework\Attributes\TestWith;
use PHPUnit\Framework\TestCase;

final class PasswordStrengthTest extends TestCase
{
    #[TestWith(['Ab1!', false], 'too short')]
    #[TestWith(['correcthorsebattery', false], 'no digit')]
    #[TestWith(['Tr0ub4dor&3xample', true], 'mixed classes')]
    public function testIsStrong(string $password, bool $expected): void
    {
        $this->assertSame($expected, (new PasswordStrength())->isStrong($password));
    }
}

go deeper

for a junior

Recall the shape: one #[TestWith([...])] per case, the array becomes the arguments, and each case runs and reports as its own test.

for a middle

Explain naming with the second argument, the argument-count warning, and why mixing #[TestWith] with a data provider drops the inline sets.

for a senior

Show when inline data stops paying off and a static provider is the better tool, and name the failures a leftover @testWith causes after an upgrade.

for a principal

Frame inline versus provider data as a readability trade-off for a whole suite, and set a convention so reviewers can see every case at a glance.

## What #[TestWith] is A **parameterised test** runs the same test method several times with different inputs. In PHPUnit 13 the lightest way to do that is the `#[TestWith]` attribute, declared in the `PHPUnit\Framework\Attributes` namespace. The attribute class is marked **repeatable** and targets **methods only**, so you stack as many as you need on one test method. Each `#[TestWith]` carries one PHP array. PHPUnit unpacks that array into the test method's parameters, exactly as if you had called the method with those values. One attribute equals one **data set**, and every data set is run and reported as a separate test, with `setUp()` and `tearDown()` around each. ## A password-strength example Suppose `PasswordStrength::isStrong(string $password): bool` decides whether a password meets a policy. Four literal cases fit comfortably inline: ```php use PHPUnit\Framework\Attributes\TestWith; #[TestWith(['Ab1!', false], 'too short')] #[TestWith(['correcthorsebattery', false], 'no digit')] #[TestWith(['Tr0ub4dor&3xample', true], 'mixed classes')] #[TestWith(['correct horse battery 42', true])] public function testIsStrong(string $password, bool $expected): void { $this->assertSame($expected, (new PasswordStrength())->isStrong($password)); } ``` The runner produces four tests. The first three carry the **name** given as the attribute's optional second argument, so a failure is reported as `testIsStrong with data set "too short"`. The last one has no name and falls back to a position label, `with data set #0`: the counter only advances for unnamed sets. ## The rules PHPUnit enforces - **The first argument must be an array.** A scalar there is reported as an invalid data set, and the method does not run. - **Names must be unique.** Two `#[TestWith]` attributes with the same name are rejected as a duplicate key. - **Too many values trigger a warning.** If a set carries more values than the method declares parameters, PHPUnit emits a warning such as `has more arguments (3) than the test method accepts (2)`. PHP itself would silently drop the extra argument, which is exactly why PHPUnit flags it. PHPUnit warnings fail the run by default. A variadic test method is exempt from this check. - **Too few values fail the test.** Missing arguments make PHP throw `ArgumentCountError` when the method is invoked, so that data set errors. - **No mixing with providers.** If a method has both `#[DataProvider]` (or `#[DataProviderExternal]`) and `#[TestWith]`, PHPUnit 13 warns that mixing is not supported and uses only the provider's data. The inline sets are dropped. ## #[TestWithJson] `#[TestWithJson]` takes the data set as a JSON string instead of a PHP array, plus the same optional name: `#[TestWithJson('["Ab1!", false]', 'too short')]`. It exists mostly for parity with older suites that wrote `@testWith` docblocks in JSON. In a new suite the PHP-array form is easier to read and lets static analysis see the values. ## When to choose #[TestWith] and when to move on | Need | Best fit | |---|---| | Two to six literal cases, visible next to the test | `#[TestWith]` | | Data built with code: objects, enums, loops, files | `#[DataProvider]` on a static method | | The same data feeding several test classes | `#[DataProviderExternal]` | Inline data keeps the cases right above the assertion, which is its whole appeal. It stops paying off when the list grows long, when values must be computed (attribute arguments are constant expressions: literals, arrays, constants, enum cases and `new` are allowed, but an ordinary function call such as `str_repeat()` is not), or when two tests should share one table of cases. ## What changed from docblocks Before attributes existed, the same thing was written as a `@testWith` tag in the method's docblock. PHPUnit 10 introduced attributes, PHPUnit 11 deprecated docblock metadata, and PHPUnit 12 removed it. PHPUnit 13 has only an attribute parser, so a leftover `@testWith` is plain comment text: the method is called with no arguments and fails with `ArgumentCountError`. Remember also that an attribute only works when its class name resolves: without `use PHPUnit\Framework\Attributes\TestWith;` the short name points into your own namespace and PHPUnit ignores it. ## Summary for an interview 1. Stack `#[TestWith([...])]` attributes, one per case. 2. Name the cases with the second argument so failures explain themselves. 3. Keep it for short literal lists; use a static data provider for anything built or shared. 4. Never combine it with `#[DataProvider]` on the same method.

  • In PHPUnit 13 on PHP 8.5, why can a #[TestWith] data set not contain str_repeat('a', 64)?
    Attribute arguments are constant expressions. Literals, arrays, constants, enum cases and `new` expressions are allowed, but an ordinary function call is not, so PHP rejects it at compile time. Once a case needs a computed value, move the data into a public static `#[DataProvider]` method, which is ordinary PHP code.
  • In PHPUnit 13, what happens when a #[TestWith] array has fewer values than the test method's required parameters?
    PHPUnit only checks for too many values. Too few surface when the method is invoked: PHP throws `ArgumentCountError`, and that data set is reported as an error while the other sets still run. A default value on the trailing parameter avoids it.

saying these in an interview costs you the question

  • Believes #[TestWith] and #[DataProvider] on one method are merged into one list
  • Thinks the second #[TestWith] argument is passed to the test as an extra parameter
  • Writes @testWith docblocks and expects PHPUnit 13 to read them
  • Assumes extra data-set values are silently ignored by PHPUnit
  • Tries to call str_repeat() or another function inside a #[TestWith] argument