skip to content

In Pest, what do the --mutate option and the covers() and mutates() functions do together, and how do you gate CI on the result?

level: seniorimportance: should knowfreq 28%

answer

  1. needs Xdebug 3+ or PCOV
  2. covers() or mutates() names the target
  3. only tests declaring a target run
  4. tested, untested, uncovered, score
  5. --mutate --min=N fails below threshold

basics

~20 s

--mutate makes Pest mutate the classes named by covers() or mutates() in your test files, rerun the tests that cover each mutation, and report which survived as untested; --mutate --min=N fails the run when the score is below N percent.

solid answer

~40 s

Pest has mutation testing built in through `pestphp/pest-plugin-mutate`, and it needs a coverage driver, Xdebug 3 or PCOV. You declare the code a test file targets with `covers(TodoService::class)` or `mutates(TodoService::class)`; both drive mutation identically, but `covers()` also narrows the code coverage report. `./vendor/bin/pest --mutate` then generates mutations in those classes, runs only the tests that cover each mutated line, and prints any mutation the suite did not catch as `UNTESTED` with a diff, plus a score. For CI, `--mutate --min=80` fails the run below 80 percent, `--parallel` spreads the work, and `--covered-only` or `--everything` change the scope. A line that should never be mutated gets a `// @pest-mutate-ignore` comment.

code

bash · 5 lines
bash
# mutate the classes named by covers()/mutates(), in parallel, fail below 80%
./vendor/bin/pest --mutate --parallel --min=80

# rerun a single surviving mutation after strengthening the test
./vendor/bin/pest --mutate --id=76d17ad63bb7c307

go deeper

for a junior

Recall the command, pest --mutate, and that a test file declares its target with covers() or mutates().

for a middle

Explain tested, untested and uncovered mutations, why a coverage driver is required, and the one difference between covers() and mutates().

for a senior

Design the CI job: scoped targets, --parallel, --min ratcheted from the current score, caching, and a policy for @pest-mutate-ignore so it does not hide weak tests.

for a principal

Decide where mutation runs pay off, for example on domain modules rather than framework glue, and how the score is used without becoming a vanity target.

## What Pest's mutation runner does Mutation testing in Pest is a feature of Pest itself: `pestphp/pest-plugin-mutate` is a required dependency of Pest 5, and the entry point is the `--mutate` option. Pest makes small edits to your source code (for example replacing a return value with `[]`), reruns the relevant tests against each edit, and reports whether a test failed. A **tested** mutation was caught; an **untested** one survived with the suite still green; an **uncovered** one sits on a line no test executes. The run prints a **score**, the percentage of mutations that were tested. Pest needs to know which lines each test executes, so mutation testing requires a coverage driver: **Xdebug 3.0+ or PCOV**. ## Declaring the target: covers() and mutates() By default Pest does not mutate your whole application. Each test file says what it is responsible for: ```php covers(TodoService::class); it('lists open todos', function () { expect((new TodoService)->open())->toHaveCount(2); }); ``` | Function | Effect on `--mutate` | Effect on `--coverage` | |---|---|---| | `covers(X::class)` | X is a mutation target | coverage report is filtered to X for these tests | | `mutates(X::class)` | X is a mutation target | no change to the coverage report | Two consequences follow from how Pest implements them: - When `--mutate` is on and you pass none of `--everything`, `--class` or a path, Pest runs **only the tests in files that call `covers()` or `mutates()`**. A file without either is not part of the mutation run at all. - For each mutation, Pest runs only the tests that cover the mutated code, and it caches results between runs, which keeps the cost bounded. ## Reading the output An untested mutation is printed with its file, line, mutator name, an ID and a diff: ```text UNTESTED app/Services/TodoService.php > Line 44: ReturnValue - ID: 76d17ad63bb7c307 ``` - The fix is a stronger assertion in the test, for example checking the returned items rather than only the HTTP status. - `./vendor/bin/pest --mutate --id=76d17ad63bb7c307` reruns only that mutation, with the same options as the original run. - A line that is deliberately untested, such as a validation rule list, can be excluded with a `// @pest-mutate-ignore` comment on the line or in the docblock above a property. ## Options that change scope and speed 1. `--parallel` runs mutations across processes and is the first thing to add. 2. `--covered-only` generates mutations only on lines that some test covers. 3. `--everything` mutates every class in the project, ignoring `covers()`; the docs pair it with `--covered-only` and `--parallel` because it is expensive. 4. `--class=App\Models` and `--ignore=App\Http\Requests` include or exclude classes. 5. `--bail`, `--stop-on-untested` and `--stop-on-uncovered` stop early; `--clear-cache` and `--no-cache` discard the cache; `--profile` lists the ten slowest mutations. ## What a covers() line does to the rest of the suite `covers()` is not only a mutation switch. Pest turns it into a `covers` declaration on every test in the file, so a normal `--coverage` run reports, for those tests, only the code of the named class. That is usually what you want for unit tests, and it is why `mutates()` exists: an integration test file that exercises many classes can name its mutation target with `mutates()` without narrowing its coverage report. Both functions accept several targets at once, for example `covers(TodoService::class, TodoRepository::class)`, and `covers()` accepts function names as well as classes. A practical convention is one `covers()` per unit-test file, pointing at the class the file is named after, and `mutates()` in feature tests that deliberately cross several classes. ## Gating CI on the score - `./vendor/bin/pest --mutate --min=80` fails the run when the score is below 80 percent and prints the actual and minimum values. - `--ignore-min-score-on-zero-mutations` keeps a run green when a change produced nothing to mutate, which matters when CI mutates only a subset. - Set the threshold from the current score and ratchet it upward; a threshold picked from the air either blocks everything or proves nothing. - Keep the mutation job separate from the fast unit-test job, since even scoped mutation runs cost many times a normal run. The interview point is the Pest-specific wiring: the target is declared per test file, only those files run under `--mutate`, a coverage driver is mandatory, and `--min` turns the score into a pass or fail.

  • You run pest --mutate and half the suite does not execute; why?
    With `--mutate` and no `--everything`, `--class` or path argument, Pest runs only the tests in files that call `covers()` or `mutates()`. Files without a declared target are excluded from the mutation run. Add the declaration to those files, or use `--everything --covered-only` to mutate the whole project.
  • When would you choose mutates() over covers()?
    When you want a class mutated but do not want to filter the code coverage report for those tests. `covers()` affects both the mutation target and the coverage report; `mutates()` affects only the mutation target.
  • Why does pest --mutate need Xdebug or PCOV on the CI image?
    Pest needs per-test line coverage to know which tests cover each mutated line, and it gets that from a coverage driver. The docs list Xdebug 3.0+ or PCOV as requirements, so the CI image must have one installed and enabled for coverage.

saying these in an interview costs you the question

  • covers() and mutates() behave differently during a mutation run
  • pest --mutate runs every test file whether or not it declares a target
  • Mutation testing in Pest works without a coverage driver
  • --min only prints a warning and never fails the run
  • A 100 percent line coverage report means --mutate will score 100