A PHPUnit 13 suite passes in default order but fails intermittently with executionOrder="random"; how do you reproduce the failure and find its cause?

## Why random order finds bugs By default PHPUnit 13 runs tests in the order it discovers them. That order is stable, so a hidden coupling - test B passes only because test A ran first and left something behind - never shows up. `executionOrder="random"` in `phpunit.xml`, or `--order-by random` (alias `--random-order`) on the command line, shuffles test classes and methods. A test that now fails intermittently is **order-dependent**: it relies on state it did not create, or it breaks state another test relies on. In a PHP test run all tests share **one process**. Anything that outlives a single test is a candidate: - **static properties**, including singletons, service locators, in-memory caches and registries; - **superglobals and environment**: `$_SERVER`, `$_ENV`, `putenv()`; - **process-wide settings**: `date_default_timezone_set()`, `setlocale()`, `ini_set()`, error handlers; - **external state**: files in a temp directory, rows in a shared test database, a fake clock left advanced. ## Step 1: reproduce deterministically Every randomized run prints the seed in its header: ``` Random Seed: 1758723977 ``` If you did not supply one, PHPUnit uses the current time. Replay the exact order with: ```bash vendor/bin/phpunit --order-by random --random-order-seed 1758723977 ``` Passing `--random-order-seed` without random order does nothing useful, and PHPUnit says so with a PHPUnit warning, which fails the run by default. Record the seed in CI logs so any red build can be replayed locally. ## Step 2: find the polluter 1. Run the replay with `--debug`, which replaces progress output with the event stream, including the order in which tests are prepared and finished. 2. The culprit is among the tests that ran **before** the failing one. Bisect: run those tests together with the failing test, halving the set until one earlier test is left. Narrowing the set of tests can change the shuffle even with the same seed, so check that each reduced run still fails rather than trusting the seed alone. 3. As a diagnostic, enable `backupStaticProperties="true"` (and `backupGlobals="true"` if superglobals are involved) with `beStrictAboutChangesToGlobalState="true"`. PHPUnit then snapshots state around each test and reports the test that changed it as risky, with a diff of what changed. The backups are too slow and too magical to keep on permanently in most suites, but they point straight at the leak. ## Step 3: fix isolation, not the order | Leak | Fix | |---|---| | Static cache or singleton | Reset it in `tearDown()`, or inject the dependency instead of reaching for static state | | `$_SERVER` / env changes | Save and restore in `setUp()`/`tearDown()` | | Time zone or locale | Set explicitly in each test that depends on it, restore afterwards | | Shared database rows | Wrap each test in a transaction, or create unique data per test | Pinning the suite back to default order hides the defect; the next refactor that adds a test in the wrong place brings it back. ## Why the seed matters in CI A randomized CI job without a recorded seed produces failures nobody can replay: the next run uses a different order, passes, and the report is closed as a flake. Keep the header line in the job log, and when a randomized run fails, copy the seed into the bug report together with the failing test's name. Some teams also pin the seed per pipeline run so that a retry of the same job uses the same order; that is fine as long as the seed still changes between pipelines. ## Interaction with other ordering features - **Dependencies:** `resolveDependencies` defaults to true, so a test with `#[Depends]` still runs after its producer in a shuffled run. - **Defects first:** `executionOrder="defects"` runs tests that failed last time first, using the recorded test run history in the cache directory. It is often combined as `defects,random`. - **Reverse:** `--order-by reverse` is a cheap, deterministic way to shake out simple A-before-B couplings. ## Where this sits in CI For a legacy project, turn random order on once the suite is green in default order, keep the seed in the job log, and treat an order-dependent failure as a real bug with an owner. It is a SENIOR topic because the fix usually changes production code: static state that makes tests order-dependent also makes long-running workers and queue consumers behave differently from a fresh request.