Why does Playwright's video: 'retain-on-failure' still cost time on tests that pass?

## What turning video on actually does Video recording in Playwright is a **browser context** capability, not a test-runner afterthought. When the runner builds the context for a test it either asks the browser to record it or it does not, and that decision is made before the first action runs -- before anyone could possibly know whether the test will pass. Because Playwright Test creates a fresh context per test, one context maps to one `video.webm` file, and a test that opens extra contexts produces one file each. That single fact explains the whole question. `'retain-on-failure'` is a **retention** policy, not a capture policy. ## The four modes and what each costs | Mode | Records on a first attempt | Keeps the file | Runtime cost on a green test | |---|---|---|---| | `'off'` | no | -- | none | | `'on'` | yes | always | full capture and encode | | `'retain-on-failure'` | yes | only when the test failed | full capture and encode | | `'on-first-retry'` | no | on the retried attempt | none on the first attempt | So there are two axes here, and candidates routinely collapse them into one: *whether frames are captured* and *whether the resulting file survives*. Only `'on-first-retry'` moves the first axis. ## What a passing test actually pays 1. The context is created with recording enabled, which allocates an encoder in the browser process. 2. The browser captures and encodes frames for the entire test, competing with the page for CPU on the same machine -- exactly the pressure that makes a marginal test flakier. 3. Frames are streamed to a `.webm` file under the test's output directory as the test runs. 4. When the test ends green, Playwright deletes the file. The saving is disk and CI upload time. The saving is **not** the recording itself, and on a busy parallel run with several workers the encoder contention is the part you feel. ## Why 'on-first-retry' is the cheap mode A retry runs in a **new context**, so the runner can decide to record that context and not the original. First attempts run at full speed with no encoder attached; only the tests that already failed once pay anything. The trade is real but usually acceptable: - you get no video for a failure that does not reproduce on retry; - you get nothing at all if the project has retries disabled; - you get a recording of the second attempt, which may differ from the first. For a food-delivery order tracker whose live status widget occasionally stalls, that is normally the right default in CI, with `'retain-on-failure'` reserved for a small suite of high-value journeys. ## Sizing the recording Frame size defaults to the viewport scaled down to fit inside 800x800. You can pin it: ```ts use: { video: { mode: 'retain-on-failure', size: { width: 640, height: 480 } }, } ``` A smaller frame means less to encode and a smaller file, at the cost of legibility -- small text in a delivery-status table can become unreadable. Video is good at showing *what happened in what order*: the map never rendered, the status jumped backwards, a dialog stole focus. It is bad at showing exact pixels, and it is not the tool for reading error text. ## When the file appears A recording is only finalised when its context closes, so the file is not usable mid-test. If you need it from inside the test, `page.video()` gives you a handle: `video.saveAs(path)` waits for the recording to finish and copies it, while `video.path()` names the eventual file. Under the test runner you rarely need either -- the runner attaches the video to the test result under the name `video` once the context has closed. ## Rules of thumb - Treat `'on'` as a debugging setting for a handful of tests, never a suite-wide default. - Read `'retain-on-failure'` as "record everything, keep the interesting ones". - Reach for `'on-first-retry'` when the run is CPU-bound or the suite is large. - Remember that a video is evidence of a run, not a reference image to compare against.