skip to content

A Playwright aria snapshot fails after you intentionally rename the Export button; how do you get it green without weakening the check?

level: middleimportance: should knowfreq 41%

answer

  1. The failure is already a diff
  2. Decide intended or regression, per line
  3. Regenerate, do not delete
  4. Keep the update flag scoped
  5. Disappearing lines deserve most suspicion

basics

~20 s

Read the YAML diff in the failure, confirm every changed line is intended, then re-run that spec with --update-snapshots so the stored template is regenerated. Commit the regenerated template with the code change and review it as a normal text diff.

solid answer

~40 s

The failure prints a YAML diff between the stored template and the live tree, so triage starts there: a one-word rename should show one removed and one added line. If everything in the diff is intended, re-run the affected spec with `--update-snapshots` (`-u`), which regenerates the expectation from the current accessibility tree, rewriting the `.yml` file for a file-backed snapshot or the inline template for an inline one. Keep the flag scoped to the spec you touched, since running it suite-wide silently blesses unrelated regressions. Then re-run without the flag and commit the template alongside the code, where a reviewer can read it. What you must not do is delete the assertion or relax it into nothing, because failing on renames is exactly what it is for.

code

bash · 3 lines
bash
npx playwright test tests/statement.spec.ts --update-snapshots
git diff tests/statement.spec.ts-snapshots/main.aria.yml
npx playwright test tests/statement.spec.ts

go deeper

for a junior

Know that the failure output is a YAML diff and that --update-snapshots, or -u, rewrites the stored template. Never reach for the flag before you have read what changed.

for a middle

Explain the mechanics: which artefact gets rewritten for inline versus file-backed templates, and why the flag must be scoped to the spec you changed rather than the whole suite.

for a senior

Show the review discipline: paired removals and additions are renames, lone removals are suspected regressions, and a template that keeps churning is a scoping defect to fix rather than to regenerate.

for a principal

Own the policy: who may regenerate, what a pull request must show, and how the team avoids the failure mode where a green build is bought by blessing whatever the page currently does.

## First, read the failure A failed `toMatchAriaSnapshot` prints a diff between the template you stored and the tree the page produced, in the same YAML you wrote. That diff is the entire triage step, and it is worth reading line by line before touching anything: ``` - - button "Export CSV" + - button "Download statement" ``` Two lines is a clean, intended rename. Ten changed lines after a one-word rename is a different story, and it means the change did more than the ticket said. ## Regenerating the expectation Once the diff is understood, re-run the affected test with `--update-snapshots`, abbreviated `-u`. Playwright regenerates the expectation from the page's current accessibility tree: for a file-backed snapshot it rewrites the `.yml` under the test file's snapshot directory, and for an inline template it produces the updated template for the test source. ```bash npx playwright test tests/statement.spec.ts --update-snapshots ``` Scope matters. Running the flag over the whole suite regenerates every template that currently fails, which is exactly how an unrelated regression gets blessed on a busy branch. Run it for the spec, or the project, that your change touches. ## Review the regenerated template as code The regenerated YAML is text, so it lands in the pull request as ordinary added and removed lines. Use that: - Check that every removed line corresponds to something you meant to change. - Look hardest at *disappearances*. A rename shows a minus and a plus; a lost heading or a dropped region shows only a minus, and that is the shape of a real regression. - Check heading levels and states, `[level=1]`, `[disabled]`, `[checked]`, which change silently and read as noise if you are skimming. - Ask whether the new template is still as tight as it was, rather than having absorbed volatile content the old one had deliberately relaxed into a regex. ## When updating is the wrong move 1. **The change was not intended.** The correct fix is in the application, not in the expectation. 2. **The template is flapping**, changing on runs where nothing changed. That is a stability problem, solved by scoping the locator tighter or relaxing volatile values into regular expressions, not by repeated regeneration. 3. **You cannot explain the diff.** Regenerating an expectation you do not understand converts an assertion into a record of whatever the page happened to do. ## A workflow that holds up 1. Reproduce the failure locally and read the YAML diff. 2. Decide, per changed line, intended or regression. 3. If intended, regenerate with `-u` scoped to that spec. 4. Re-run without the flag, so the updated template is proved green on its own. 5. Commit the template change together with the application change, so a reviewer sees the rename and its expectation in one diff. ## Why this discipline is the whole value Deleting the assertion, or loosening it until nothing can fail, removes the only check that noticed the export button's label at all. The structural snapshot earns its keep precisely because it fails on renames, so the answer to a rename is to record the new name deliberately, not to stop looking.

  • What in a regenerated template diff should worry a reviewer most?
    Lines that only disappear. A rename produces a paired removal and addition, but a dropped heading, a vanished region, or a state attribute that quietly went away shows as a removal alone. Those are the shape of a regression being recorded as the new expectation, and they deserve a question before approval.
  • Why not run the update flag across the whole suite before a release?
    Because it regenerates every currently failing template, not just the ones your change explains. Genuine regressions elsewhere in the suite are rewritten into expectations and the build turns green with the defect committed. Scope the flag to the spec or project you touched, and let unrelated failures stay failures.

saying these in an interview costs you the question

  • Deletes the assertion instead of updating the template
  • Runs the update flag suite-wide without reading diffs
  • Says a regenerated template cannot be reviewed
  • Relaxes every node to a bare role to stop failures
  • Treats a shrinking template as noise rather than a regression