skip to content

When should a Playwright suite fulfil from fixtures instead of fetching and mutating the real API?

level: principalimportance: nice to knowfreq 24%

answer

  1. Speed and honesty pull opposite ways
  2. Three stances a handler can take
  3. Green forever against a dead contract
  4. Where drift is allowed to surface
  5. Provenance and refresh of a fixture

basics

~20 s

Fulfil from fixtures for everything that runs on every commit, because those tests must be fast, hermetic and deterministic. Reserve fetching and mutating the real response for a thin, separately scheduled layer whose job is noticing that the upstream contract has drifted.

solid answer

~40 s

Each handler takes one of three stances: fulfil from a fixture, `route.fetch()` and mutate, or `route.continue()` untouched. Fixtures are fast, offline and deterministic but cannot notice that the third-party forecast API changed shape; fetch-and-mutate keeps the real payload and catches drift, at the cost of a dependency that can fail for reasons unrelated to your change. The workable mix is fixtures for behaviour tests on every commit, plus a small scheduled layer that touches the real service so drift surfaces as an expected failure rather than a production incident. Reach for fetch-and-mutate when the payload is large and the edit is small — one field forced to an extreme is more honest than a hand-typed 400-line fixture. Make the stance explicit per test, and make fixture provenance traceable.

code

typescript · 21 lines
typescript
import { test, expect } from '@playwright/test';
import base from './fixtures/forecast.oslo.json';

// Commit-gate test: hermetic, fixture-fulfilled.
test('renders the forecast', async ({ page }) => {
  await page.route('**/api/forecast**', route => route.fulfill({ json: base }));
  await page.goto('/dashboard?city=oslo');
  await expect(page.getByTestId('temperature')).toHaveText('21°C');
});

// Scheduled contract check: real payload, minimal edit.
test('@contract live payload still drives the dashboard', async ({ page }) => {
  await page.route('**/api/forecast**', async route => {
    const response = await route.fetch();
    const live = await response.json();
    live.current.tempC = 45;
    await route.fulfill({ response, json: live });
  });
  await page.goto('/dashboard?city=oslo');
  await expect(page.getByTestId('heat-warning')).toBeVisible();
});

go deeper

for a junior

Know that stubbing from a fixture keeps a test fast and repeatable, and that a test which calls a real third-party service can fail for reasons that have nothing to do with your change.

for a middle

Explain the trade concretely: fixtures give determinism but go stale, fetching the real response catches shape changes but adds a dependency, and each has a place in a suite.

for a senior

Show where you draw the line — hermetic tests on the commit gate, a thin scheduled layer against the real service — and how you keep fixture provenance and refresh from becoming nobody's job.

for a principal

Own the whole policy: the mix, where each layer runs, who reacts when the contract check fails, and how many handler layers a suite may grow before nobody can answer who answered a request.

## Three stances a handler can take Every route handler in a suite is making one of three bets about a dependency such as a third-party forecast API behind a weather dashboard. - **Fulfil from a fixture** — the test owns the payload. Fast, hermetic, deterministic, runs offline. - **Fetch and mutate** — `route.fetch()` gets the real response, the handler edits it and fulfils with it. Real shape, targeted control, real dependency. - **Continue untouched** — the request goes to the real service. Maximum fidelity, minimum control. The lead's job is deciding the mix, not decreeing one answer. A suite that is 100% fixtures passes forever against a contract that no longer exists. A suite that is 100% live is a status page for someone else's service. | stance | determinism | catches contract drift | cost of ownership | |---|---|---|---| | fixture fulfil | high | no | fixtures must be refreshed deliberately | | fetch and mutate | medium | yes, incidentally | needs network, credentials, patience | | continue | low | yes | flaky, slow, unownable in CI | ## How the decision usually resolves 1. **Default to fixtures for behaviour tests.** Anything asserting rendering, state transitions, formatting or error UI does not need a real server, and the tests that run on every commit must not depend on one. 2. **Keep a thin layer of contract checks that touch the real service**, run on a schedule rather than per commit, so drift surfaces as a specific, expected failure instead of as a production incident. 3. **Reach for fetch-and-mutate when the payload is large or the edit is small** — one field forced to an extreme value against an otherwise real body is far more honest than a hand-typed 400-line fixture. 4. **Make fixtures traceable to a real capture.** A fixture whose provenance nobody remembers is a liability; one generated from a recorded response and dated is an asset. ## The failure modes to name out loud - **Silent drift.** The upstream adds a required field, the fixtures never do, and the suite stays green while the dashboard breaks. This is the tax you pay for hermeticity; the contract layer is what pays it. - **Fixture sprawl.** Every test forks its own copy of the payload, so one contract change means editing forty files. Centralise the base payload and let tests override the field they care about. - **Accidental live traffic.** A handler that falls through to `continue()`, or a URL pattern that misses, quietly sends CI traffic to a third party. Worth a guard that fails the run on unstubbed calls. - **Over-fitting to the mutation.** Fetch-and-mutate tests that assert on unedited parts of a live payload fail whenever the weather changes. Assert on what you forced, not on what the service happened to send. ## What to hold the team to Make the stance explicit per test rather than per suite, so a reader can tell at a glance which requests are real. State where the fixtures came from and how they are refreshed. Budget the live layer honestly — its purpose is early warning about the contract, so treat a failure there as information about the dependency, not as a broken test to retry. And keep the number of places a request can be answered small: the more layers of handlers a suite grows, the harder it is to answer the only question that matters in a debug session, which is "who answered this request, and with what?".

  • How do you stop fixtures from drifting silently away from the real contract?
    Give every fixture a provenance — generated from a real captured response, dated, and refreshed on a schedule — and keep a small live layer that would fail when the upstream shape moves. Centralise a base payload so tests override only the field they care about; forty forked copies make a contract change unaffordable.
  • How do you prevent a suite from quietly sending CI traffic to a third-party service?
    Assume any unmatched request is a mistake. A catch-all handler that aborts or fails the run on unstubbed calls turns silent live traffic into an obvious error, and it also catches the fallback that reached the end of the chain or the URL pattern that stopped matching after a query parameter changed.
  • What makes a test built on fetch-and-mutate flaky in practice?
    Asserting on parts of the payload you did not force. If the test checks the live temperature, the wind or the ordering of a list, it fails whenever the weather changes. Assert on the field you edited and on the UI state it should produce, and keep the mutation as small as possible.

saying these in an interview costs you the question

  • Treats a fully mocked suite as proof the integration works
  • Runs live third-party calls on every commit
  • Keeps fixtures nobody can trace to a real response
  • Forks the whole payload into every test file
  • Retries a failing contract check instead of reading it
  • Asserts on live data the service changes hourly