skip to content

In Playwright, which `Reporter` hooks does a custom reporter implement to record every test's outcome?

level: seniorimportance: nice to knowfreq 32%

answer

  1. A class, not a callback
  2. Events arrive in the runner process
  3. Begin, per attempt, end
  4. Options reach the constructor
  5. The closing hook can change exit status

basics

~10 s

A class implementing Reporter from @playwright/test/reporter, wired through the reporter option. onBegin starts the run, onTestBegin and onTestEnd bracket each attempt, onStepBegin and onStepEnd cover steps, and onEnd then onExit close the run.

solid answer

~40 s

A custom reporter is a module whose default export implements `Reporter` from `@playwright/test/reporter`, listed as `reporter: [['list'], ['./payroll-reporter.ts', { slowMs: 60000 }]]` - the options object reaches the constructor. The lifecycle is `onBegin(config, suite)`, then per attempt `onTestBegin`, `onStepBegin`/`onStepEnd`, `onTestEnd(test, result)`, then `onEnd(result)` and finally `onExit()`. `onError` covers failures outside any test. Reporters run in the runner process, not a worker, so they see only serialized `TestCase` and `TestResult` data - status, duration, errors, attachments. `onTestEnd` fires once per attempt, so final counts belong in `onEnd` via `suite.allTests()` and each test's `outcome()`. `onEnd` is awaited and may return `{ status: 'failed' }` to override the run's exit status.

code

typescript · 31 lines
typescript
// payroll-reporter.ts
import { promises as fs } from 'fs';
import type { Reporter, Suite, TestCase, TestResult, FullResult, FullConfig } from '@playwright/test/reporter';

class PayrollReporter implements Reporter {
  private slow: string[] = [];
  private suite!: Suite;

  constructor(private options: { slowMs?: number } = {}) {}

  onBegin(_config: FullConfig, suite: Suite) {
    this.suite = suite;
  }

  onTestEnd(test: TestCase, result: TestResult) {
    if (result.duration > (this.options.slowMs ?? 60000))
      this.slow.push(`${test.titlePath().join(' > ')} ${result.duration}ms`);
  }

  async onEnd(result: FullResult) {
    const flaky = this.suite.allTests().filter(t => t.outcome() === 'flaky').length;
    await fs.writeFile('results/slow-tests.txt', this.slow.join('\n'), 'utf8');
    console.log(`payroll run ${result.status}: ${this.slow.length} slow, ${flaky} flaky`);
  }

  printsToStdio() {
    return true;
  }
}

export default PayrollReporter;

go deeper

for a junior

Know that reporters are pluggable: a class implementing the Reporter interface, listed in the reporter option, gets called as tests begin and end. You rarely write one before you have used the built-ins hard.

for a middle

Explain the hook order from onBegin through onTestEnd to onEnd and onExit, and that options in the reporter entry arrive as the constructor argument.

for a senior

Show the production instincts: tally in onEnd rather than onTestEnd because attempts repeat, buffer writes instead of doing IO per test, and keep a built-in reporter alongside yours while the custom one is unproven.

for a principal

Decide when a bespoke reporter is warranted at all versus consuming a built-in machine format, and own the maintenance cost of a component that sits in the exit path of every pipeline.

## A reporter is a class in the runner process A custom Playwright reporter is a module whose default export is a class implementing the `Reporter` interface from `@playwright/test/reporter`. It runs in the **runner process**, not in a worker, so it never sees a `page`, a fixture or a browser — only the serialized `TestCase` and `TestResult` objects the workers report back. Anything a reporter needs from the browser has to have been attached during the test. ## Wiring it up - `reporter: './payroll-reporter.ts'` runs yours **instead of** the built-ins; list them together to keep both: `reporter: [['list'], ['./payroll-reporter.ts']]`. - Options ride in the entry — `['./payroll-reporter.ts', { slowMs: 60000 }]` — and arrive as the constructor argument. - Implement `printsToStdio()` and return `false` when your reporter writes only to a file, so Playwright knows it can add its own progress output. ## The hooks, in order 1. `onBegin(config, suite)` — once, before any test runs; `suite` is the whole tree, so `suite.allTests().length` is the planned test count. 2. `onTestBegin(test, result)` — a test attempt starts. 3. `onStepBegin(test, result, step)` / `onStepEnd(...)` — each `test.step`, nested. 4. `onStdOut(chunk, test, result)` / `onStdErr(...)` — output captured from the worker. 5. `onTestEnd(test, result)` — the attempt finished, with its status, duration, errors and attachments. 6. `onEnd(result)` — once, after everything; `result.status` is `passed`, `failed`, `timedout` or `interrupted`. 7. `onExit()` — last of all, after every reporter's `onEnd`, for a final flush. `onError(error)` fires for errors that belong to no test, such as a failure while loading a spec file. ## What the objects carry | Object | Useful members | | --- | --- | | `TestCase` | `title`, `titlePath()`, `location`, `annotations`, `expectedStatus`, `outcome()` | | `TestResult` | `status`, `duration`, `errors`, `attachments`, `steps`, `stdout`, `retry` | | `FullResult` | `status` for the whole run | The trap is arithmetic: `onTestEnd` fires **once per attempt**, so a suite that retries reports some tests more than once. Counting there gives inflated totals. Do the final tally in `onEnd` by walking `suite.allTests()` and reading each test's `outcome()`, which resolves to one verdict per test. ## Finishing the run `onEnd` may be `async` and Playwright awaits it, which makes it the place to write a file or post a summary — not a fire-and-forget `void` call in `onTestEnd`. It can also return `{ status: 'failed' }` to override the run's exit status, which is how a reporter enforces a policy of its own, such as a slow-test budget on a payroll suite, without failing any individual test. `onExit()` is the very last chance to flush, after every reporter has finished. ## Practical rules - Never block: work done in `onTestEnd` runs inline with the run. - Buffer in memory, write once in `onEnd`; a file opened per test is a reliable way to make a large suite slower. - Use `titlePath()` rather than `title` for identity — titles repeat across files. - Keep a built-in reporter alongside yours until the custom one is proven; losing the terminal output while debugging a reporter is a bad afternoon. - Read attachments from `result.attachments` rather than re-deriving paths; the reporter sees the copies, not the originals.

  • Why can a custom reporter not touch page or a fixture while handling onTestEnd?
    Reporters run in the runner process, not in the worker that executed the test, and by the time `onTestEnd` fires that worker's browser and fixtures are already being torn down. All the reporter receives is serialized `TestCase` and `TestResult` data. Anything needed from the page must have been attached during the test itself.
  • What does returning { status: 'failed' } from onEnd do?
    It overrides the run's final status, so the process exits non-zero even when every test passed. That is how a reporter enforces a policy of its own - a slow-test budget, a missing-artifact check - without having to fail an individual test to express it.

The reporter is a stenographer sitting in the courtroom, not a participant: it hears everything said and writes the record, but it cannot question the witness.

saying these in an interview costs you the question

  • Thinks a reporter runs in the worker with page access
  • Assumes onTestEnd fires exactly once per test
  • Believes adding a custom reporter keeps the built-ins running
  • Writes a file per test inside onTestEnd
  • Invents hooks such as onSuiteEnd that do not exist
  • Treats onEnd as fire-and-forget rather than awaited