How do you size the cost of moving a legacy insurance-quote suite to Playwright before committing?

## Why per-test averages lie The instinct is to multiply test count by an hours-per-test figure. It fails because migration cost is not distributed evenly: in a legacy insurance-quote suite, a hundred straight form-fill-and-assert tests port almost mechanically, while a dozen tests around document upload, generated PDF quotes, third-party payment redirects, nested iframes and multi-tab flows consume most of the calendar. An average built on the easy majority produces an estimate that is wrong in the direction that hurts: optimistic, and defended in public. ## Port a hard slice first 1. Pick ten to twenty tests chosen for **difficulty and variety**, not for convenience: the file-upload case, the iframe case, the popup case, the authenticated case, the one that hits an external redirect, and the flakiest test in the suite. 2. Have the people who will do the real work port them, on the real CI, until they are green three consecutive runs -- not green once on a laptop. 3. Record hours per test **per category**, and note what got deleted or merged along the way. 4. Extrapolate by category against the real inventory, then add a tail factor for the tests nobody has read in two years. ## Inventory the couplings, not just the tests Test bodies are rarely the expensive part. The estimate has to price everything the suite leans on: - Page objects and helper libraries -- how many are structural, and how many exist only to paper over waiting? - Custom waits and retry loops, which usually collapse into web-first assertions and shrink the port. - Grid-specific plumbing: remote endpoints, capability matrices, tunnels into a private environment. - Test data setup and environment fixtures, which are usually reusable and should not be re-estimated. - Reporting and dashboard integrations that the business reads, and the CI wiring around them. - Docker images, browser installation and cache warming on the runners. ## The coexistence bill Almost every real migration runs two suites for a while, and that window is a cost line of its own: | Cost line | How to measure it | Typical shape | |---|---|---| | Duplicate CI minutes | current job cost times the overlap window | linear, predictable | | Duplicate triage | failures per week times minutes per failure, doubled | the one that hurts | | Split ownership | how many people must know both toolchains | grows quietly | | Coverage drift | tests changed in the old suite after the port started | needs a freeze rule | Two rules keep that window finite: freeze feature work on the old suite once the port starts, and give the window a date with a named owner rather than "when we get to it". ## Write the decision down before you start - State the estimate as a range with its assumptions, and name what would invalidate it. - Name the exit criteria: the ported suite is green at the same coverage for two weeks, at or below the old wall-clock time, and the team can triage a failure without help. - Name the kill switch: what result from the slice means you stop and keep the current suite. - Book the pruning decisions separately -- deciding which tests no longer earn their runtime is a suite-health question that should not hide inside a migration estimate. ## The number that actually persuades An estimate lands when it is expressed as payback rather than effort: "six engineer-weeks, against four hours a week of grid maintenance and roughly a day a week of failure triage we expect to halve". The slice gives you the numerator, the current suite's own telemetry gives you the denominator, and a migration that cannot state both is being argued on taste.