A new harness runtime runs beside the old one while cases migrate — what must be true for both to stay green?

## What a harness runtime is, and what it is not A **harness runtime** is the machinery a case executes on: the component that discovers cases, schedules them, fires setup and teardown at defined points, decides the unit of parallelism, and writes outcomes somewhere. It is not the suite. The suite is the *behaviour* the cases encode — the business actions they perform against the deployed system and the outcomes they assert. That distinction is the entire migration strategy: you replace machinery while preserving behaviour, and every decision below follows from protecting the part that took years to accumulate. Running a new runtime **beside** the old one is the strangler shape applied to your own test code. Both live in the same repository, both run in the same pipeline, both drive the same deployed target, and cases move in batches. At any moment the pack is part old and part new, and what makes that survivable is that a reader can still answer one question when something goes red: *is the product broken, or is the migration broken?* ## Four invariants that keep both sides green 1. **One implementation of behaviour, two entry points.** Before a single case moves, lift the business actions, the data setup and the assertion helpers into a layer that neither runner owns. A case in either runtime becomes a thin entry point that calls into that layer. A fix then lands once and both runtimes see it. The alternative — copying each case and editing both from then on — diverges at the first behaviour change and doubles the upkeep you were trying to reduce. 2. **One deployed target and one data story.** Both runtimes drive the same running system and create their data the same way. Give the new runtime its own target and every divergence acquires a second candidate explanation, so you can no longer read a split as evidence about the migration. 3. **One result store.** Both runners report into the same place, each outcome tagged with which runtime produced it. A divergence then presents itself as a searchable fact — the same case passing on one side and failing on the other — instead of as two reports nobody compares. 4. **One green bar.** Merges gate on both runtimes for as long as both exist. The moment the old runner's outcome stops blocking a merge it stops being maintained, and the fallback you are paying for quietly evaporates. | Choice during the overlap | What it buys | What it costs | | --- | --- | --- | | Shared behaviour layer, thin entry points | One fix reaches both runtimes; divergence means the runtime, not the code | An extraction pass before any case moves | | Copied case bodies per runtime | Nothing to design up front; first cases move immediately | Every later fix applied twice; silent drift between halves | | Single deployed target | A split is attributable to the runtime | Contention between the two runs on shared data | | Separate targets per runtime | Runs cannot interfere | Two explanations for every divergence; the signal is lost | ## Telling a migration defect from a product defect The characteristic event is a migrated case that fails on the new runtime while the old one passes on the same commit. Re-run the pair on a commit the old runtime was green on; if the split persists on unchanged code, the difference lives in the migration surface, not the product. That surface is small and enumerable: discovery, the order setup and teardown fire in, the unit of parallelism, how the new runner waits for something to become actionable, and how a failure is captured. Then reduce the case: strip it to a single shared action call, assert nothing, and add layers back until the split reappears. What you must not do is label the split intermittent. A reproducible runtime-specific difference is a migration defect wearing a convenient name, and the label lets it survive to cutover, when it is indistinguishable from the hundred other things that broke that day. ## The overlap window is the expensive half Every moved case runs twice: doubled pipeline minutes, two configurations to keep valid, and two chances for an unrelated intermittent failure to block a merge. That cost buys a controlled cutover and is worthless as a steady state, so the overlap needs a shape: - Publish the count of unmigrated cases on a fixed cadence, and treat a flat week as a decision point rather than as a delay. - Move a case and delete its old copy in the **same** change, so no case is ever knowingly maintained in two places. - Keep an explicit list of cases the new runtime genuinely cannot run, with the capability each is waiting on. A growing list means the new runtime is missing something the plan assumed, which you want to know in week three rather than week thirty. - Set the end date at the start. A side-by-side with no end date is not a migration; it is a decision to run two suites forever, taken by nobody. At the end, delete the old runner, its configuration, its reporting adapter and any helper that existed only to serve it — in one change, not gradually. Half a retired runtime is something every future reader must investigate before being sure it is dead.