Once a Pact between two services verifies green, which end-to-end tests can you delete and which does it provably not cover?
answer
- Ask what the replay actually compared
- One pair, one message shape only
- Topology and stored data stay uncovered
- Keep multi-service journeys and migrations
- Delete duplicated single-pair shape assertions
basics
~20 sA verified pact retires end-to-end tests that existed only to check one consumer-provider message shape: fields, status codes, headers. It cannot cover deployment topology, data migrations, multi-service business flows or performance, which still need a real environment.
solid answer
~40 sVerification proves one thing precisely: for the interactions a consumer recorded, the running provider's real responses match. Everything you delete has to follow from that. What goes: end-to-end scenarios whose only assertion is that one service returns the fields another reads — response shapes, status codes, recorded headers. On a 23-service marina berth-allocation estate that was 61 of 147 nightly scenarios. What stays: journeys crossing three or more services, data migrations and backfills, deployment topology such as routing, TLS and auth wiring, and anything about latency or load. Also semantic changes behind an identical shape — a tariff switching from pounds to cents keeps every interaction green. And verification proves that *those two versions* can talk, not that those two versions are the ones deployed together.
go deeper
Recall that a contract test checks the messages exchanged between two services and runs without a full environment, which makes it far faster and steadier than an end-to-end run.
Be able to state exactly what a verification run compares, recorded requests and responses against the real provider, and therefore why only single-pair shape assertions become redundant.
Show judgement about which scenarios you delete, which you rewrite as consumer tests, and how you proved the deletion was safe rather than merely faster.
Own the estate-level target: how much end-to-end runtime the contract layer has to buy back, who signs off on deletions, and which residual risk you consciously accept.
## What a green verification actually establishes Provider verification replays the interactions recorded in a pact against the running provider and compares the real responses with what the consumer expected. The evidence it yields is narrow and precise: **for this consumer version and this provider version, across the interactions someone actually recorded, the messages line up.** Every deletion you make has to follow from that one sentence. What does follow is a specific class of end-to-end test: the ones whose only real assertion is that one service returns the fields another service reads. On a 23-service marina berth-allocation estate that class was 61 of 147 nightly scenarios — expensive proxies for a message-shape check, each one booting an environment to assert a berth id and a status string. A verified pact does that job per service pair, in seconds, with no environment at all. ## What it provably cannot cover | End-to-end concern | Covered by a verified pact | Why | | --- | --- | --- | | Field shape of a request or response between one pair | Yes | Exactly what the replay compares | | Status codes and headers on recorded interactions | Yes | Both are part of the recorded interaction | | The two versions actually running together | No | Verification compares versions, it does not deploy them | | Topology: DNS, TLS, gateway routing, auth wiring | No | The replay bypasses the real network path | | Data migrations and backfills | No | Stored state changes are outside the message | | Business flows spanning three or more services | No | A pact is bilateral by construction | | Latency, throughput and timeout behaviour under load | No | No load is generated and no clock is asserted | | Meaning behind an identical shape: units, currency, ordering | No | The shapes match while the semantics diverge | The last row is the one candidates miss. A provider that starts returning a berth tariff in cents rather than pounds keeps every recorded interaction green: the field is still a number, the type matcher still passes, and the consumer still shows a wrong price to a customer. Contract testing is a shape guarantee, and a shape guarantee is silent about meaning. ## A worked reduction The safe sequence on the marina estate looked like this, and the order matters. 1. **List the pairs that have a verified pact**, not the pairs that have a pact. Published and unverified is not evidence. 2. **Read the recorded interactions, not the pact's existence.** For each end-to-end scenario, find the interaction that covers the same request. If there is no interaction, the scenario is not redundant, however green the pair looks. 3. **Delete only single-hop shape assertions.** Of the 61 candidates, 47 were deleted outright and 14 were rewritten into consumer tests, because their real subject was the consumer's own handling of an error body. 4. **Keep a thin deployment smoke test.** One journey per environment that performs a real authenticated write proves the wiring that contracts cannot see, and it costs a fraction of what was removed. 5. **Re-measure.** The nightly run fell from 38 minutes to 12, and the flake rate fell with it, because most of the flakiness lived in the deleted class. ## What stays, and why it stays Four categories survive any amount of contract coverage. - **Cross-service journeys.** A berth reallocation that touches booking, allocation, tariffs and notifications is not the sum of three bilateral agreements. Each pair can be individually correct while the sequence is wrong, and no pact spans a sequence. - **State and migrations.** A pact says nothing about whether last month's berths were backfilled correctly, because the assertion is on a message, not on stored data. - **Topology and configuration.** Wrong hostname, expired certificate, missing scope on a token, a route that never reaches the service: verification replays past all of it. - **Performance and resilience.** Timeouts, retries, and behaviour under concurrency are all outside the recorded interaction. ## The trap: version pairing There is one more gap, and it is the reason a team that deleted well can still be surprised. A verification result attaches to a **specific consumer version and a specific provider version**. It is evidence that those two can talk, not that those two are the versions running side by side in an environment. Answering that second question is what the broker's `can-i-deploy` query exists for, and it needs the deployments recorded before it can answer at all. A team that deletes an end-to-end suite while still deploying blind has swapped one blind spot for another, and should expect the incident to arrive within a release or two. The honest summary to give an interviewer: a verified pact buys you the right to stop paying environment prices for a message-shape assertion, and nothing else. It removes the largest, slowest and flakiest slice of a typical end-to-end suite, which is why it is worth doing, and it leaves the journeys, the data and the wiring exactly where they were.
- Which end-to-end tests would you keep even if every service pair in the estate had a verified pact?Journeys that cross three or more services, because a pact is bilateral and a sequence can be wrong while every pair is right. Anything asserting stored state after a migration or a backfill. One thin deployment smoke test per environment that performs a real authenticated write, which is the only thing proving routing, certificates and token scopes. And the load profile, which no contract touches.
- How do you avoid deleting an end-to-end test for an interaction no pact actually records?Check the interaction, not the pair. A green verification between two services only covers the requests some consumer test exercised, so map each candidate scenario to a specific recorded interaction before deleting it. If no interaction covers that request, the scenario is the only evidence you have and it stays. Pact coverage is the consumer's test coverage, nothing more.
- A semantic change keeps every recorded interaction green. How would you catch it?Not with contract testing, which compares shapes. A tariff field moving from pounds to cents stays a number and passes every type matcher. Catching it needs either an explicit consumer assertion on a known value, a unit or currency encoded in the payload so the change becomes structural, or a business-level journey that checks the amount a customer sees.
A verified pact is a certificate that two parts fit each other. It says nothing about whether the assembled machine was wired up, whether it starts, or how fast it runs.
saying these in an interview costs you the question
- Claims contract tests replace the whole end-to-end suite
- Thinks a verified pact proves both versions are deployed together
- Deletes an end-to-end test for an interaction no pact records
- Believes verification covers latency, retries or timeouts
- Assumes a bilateral pact covers a multi-service business flow