skip to content

For a given API, how do you choose between a Pact verification gate and a spec-diff gate, and when do you run both?

level: principalimportance: should knowfreq 40%

answer

  1. Cost is measured in cooperation, not CPU
  2. One is unilateral, one an agreement
  3. Whole surface versus named blast radius
  4. Unknown consumers rule out verification
  5. Neither sees a change of meaning

basics

~20 s

Choose by the cooperation you can obtain. A spec-diff gate needs only two revisions of the description and no other team. A Pact gate needs consumers publishing contracts and a provider build verifying them, and it names who breaks.

solid answer

~50 s

A spec-diff gate is cheap and unilateral: two revisions of the description, a rule set, seconds of CPU, nobody else's calendar. It covers the whole described surface, including endpoints no consumer test touches, but knows nothing about who calls what. A Pact verification gate costs far more - each consumer must write and publish a pact, the provider must run verification with working provider-state handlers, and a broker must hold the artefacts - and it buys the one thing the differ cannot produce: a failure that names the consumer. So the decision follows the consumer population. Unknown or external consumers means the differ is the only gate you can actually run. A short list of internal consumers who will maintain their contracts justifies verification. Most mature setups run both, because the differ has total surface coverage with no usage knowledge and verification has real usage knowledge over a partial surface.

code

bash · 6 lines
bash
# cheap and unilateral: did the described interface shrink?
oasdiff breaking published/openapi.yaml candidate/openapi.yaml

# expensive: needs consumer-published pacts and a provider fixture,
# and answers a different question - which consumer breaks?
./gradlew pactVerify

go deeper

for a junior

Recall that the two checks answer different questions: one compares two versions of a description, the other checks a provider against expectations a consumer recorded. Knowing that they are not interchangeable is enough here.

for a middle

Explain what each gate needs before it can run and what a failure from each one tells you, and be able to say why a differ covers endpoints no consumer test does.

for a senior

Argue the operational trade-off with evidence: coordination cost across teams, how a verification gate decays when a consumer goes quiet, and what you would put in place to keep either honest.

for a principal

Own the funding decision and the coverage claim that goes with it. State plainly which surface is guarded by which gate, and refuse to let a partly-populated result be reported as coverage it never had.

## Two gates, two very different bills Both gates stop a bad release, but they are not comparable purchases. Line up what each one actually asks of the organisation: | | Spec-diff gate | Pact verification gate | | --- | --- | --- | | Inputs it needs | a baseline revision and the candidate description | pacts published by consumers, a running provider, a broker holding the artefacts | | Whose cooperation | nobody's beyond the repository that owns the description | every consumer team that must write and publish a pact, plus the provider build | | Time to stand up | hours - a step, a baseline source, a rule set | weeks to months, mostly other teams' calendars | | Runtime per build | seconds | seconds to minutes, plus provider-state fixtures to keep working | | What a failure tells you | this described change is classified breaking | this named consumer's recorded expectation is no longer met | | Coverage of consumers you do not know about | complete, since it does not depend on consumers at all | none | | Coverage of meaning | none | only what a consumer explicitly asserted | | Ongoing maintenance | rule set and the accepted-exception list | provider-state handlers, consumer test upkeep, broker hygiene | The decisive row is **whose cooperation**. A spec-diff gate is a decision one team can make on a Tuesday. A pact gate is an agreement between teams, and it degrades quietly the moment one consumer stops maintaining its contract. ## Reading the choice off the consumer population - **Few consumers, known, in reach, willing to invest** - the pact gate earns its cost. A failure naming the consumer is worth far more at three in the morning than a report saying a field disappeared. - **Many consumers, or external, or unknown** - the differ is often the only gate you can actually run, because you cannot make anyone write a contract test. It also covers the whole described surface, including the endpoints nobody remembers exist. - **Consumers who will not write tests but you do publish a description** - cross-checking a provider's published specification against expectations recorded from consumer side gives a weaker guarantee than full verification, but a real one, and it does not require the consumer to run a provider fixture. - **A large asynchronous surface** - on a settlement platform with 6 HTTP consumers and an 88-topic event bus, the sensible split is usually not one gate for everything: the synchronous surface can carry consumer contracts where the consumer list is short, while the event surface leans on schema checks because the readers are numerous and anonymous. ## Where the coordination cost really lands The cost of a pact gate is rarely the CPU. On that same settlement platform, the provider team sat 9 hours ahead. A consumer expectation published at 17:00 local time turned the provider's build red overnight, and the two teams then negotiated across a 3-hour overlap window each day. The mitigations are real - marking new expectations so they cannot fail the provider's build until it has verified them once, and letting the provider fetch only the consumer versions it cares about - but they are work, and they are work the differ never asks for. A spec-diff gate has no cross-team failure mode at all, because there is no second team in the loop. That asymmetry is the honest answer to "why not always Pact?". Verification buys you a named blast radius, and you pay for it in other teams' attention. ## Why both is the usual end state They fail in opposite directions, which is exactly why the pair is stronger than either: 1. The **differ runs early and cheaply** on the description, catching the accidental removal before review, for the entire surface including the parts no consumer test covers. 2. **Verification runs later** against real consumer expectations, and answers the question the differ cannot: whether this specific change breaks a named, identified consumer. 3. Each covers the other's blind spot - the differ has total surface coverage and zero usage knowledge; verification has real usage knowledge over a partial surface. The limit of the pair is worth saying aloud, because it prevents the false sense of completeness: neither gate sees a change of meaning in an unchanged shape. That is why a convention - new field for new semantics, never a redefinition in place - does work that no gate does. ## What you are actually deciding The choice is not "which tool is better", it is which of these you would rather be true when something goes wrong: - a gate that flags a change nobody was using, costing a conversation - the differ's characteristic false alarm; - or a gate that stayed green because the consumer who broke had never published a contract - verification's characteristic silence. Fund the differ everywhere, because it is nearly free and covers the surface completely. Fund verification where the consumer relationship is close enough that both sides will keep it alive, and be explicit about the surface it does not cover. When consumer coverage is partial, say so in the same breath as the verdict: a gate whose coverage is unstated is more dangerous than one that is absent.

  • Your provider serves 14 external integrators you have no build relationship with. Which gate can you actually run?
    Only the spec-diff gate. Consumer-recorded verification requires those integrators to write contracts and publish them, which you cannot compel and should not assume. Run the differ on every change to the published description, publish a machine-readable changelog from its report so integrators can diff releases themselves, and treat any accepted breaking change as an explicit, reviewed record rather than a quiet exception.
  • How do you stop a Pact gate from decaying once it is in place?
    Watch the coverage, not the colour. Track which consumers have published a contract recently and which have gone quiet, because a consumer that stopped publishing leaves a green result that means nothing. Keep provider-state handlers owned by the provider team as production code, and treat a permanently excluded consumer as a coverage decision to be stated, not as a build annoyance to be silenced.
  • If you could only fund one gate this quarter, which and why?
    The spec-diff gate, in almost every case. It is the cheaper one, it needs no other team's agreement, and its coverage of the described surface is complete rather than partial. Verification is the higher-value signal but it is an organisational commitment; buying it half-heartedly produces a partly-populated matrix that reads as safety and is not.

saying these in an interview costs you the question

  • Compares the two gates on runtime instead of cooperation cost
  • Assumes every consumer can be made to publish a contract
  • Treats a green verification result as full consumer coverage
  • Thinks a spec differ can name the consumers that break
  • Believes running both closes the semantic-change gap