skip to content

A Pact file and an OpenAPI compatibility check both pass for the same endpoint - what does each one still let through?

level: seniorimportance: nice to knowfreq 31%

answer

  1. One executes the service, one does not
  2. Behavioural evidence versus a declaration
  3. Pacts only cover consumers who tested
  4. A diff never sees provider drift
  5. Additive fields still break strict consumers

basics

~20 s

A pact only covers interactions a consumer recorded, so it misses untested fields, consumers with no test, and meaning changes behind an identical shape. A specification diff never runs the service, missing drift and additive changes that break strict consumers.

solid answer

~40 s

They are different kinds of evidence. Verification **executes the provider** and compares real responses with a consumer's recorded expectations. A compatibility check **compares two documents** and never starts the service. A green pact still lets through anything no consumer exercised, any consumer who never published a test at all, and semantic changes under an unchanged shape — a tariff moving from pounds to cents keeps every type matcher happy. A green diff still lets through drift, because the document is not the service, and additive changes that break a consumer rejecting unknown fields. It also cries breaking over removals nobody consumed. So run both: the diff is the broad cheap floor covering consumers you cannot enumerate, the pact is deep behavioural proof on the edges that churn.

go deeper

for a junior

Recall that a pact is recorded by a consumer and checked against a running provider, while a specification diff only compares two documents and never starts anything.

for a middle

Explain what each artefact is evidence of: a pact about observed behaviour for consumers who wrote tests, a diff about declared changes across the whole surface for everyone.

for a senior

Be ready to name the real failure each one waved through on a system you operated, and describe how you covered the gap without simply running both blindly.

for a principal

Decide which guarantee the organisation buys for which class of interface, who pays for keeping the published specification honest, and how the two verdicts are allowed to interact.

## Two artefacts, two kinds of evidence A consumer-recorded pact and a compatibility check over an OpenAPI document look interchangeable on a slide and are nothing alike as evidence. One is produced by executing software; the other is produced by reading two files. | | Consumer-recorded pact | Specification compatibility check | | --- | --- | --- | | What is compared | The provider's real responses against a consumer's recorded expectations | The new specification document against the previously published one | | Is the provider executed | Yes, during verification | No, never | | Source of truth | What a consumer actually does | What the provider declares it does | | Consumers covered | Only those who wrote and published a test | Every consumer, known or not | | API surface covered | Only interactions someone exercised | The whole declared surface | | Typical verdict | This provider version satisfies this consumer version | This change is breaking or non-breaking | | Cost to a consuming team | They must write and maintain a test | None at all | The rows in bold contrast are the two you must be able to say out loud: **a pact is behavioural evidence about known consumers; a compatibility check is a declarative statement about all consumers.** Neither dominates the other, which is why both survive in mature estates. ## What each one lets through A pact lets through: - **Anything no consumer recorded.** A field removed from an endpoint that no consumer test exercises passes every verification in the estate. Coverage of a pact is the consumer's test coverage, no more. - **Consumers who never wrote a pact.** An internal team that skipped the practice, or an external client you never met, is simply absent from the evidence. - **Semantic change under an identical shape.** A price switching from pounds to cents, timestamps switching from UTC to local, an array whose ordering silently becomes significant: shapes match, meaning breaks. A specification compatibility check lets through: - **Drift.** The document is not the service. If the deployed code stopped matching its own specification, the check compares two accurate-looking documents and approves a change the running service will not honour. - **Additive changes that break a strict consumer.** Adding an optional response field reads as non-breaking in any diff, and fails immediately against a consumer that rejects unknown properties. - **Changes nobody needed to worry about.** The mirror image: removing a declared field that no consumer has ever read is reported as breaking, so the gate blocks a change that would have hurt nobody. That noise is what erodes trust in the gate. ## Why passing one says nothing about the other The two checks answer different sentences. Verification answers *did this build of the provider behave the way a real consumer expects*. A compatibility check answers *is this document a compatible successor to the last document*. Nothing connects them, because nothing forces the document and the build to agree in the first place. That independence is easiest to see in the two crossing cases. A provider can ship a perfectly compatible specification while its implementation returns something else entirely — the diff is green, verification is red, and verification is right. A provider can also pass verification against every recorded pact while deleting a documented endpoint that only an unmeasured external client uses — verification is green, the diff is red, and the diff is right. ## Using them together deliberately The two checks divide the risk cleanly, and the sensible arrangement is to run both and know which one you are trusting for what. 1. **Use the compatibility check as the broad, cheap floor.** It covers the whole declared surface and every consumer including the ones you cannot enumerate, and it costs a consuming team nothing. 2. **Use pacts as narrow, deep evidence on the edges that matter.** High-churn interfaces between teams that will maintain the tests get real behavioural proof. 3. **Close the drift gap explicitly.** The compatibility check is only trustworthy if the published specification tracks the running service, so generate or validate it from the implementation rather than maintaining it by hand. 4. **Do not let one verdict silence the other.** A team that treats a green diff as permission to skip verification has swapped behavioural evidence for a document review, and will discover the difference during an incident. The interview-grade summary: a pact tells you the provider really did what a consumer really needed, for the consumers who bothered to say what they needed. A compatibility check tells you the provider's declaration did not change in a way that would hurt someone, for every consumer, on the assumption that the declaration is true. The failures each one waves through are precisely the gaps the other fills.

  • The published specification and a verified pact disagree about an endpoint. Which do you trust?
    The verification, for behaviour: it executed the running provider and observed what came back, while the specification only records intent. The disagreement itself is the finding, because it means the document has drifted from the code. Fix the source rather than the symptom by generating or validating the specification from the implementation, so the compatibility gate stops issuing verdicts on a stale document.
  • How would you cover an endpoint that has a published specification but no consumer pact?
    Recognise it has no behavioural evidence at all. Either find the real consumer and get one recorded expectation for the path that matters, or accept the compatibility gate as the only guarantee and record that decision. The second is legitimate for a stable, low-churn endpoint; it is a bad default for the interfaces that change every sprint.

A specification diff reads the menu for changes between printings. A pact watches the kitchen actually cook the one dish a regular customer ordered.

saying these in an interview costs you the question

  • Calls a pact file just an OpenAPI document in another shape
  • Thinks a passing specification diff means consumers still work
  • Believes an additive field can never break a consumer
  • Assumes a published specification matches the running service
  • Says one of the two checks makes the other redundant