What does an OpenAPI spec differ such as oasdiff actually compare, and what verdicts can it report?

## What a spec differ takes as input A **spec differ** compares two revisions of one machine-readable interface description and reports how the second departs from the first. For a REST API that description is an OpenAPI document and the usual tools are `oasdiff` and `openapi-diff`; the Protobuf equivalent is `buf breaking`, and a graph API has its platform's schema check. All of them take exactly two artefacts: - the **baseline** - the revision that is already published or already deployed, normally read from a released git ref or from a store that holds the last approved revision; - the **candidate** - the revision produced by the branch under review. Nothing else takes part. No provider process starts, no HTTP request is sent, no consumer's test suite runs, no production traffic is inspected. That is worth stating plainly, because the phrase "contract check" gets used for both this and provider verification against a recorded pact, and the two read completely different inputs. ## What it walks, and how it classifies what it finds The differ parses both documents into a structure and walks it: paths and operations, parameters and their required-ness, request bodies, response codes, response schemas and the types inside them, enum members, and security schemes. Every difference becomes a **typed change** - an entry naming what changed, where in the document it changed, and which of the tool's rules it matched. Each typed change then gets a **verdict**. The vocabulary varies between tools, but the classes are consistently three: | Verdict class | What it covers | Usual CI treatment | | --- | --- | --- | | Breaking (error) | a change the tool's rules say can break an existing caller of the described interface: a removed operation, a response field that disappears, a request parameter that becomes required | fail the job | | Non-breaking (warning) | a change the rules treat as safe for a tolerant caller but worth a human glance: a new optional parameter, an added response field | annotate, do not fail | | Informational | text carrying no interface obligation: descriptions, summaries, examples | report only | The taxonomy those rules encode is API-versioning practice. What matters about the tool is that it applies that taxonomy **mechanically, to a document**, with no judgement about whether the flagged change matters to anyone alive. Two mechanical consequences follow, and both surprise people: 1. **The comparison is not symmetric.** Which revision you pass as the baseline defines the question being asked. Baseline-then-candidate asks whether callers written against the old description still fit the new one. Swap the two and you are asking the opposite question, and you get a different set of changes back. 2. **Every verdict is about the description, not the deployment.** If the running service already disagreed with its own OpenAPI document, both revisions can be equally wrong and the diff will still come back clean. ## How that differs from what a consumer-recorded pact compares A pact file is a JSON artefact that a consumer's own test run produced: a list of interactions, each carrying a request the consumer said it would send, the response it said it expected, and a named provider state. Provider verification replays those interactions against a **running provider** and checks the responses that actually come back. So the two checks answer different questions from different inputs: | | Spec differ | Pact provider verification | | --- | --- | --- | | Inputs | two revisions of one description | one consumer's recorded interactions plus a running provider | | Question answered | did the described interface shrink? | does this provider still satisfy this consumer? | | Needs another team? | no | yes, a consumer must have written and published a pact | | Sees real responses? | no | yes | | Names who is affected? | no | yes, by consumer name, for consumers that have pacts | | Blind to | meaning, usage, the running service | anything no consumer wrote an expectation for | That difference decides where each is useful. The differ's strength is that it needs nobody's cooperation and covers the whole described surface, including operations nobody has ever called; its weakness is that it has no idea who calls what. Verification's strength is that a failure names a consumer; its weakness is that coverage stops at the expectations consumers actually recorded. ## Reading the result in CI What the build gets back is a report of typed changes plus a process exit status, and the step is normally configured so a breaking classification fails the job while everything else is annotation. Three things are worth doing with that output rather than just reading red or green: - keep the report as a build artefact, so the difference between two releases can be reconstructed months later; - record an accepted breaking change as an explicit, reviewed entry in the repository, rather than rerunning with the check switched off; - remember that a clean result is a claim about two files, and say so out loud when someone reads it as clearance to deploy.