skip to content

As CI steps, what do an OpenAPI differ and buf breaking have in common, and what does the build get back?

level: middleimportance: nice to knowfreq 28%

answer

  1. Same five stages, different artefact
  2. It must obtain a baseline first
  3. A rule set decides what counts
  4. Exit status plus typed change entries
  5. No consumer names come back

basics

~20 s

Every spec-diff check shares one shape: obtain a baseline revision, compare the candidate against it, classify each difference by a rule set, and turn that into an exit status. The build gets typed change entries plus pass or fail.

solid answer

~40 s

An OpenAPI differ such as `oasdiff`, `buf breaking` for Protobuf, and a graph API's schema check are the same CI step over different artefacts. Each one needs a **baseline** revision it can address - a release ref, a published module, a registry's current schema - parses it alongside the candidate, applies a **rule set** that decides what counts as breaking, and emits typed change entries with severities. The job gets back an exit status plus a report it can attach to a pull request. `buf breaking` makes the rule set explicit through categories, so you declare whether you are promising wire compatibility or the stricter file- and generated-code-level stability. What none of them return is a list of affected consumers, or any evidence about traffic: they compare descriptions, not deployments.

code

yaml · 6 lines
yaml
version: v1
breaking:
  use:
    - WIRE_JSON
  ignore:
    - internal/scratch

go deeper

for a junior

Recall that these checks run in the pipeline and compare two versions of an interface description. Being able to say that the build fails when a change is classified as breaking is enough here.

for a middle

Describe the shared step: baseline, parse, rule set, classification, exit status. Be able to say what the report contains and, just as importantly, what it leaves out.

for a senior

Talk about the choices inside the step - where the baseline comes from, which rule category the interface is promising, how accepted exceptions are recorded - and what each choice quietly changes about coverage.

for a principal

Argue for a consistent shape across many services and ecosystems, so a green check means the same thing everywhere, and own the cost of the rule sets and exception lists that decision creates.

## One shape, three ecosystems Breaking-change checkers look like different tools until you line up what they do in a pipeline, at which point they turn out to be the same five-stage step: 1. **Obtain a baseline revision** of the interface description - from a released git ref, from a published module artefact, or from a store that holds the last approved revision. 2. **Parse both revisions** - the baseline and the candidate produced by the branch under review - into a structure the rules can walk. 3. **Apply a rule set** that decides which structural differences count as breaking. 4. **Classify every difference** and emit a report of typed change entries. 5. **Turn the classification into an exit status** the job can gate on. What varies between family members is only the artefact and how the baseline is addressed: | Ecosystem | Artefact compared | Common checker | Baseline usually addressed as | | --- | --- | --- | --- | | REST | an OpenAPI document | `oasdiff`, `openapi-diff` | a file path, checked out from a release ref | | Protobuf / gRPC | the `.proto` sources of a module | `buf breaking` | a git ref, tag, or a published module image | | Graph API | the schema definition | the graph platform's schema check | the currently published schema in its registry | None of the three is a test. None of them starts the service, sends a request, or looks at traffic. They are static comparisons of authored descriptions, and that is the property that makes them cheap enough to run on every pull request. ## What the build actually gets back A spec-diff step hands the job two things and only two things: - **an exit status** - the gateable signal, normally non-zero when at least one difference was classified as breaking; - **a report of typed changes** - one entry per difference, each carrying a rule or change identifier, a severity, and a location inside the document. Most of these tools can emit that report in a machine format as well as text, so a pipeline can attach it to a pull request as an annotation. Just as important is the list of things it does **not** hand back: - it does not name a single affected consumer, because no consumer artefact was read; - it does not say whether the flagged operation or field is called by anyone; - it does not say whether the running service matches either document; - it does not know what a value *means*, only what type it is declared as. A team that reads the report as "these callers will break" is over-reading it. The honest reading is "these described elements changed, and the rule set classifies these as capable of breaking a caller". ## Rule sets: choosing the promise you are making The interesting knob in this family is the rule set, because it is where you declare which promise the interface is keeping. `buf breaking` makes this explicit by grouping its rules into categories, and you pick the category in the module's configuration. At the permissive end sit the rules that protect **wire compatibility** - what a serialized message on the network can still be read as. Above that sit rules that also protect the JSON representation, and above those the strictest categories that protect file- and generated-code-level stability, so that regenerating a client does not break compilation. Choosing a stricter category means more changes get flagged; choosing a looser one means the check keeps quiet about changes that break generated code but not the wire. An OpenAPI differ has a comparable dial in the form of which change classes it fails on. Either way, the rule set is a decision, not a default worth ignoring, and it belongs in a reviewed configuration file next to the interface it guards. Every checker in the family also has some way to record an **accepted exception**: a list of change identifiers to ignore, or paths excluded from the check. Two habits keep that from rotting: - keep the exception list in version control next to the description, so accepting a breaking change is a reviewed diff rather than an invisible one; - keep exceptions narrow and dated in a comment - a blanket suppression turns a gate into decoration, and nobody notices for months. ## Where the family stops Because all three checkers share a shape, they also share a limit. The report describes differences between two descriptions and stops there. It cannot tell you whether the change is a problem in practice, who would notice, or whether the deployed service ever behaved the way its description claimed. That is the seam where consumer-recorded contract verification and usage telemetry are added beside the differ, not the seam where a stricter rule set helps.

  • Why does the baseline source matter more than the checker you pick?
    Because the baseline defines what the check is protecting. Comparing against the previous commit only catches breakage introduced by that commit, so a long-lived branch can accumulate a breaking change while every individual diff stays clean. Comparing against the revision that is actually published asks the question that matters: can the callers of what is out there now still use what is about to ship?
  • What is wrong with a broad ignore list in a breaking-change configuration?
    It silently narrows the check's scope with no expiry. A path or rule excluded once tends to stay excluded, so the gate keeps reporting green over an area nobody is guarding any more. Keep exceptions narrow, keep them in version control beside the description so accepting one is a reviewed diff, and revisit them when the interface changes shape.

saying these in an interview costs you the question

  • Expects the report to list which consumers will break
  • Thinks the checker calls the service or replays traffic
  • Leaves the rule set at whatever the tool defaults to
  • Diffs against the previous commit instead of the published revision
  • Treats a blanket ignore list as an accepted exception