An OpenAPI diff gate reports no breaking changes for a settlement API release. What can that check still not see?
answer
- It compares structure, never meaning
- Same shape, different units or basis
- Ordering and nullability are undocumented
- The document is not the deployed build
- Clean diff, unknown consumers
basics
~20 sA clean spec diff proves only that the description did not shrink. It cannot see a field whose shape stayed identical while its meaning changed, behaviour clients rely on that the document never described, or whether the deployed service matches it.
solid answer
~50 sThe differ compares declared structure, so three things pass it in silence. First, **semantic change**: on a livestock-auction settlement service, `settlementAmount` switched from gross of the buyer's premium to net - same type, same required-ness, diff completely clean, every downstream ledger wrong by the premium. Second, **undescribed behaviour**: collection ordering, page stability, whether absent and `null` mean the same thing, which error bodies really appear, response timing. None of it is in an OpenAPI document, so none of it can be in the diff. Third, the document **is not the deployment** - if the service had already drifted from its own description, both revisions are equally wrong and the check still passes. And since no consumer artefact is read, the differ cannot say whether anyone calls the thing it flagged, or the thing it did not.
code
json · 12 lines{
"release_2026_08": {
"lotId": "L-40318",
"settlementAmount": 12474.50,
"currency": "GBP"
},
"release_2026_09": {
"lotId": "L-40318",
"settlementAmount": 11824.17,
"currency": "GBP"
}
}go deeper
Recall the one-line limit: the check compares two description files, so anything not written in those files is invisible to it. That includes what a number actually means.
Explain the three blind spots with an example each - unchanged shape with changed meaning, undescribed behaviour such as ordering, and drift between document and deployed build.
Show you have been burned: describe the evidence you put beside the gate, and be careful in how you report the result so a statement about two files is not read as a statement about a live system.
Own the conventions that make semantics visible across many teams - renaming rather than redefining - and decide what other evidence a release requires when the gate's guarantee stops this far short.
## The claim a clean diff actually makes When an OpenAPI differ reports no breaking changes, the sentence it has proved is narrow: *between these two revisions of this document, no difference matched a rule classified as breaking*. Three separate things sit outside that sentence, and each of them has taken down a release. ## Blind spot one: the shape is unchanged, the meaning is not A differ compares declared structure - paths, types, required-ness, enum members. It has no access to what a value means. So the most dangerous class of change passes it in silence: same field, same type, same required-ness, different semantics. On a livestock-auction settlement service, `settlementAmount` was quoted gross of the buyer's premium. A release changed it to net. Identical schema, identical types, no rename, no removal - the differ reported nothing at all, and every downstream ledger that had been reconciling on that number was quietly wrong by the premium on every lot for two days. The family this belongs to is larger than it looks: - **units or basis changed** - gross to net, pence to pounds, inclusive to exclusive of tax; - **an enum member reused** - the member name survives while what it signifies is redefined; - **a status code kept, meaning widened** - `200` that used to mean settled now also covers partially settled; - **an identifier whose issuing authority changed** - same string type, different namespace, so lookups silently miss; - **a nullable field that starts arriving null** - the document permitted it all along, so nothing changed on paper. Not one of those is visible to a structural comparison, and no stricter rule set will find them. The counter-move is a convention rather than a tool: when meaning changes, change the name, so the change becomes structural and the gate can see it. ## Blind spot two: behaviour the description never had words for Consumers depend on far more than a description contains. An OpenAPI document says a response is an array of lots; it rarely says the array is ordered by settlement time, yet a client is paginating on that assumption. Typical undescribed behaviour that real clients rely on: - the **order** in which a collection comes back, and whether that order is stable between pages; - whether an **absent** field and an explicit `null` are treated as the same thing; - which error bodies actually appear, when only the success responses were ever documented; - **idempotency** and retry behaviour on a repeated request; - timing - a call that used to return in tens of milliseconds and now takes seconds is not a schema change, and a client with a tight read timeout still breaks. None of that is in the document, so none of it can be in the diff. A change can leave the description byte-identical and still break every caller. ## Blind spot three: the document is not the deployment The differ compares two files. It never asks the running service anything. Two consequences: - if the implementation had already drifted from its own description, both revisions are equally wrong and the diff is still clean; - the differ knows nothing about **who** calls the interface. It flags an operation no client has touched in a year exactly as loudly as the one every client uses, and stays silent about a consumer relying on undocumented behaviour. ## Why "no breaking changes" is not "safe to deploy" The two sentences quantify over different things: | The gate says | What it proves | What it does not prove | | --- | --- | --- | | no breaking changes detected | these two descriptions differ only in ways the rule set calls safe | that the deployed build matches either description | | no breaking changes detected | the described surface did not shrink | that unchanged fields still mean the same thing | | no breaking changes detected | nothing structural broke on paper | that any actual consumer is unaffected, since no consumer was consulted | So what belongs beside the gate, if the gate is doing its job and still not enough? 1. **Consumer-recorded verification** for the consumers you know about: a pact failure names a consumer, which is the one thing a differ can never produce. 2. **Usage evidence** per operation and per field, so an accepted removal is a decision about observed traffic rather than a guess. 3. **A naming convention for semantic change**, so redefinitions become renames and therefore become visible to the structural check. 4. **A short observation window after release** on the numbers that matter, because a value whose meaning shifted shows up as a reconciliation discrepancy long before anyone reads a schema. The professional habit is to report the gate's result in the words it earned. "The spec diff is clean" is true and useful. "The spec diff is clean, so this is safe to release" quietly upgrades a statement about two files into a statement about a running system and a population of consumers the check never looked at.
- How would you make a semantic change visible to a structural gate?Turn it into a structural one. Add a new, differently named field carrying the new meaning, leave the old field with its original semantics until callers have moved, then remove the old field as a deliberate, flagged breaking change. The differ then sees an addition, and later a removal, both of which its rules understand. Redefining a field in place is the only variant no gate can catch.
- A diff flags the removal of an endpoint that telemetry shows nobody has called in months. What now?The differ has done its job by raising it; the decision needs evidence it cannot supply. Check whether the telemetry covers every caller path, including batch jobs that run monthly and any client that would only call it in an error path. If the evidence holds, record the removal as an explicitly accepted exception in the repository rather than loosening the rule set for everything.
- Does a stricter rule set close any of these gaps?No. Every gap here is outside the comparison, not below its threshold. Strictness only changes which structural differences get classified as breaking; meaning, undocumented behaviour and the state of the deployed service remain invisible however strict the rules are. Tightening rules to compensate mostly produces noise and trained-in overrides.
It is a stocktake that counts the boxes and never opens one: the count matches perfectly while the contents have been swapped.
saying these in an interview costs you the question
- Reads a clean diff as clearance to deploy
- Believes a stricter rule set would catch a meaning change
- Assumes the document matches the running service
- Thinks a differ knows which endpoints are actually called
- Redefines an existing field instead of adding a new one