How do you stop an accidental breaking change to a JSON HTTP API from reaching production? Describe the automated checks you would put in a CI pipeline.
answer
- schema as build artifact + baseline diff
- fail on removal/narrowing/new-required
- pacts: what consumers actually assert
- usage telemetry answers 'safe to delete?'
- units and defaults need value-asserting tests
basics
~20 sGenerate the API schema from code, diff it against the released baseline in CI, and fail the build on breaking diffs. Add consumer-driven contract tests so real client expectations are verified, and back both with production traffic analysis for undocumented usage.
solid answer
~50 sThree layers, in increasing cost: **1. Schema diffing.** Generate the OpenAPI document from the code on every build and diff it against the last released version with a compatibility linter. Removals, type narrowing, new required request fields and status-code changes fail the build; additions pass. This catches syntax-level breaks cheaply and needs no client cooperation. **2. Consumer-driven contract tests.** Each consumer publishes the interactions it actually relies on; the provider's pipeline replays them against the real implementation and fails if any expectation is unmet. This catches what the schema cannot: which fields anyone actually reads, so you also learn what is safe to remove. **3. Semantics.** Schemas cannot see units, meaning, or default changes. Cover those with provider tests asserting concrete values, plus review discipline. Complement all three with production telemetry — per-field and per-endpoint usage by client — so removals are evidence-based rather than hopeful.
go deeper
Know that the schema should be generated and diffed in CI, and that removals should fail the build.
Describe the diff classification rules and the difference between schema-level and consumer-driven checks.
Cover the semantic blind spot, production usage telemetry, and how pacts justify deletions; make the gate blocking with an audited override.
Position it as governance: who owns the baseline, how consumers are onboarded to pacts, and the cost trade-off between a strict gate and delivery speed.
## Why review is not enough Breaking changes are usually accidents: a refactor renames a DTO property, a serializer config change starts omitting nulls, a validation annotation is tightened, an enum is converted to a stricter type. None looks like an API change in the diff. The defence has to be mechanical. ## Layer 1 — schema generation plus compatibility diff Make the API description a **build artifact**, generated from the running code (annotations, routes, DTOs), not a hand-written file that drifts. Store the last released version as a baseline; on every pull request, regenerate and diff. Classify the diff automatically. Fail on: removed path or operation, removed response property, narrowed type, new required request property, added enum value in a *request* the server now rejects elsewhere, removed response status code, changed error schema. Pass on: new operation, new optional request property, new response property. Emit warnings for grey-zone changes such as new response enum values so a human decides. This is cheap, deterministic, runs in seconds, and needs no consumers to participate. Its blind spot is everything not expressible in the schema. ## Layer 2 — consumer-driven contract tests Each consumer team writes tests against a mock of the provider and publishes the resulting **pact**: the requests it makes and the response fields it asserts on. The provider's CI replays every published pact against the real service. If the provider stops returning a field a consumer asserts, the *provider's* build goes red — before deployment, without a full integration environment. The strategic value is the inverse question. A schema diff tells you `legacy_id` was removed; the pact broker tells you nobody has read `legacy_id` for six months, so removing it is safe. That turns "we can never delete anything" into an evidence-based decision. The cost is organisational: it only works if consumers maintain their pacts and versions are tagged per environment so you know what is actually deployed. ## Layer 3 — semantic checks Units, meanings and defaults are invisible to both layers above. Guard them with provider tests that assert **concrete values** — a fixture whose `amount` is 1250 and whose `currency` is EUR, a default page size assertion, an assertion that an unknown id yields 404 and not 200-with-empty-body. When such a test fails, the change is intentional and needs a version or deprecation plan. ## Production evidence Instrument the edge: request counts per endpoint, per version, per client, and where feasible per response field consumed (via field-selection parameters, GraphQL-style projections, or SDK telemetry). This is the only source of truth about undocumented usage and it is what makes the sunset decision defensible. Pair it with a staging "chaos" mode that injects unknown fields and new enum values into responses, so intolerant clients fail during testing. ## Making it stick The gate must be blocking, with a documented, deliberately awkward override that records the justification and notifies consumers. If bypassing is easy, the check degrades into a warning nobody reads, and the first person to learn about the break is a customer.
- What can a schema diff not catch?Anything not expressed in the schema: units and semantics (seconds to milliseconds), default values such as page size or sort order, ordering and pagination behaviour, rate-limit or timeout changes, and which fields consumers actually depend on. Cover these with tests that assert concrete values and with consumer-driven contract tests.
- How do consumer-driven contract tests help you delete a field rather than just protect it?The pact broker records exactly which fields each consumer asserts on, so a field with no pact and no production telemetry is a demonstrably unused field. That evidence converts removal from a risk argument into a data-backed decision, which is what keeps an API from accumulating fields forever.
saying these in an interview costs you the question
- Relying on code review to catch breaking changes
- Hand-maintaining the OpenAPI file separately from the code so it drifts from reality
- Treating the compatibility check as advisory rather than build-blocking
- Assuming a green schema diff proves compatibility, ignoring semantic changes
- Confusing end-to-end integration tests with contract tests — they are slower and still miss unused-field evidence