A schema-derived GraphQL mock keeps a client suite green - which real bugs does it hide?

## An incident, from the client side An insurance claims console renders a claim detail page: status, settlement amount, documents, and an adjuster-notes panel. The client suite runs against a mock derived from the schema and is green - every screen, every state, hundreds of assertions. In staging, signed in with a policyholder token rather than an adjuster one, the notes panel came back empty and the page then threw while formatting it. The cause was on the server: the permission check for `adjusterNotes` lives inside that field's own resolver, so it ran *after* the parent claim had already resolved. When it denied, the field came back null while every sibling field resolved normally, and the response carried both a populated claim and an entry in its errors list. That response is one the client had never seen. Not because nobody thought about permissions, but because **a schema-derived mock cannot produce it**. Every mocked response is a complete success in which every selected field holds a value of the right type. The render path had been written, reviewed and tested against exactly one class of payload. ## The blind spots, and why each one is structural **No viewer, so no authorization.** A mock answers from types. There is no caller, no token, no role, and the schema does not state which fields are restricted, so every field is served to everybody. Any bug where the answer depends on *who is asking* is invisible by construction. **Per-field values, so impossible states.** The mock fills each field independently. Nothing coordinates `status` with `settlementAmount`, or `closedAt` with `openedAt`. Two consequences, and the second is the one people miss: the mock will hand you combinations that cannot occur, *and* it will never reliably hand you the combinations that do. A screen whose layout branches on "settled with a payout" may simply never have been rendered in that state. **Always a success, so no failure paths.** A mocked response never arrives with a null where a value was expected, never carries an errors entry alongside data, and never fails mid-document. Every client branch that handles a partial or failed read is untested, which is precisely the branch that runs on a bad day. **Instant answers, so no cost signal.** Mocked fields return in microseconds and cost nothing regardless of depth. A document that fans out across a dozen backend calls, or that a server-side cost limiter would reject outright, is indistinguishable from a trivial one. A mock-backed suite cannot tell you a document fits a 340 ms p99 budget, and it will not warn you when a newly-added nested selection quietly triples the work. **Only as current as its input.** The mock answers from a copy of the schema. If that copy is three weeks old, the suite validates client documents against a fiction: it will accept selections of fields that have since been removed and reject fields that now exist. Staleness is a bug class of its own and the only one on this list you can fully close, by pulling the schema from the deployed source of truth in CI and failing on drift. ## What to do about it The answer is not a more elaborate mock. Each of these gaps can be *simulated* - you can make a field's mock function throw to produce a null and an error entry, you can hand-write a coherent settled claim, you can add artificial delay - and simulating them is worth doing for the handful of client paths that matter, because it exercises **your** handling. What simulation can never give you is evidence that the **server** behaves that way. Those are different claims, and interviews are usually probing whether a candidate can tell them apart. So split the confidence deliberately: - **Mock-backed, and generously**: layout, formatting, navigation, loading and empty states, list-length extremes, component contracts. Fast, offline, no server to run - this should be the bulk of the suite. - **Server-backed, and sparingly**: who may read which field, which field combinations are reachable, what comes back when something is denied or fails, and whether the documents you ship are within budget. Small, slower, and non-negotiable before release. - **Schema-backed and automated**: validate the documents the client will ship against the authoritative schema in CI, and keep the mocked copy refreshed from the same source. The one-sentence version, and a good thing to say out loud in an interview: a green mock-backed suite proves the client can render the schema. It does not prove the client works against the server, and treating it as release evidence is how a permission bug reaches staging with a hundred passing tests behind it.