Your team is considering adding caller-controlled field selection and relation expansion to a high-traffic public JSON API. What are the practical limits and operational risks of that kind of over-fetch control, and how would you contain them?

## What over-fetch control actually buys Sparse fieldsets and expansion trade a fixed representation for a caller-chosen one. The upside is genuine: less bandwidth, fewer round trips, one canonical resource instead of a zoo of screen-shaped endpoints. But every one of those wins is bought with something, and at public-API scale the bills arrive in five places. ## 1. Caching HTTP caching keys on the effective URL. `GET /articles/42` is one entry; `GET /articles/42?fields=title` and `?fields=title,body` and `?fields=body,title` are three more — and note that parameter *order* and whitespace make textually different URLs for semantically identical selectors unless you normalise. A resource that used to be a single hot CDN entry becomes a long tail with poor hit rates, so origin load rises even though each response is smaller. Mitigations: canonicalise the selector server-side (sort and dedupe field names, redirect or rewrite to the canonical form), restrict caching to a small allow-list of selectors, or accept that shaped responses are private/uncached and only the default representation is edge-cached. ETags must be computed over the *projected* body; reusing the full-representation ETag will serve a client the wrong shape on a conditional request. ## 2. The contract becomes a family of contracts A schema is a promise about a response. Once fields are optional at the caller's discretion, the promise becomes conditional: "`author` is present iff you asked for it, and is a string unless expanded, in which case it is an object." Typed client generators either mark everything optional — pushing null-checks into every consumer — or generate per-selector types. Contract tests can no longer enumerate the space. The practical answer is to keep the *set* of shapeable fields small and stable, always return an identity core (`id`, type, version/ETag-relevant fields) regardless of selector, and never let selection change field *names* or nesting, only presence. ## 3. Cost becomes a client-controlled input This is the operational risk that actually pages someone. Expansion multiplies backend work: one request with four expanded paths over a 100-item page can become hundreds of downstream calls. A naive rate limit counting requests-per-second no longer bounds load, because requests are no longer comparable. Containment: assign each request a **cost weight** derived from page size × (1 + expansions × depth), charge that weight against the caller's quota, enforce a per-request fan-out ceiling and deadline, and disallow expansion entirely on the highest-traffic list endpoints. Reject rather than degrade: a `400` naming the too-expensive selector is far better than a timeout at p99. ## 4. Authorization moves down to the field With a fixed representation, you can authorize once per endpoint and redact a known set of fields. With caller-chosen shapes, every requested field and every expanded path needs its own check, evaluated for this caller on this resource. The failure mode is subtle: a caller asks for a field they cannot see and the code, having already fetched the row, serializes it. Design the projection layer so the permitted field set is computed first and the requested set is *intersected* with it — never the reverse. Decide and document the behaviour for unauthorized-but-requested fields: omit silently (leaks nothing, confuses clients) or `403` (clear, but reveals the field exists). ## 5. Savings you did not actually make If the handler loads the full aggregate and prunes at serialization, you cut bytes and nothing else — often the smaller share of the cost. Real projection pushes down: select fewer columns, skip joins that only feed omitted fields, skip downstream service calls whose results were projected away. That plumbing is the expensive part of the feature and the reason many teams get less benefit than they expected. ## The pragmatic middle Most APIs do not need a fully general projection language. A workable ladder: (1) trim the default representation so it is genuinely useful; (2) add two or three named presets (`view=summary|full`) that are cacheable and testable and absorb the majority of traffic; (3) add narrow, allow-listed expansion for the handful of relations clients demonstrably need; (4) only then consider general field selection, and instrument which selectors are actually used so you can prune the grammar later. Measure before and after — payload bytes, origin hit rate, p99 latency, downstream call count — because the feature can easily make the system slower overall while making individual responses smaller.