A platform centralizes TLS termination, authentication, rate limiting, and caching into one API gateway layer in front of dozens of microservices. What does this concentration cost the system architecturally, and under what circumstances would you deliberately NOT centralize one of these concerns?

## Duplication traded for coupling Gateway offloading is fundamentally a trade of **duplication for coupling**. | Before centralization | After centralization | |---|---| | every backend service independently implements TLS, authentication, rate limiting, and caching, which means N slightly different implementations, N sets of bugs, N teams each needing security and HTTP expertise, but also N independent blast radii, a bug or outage in one service's auth code affects only that service | there's exactly one implementation of each concern, consistent everywhere, maintained by presumably one team of specialists, but now every service's behavior for that concern is only as correct, as available, and as fast as the shared gateway is | ## The first cost — a genuine single point of failure The most important cost is that the gateway becomes a genuine **single point of failure** for the entire system's public entry point. A bug in the gateway's rate-limiting logic, a bad certificate rollout, a memory leak under load, or a misconfigured deploy doesn't take down one service, it takes down every service simultaneously, because every request to every backend flows through this one component first. This is qualitatively different from a single backend service failing, which degrades one feature; a gateway failure is a full outage. Mitigating this requires the gateway itself to be built to a much higher availability bar than any individual backend service: - redundant instances across availability zones - careful canary/rolling deploys - circuit breakers and fallback behavior for its own dependencies (like the Redis store backing rate limiting) - aggressive monitoring, because its failure mode is total, not partial ## The second cost — ownership becomes a bottleneck The second cost is organizational, not just technical: centralizing these concerns usually means centralizing ownership too. One team, often called the platform or gateway team, now owns a component every other team's traffic depends on. This creates a **coordination bottleneck**: if a service team needs a new auth scope supported, a bespoke rate-limit shape, or a caching exception, they now depend on the gateway team's roadmap and risk tolerance, rather than just shipping a change in their own service. Mature organizations handle this by making the gateway highly configurable via self-service policy (declarative rate-limit configs per route, pluggable auth scopes) rather than requiring gateway-team code changes for every service's needs, but that configurability itself has to be built and maintained, which is real ongoing cost, not a one-time setup. ## The third cost — latency and design discipline The third cost is latency and request-path complexity: every request now takes an extra hop through the gateway before reaching its actual destination, and every additional cross-cutting check (TLS handshake already paid once at connection level, but auth lookup, rate-limit counter check, cache lookup) adds processing time on that hop. For most services this is negligible, single-digit milliseconds, but for genuinely latency-critical hot paths (real-time trading, some gaming backends), that overhead can matter enough to justify a different, more distributed approach. There's also a **design-discipline risk**: because the gateway is a convenient, central place to add logic, teams are tempted to push things there that don't belong, like fine-grained business authorization or multi-service response composition, drifting the gateway from cross-cutting concerns (offloading) toward domain logic or request orchestration (which is properly a separate concern, gateway aggregation, and arguably shouldn't live in the same component either, since it has very different scaling and failure characteristics). ## When to leave a concern where it is Deliberately not centralizing a concern makes sense in a few concrete circumstances. 1. **First**, when a concern genuinely needs deep per-service nuance that a shared gateway config can't express cheaply, for example wildly different rate-limiting shapes (one service needs a token-bucket keyed by a composite of user+resource, another needs a simple per-IP window), pushing that into a generic gateway policy layer can become more awkward than just implementing it locally, or via a library shared across services rather than a runtime gateway. 2. **Second**, when the latency cost of an extra hop is unacceptable for a specific hot path, some architectures route that path around the general-purpose gateway through a lighter, purpose-built entry point. 3. **Third**, and increasingly common at scale, a service mesh (Istio, Linkerd) offers a middle ground: instead of one central chokepoint gateway, each service gets its own sidecar proxy handling TLS, retries, and sometimes auth locally, giving the consistency benefits of centralization (one proxy implementation, uniformly deployed) without the single-chokepoint blast radius, since a sidecar crash only affects its own pod, not the whole fleet, at the cost of running many more proxy instances and a more complex control plane to configure them all consistently. Netflix's Zuul-then-service-mesh evolution and many large platforms' move from a monolithic API gateway toward mesh-plus-thin-edge-gateway architectures are real-world instances of organizations making exactly this trade-off as they outgrew a single chokepoint's blast radius and bottleneck.