You're designing a rate limiter for an API gateway deployed across three geographic regions, each with many stateless gateway instances, enforcing one global per-API-key limit. A single centralized Redis in one region adds cross-region latency to every request and becomes a single point of failure. Walk through the design trade-offs: what happens if you use per-region local counters instead of a shared store, and what happens if the central Redis becomes unreachable?

## Two compounding problems At gateway scale — many stateless instances across multiple regions, high request-per-second volume — the naive "every request checks a single centralized Redis" design runs into **two compounding problems**, and understanding both is what separates a limiter that survives production from one that becomes the next incident. ## Latency, and the gap local counters open The first problem is **latency**: if Redis lives in one region, every request originating in a different region pays a cross-region round trip, commonly tens of milliseconds, purely for the rate-limit check, before any actual business logic runs. At high request volumes this either becomes a meaningful tax on p99 latency for every single request, or forces batching/pipelining tricks that add their own complexity and staleness. The mechanism-level fix people reach for first is **per-region local state**: each region's gateway instances check a rate limiter backed by local memory or a regional Redis, incurring no cross-region hop. This is fast, but it creates a correctness gap: - the whole point of a global per-API-key limit is that it should reflect the client's total usage across all regions, - a purely local counter has zero visibility into what the same client is doing in the other two regions. A client that discovers, deliberately or by accident (e.g. a multi-region deployment of their own client fleet), that requests get load-balanced across regions can send up to N times their intended limit, where N is the number of independent regional counters, simply by spreading traffic across them. ## The mitigations, and what each one buys The standard mitigation is to accept an explicit accuracy/latency trade-off rather than pretend both can be had perfectly. 1. **Static budget partitioning** — divide the global limit across regions in proportion to expected traffic share, e.g. a 300/sec global limit becomes 100/sec enforced independently in each of three regions, which is simple and requires zero cross-region communication, but is wasteful when traffic is uneven: a region with little traffic wastes its allotted budget while a hot region's clients get throttled even though global capacity remains. 2. **A more adaptive approach** periodically, e.g. every 1-5 seconds, aggregates regional counts to a shared view and rebalances allotments, trading some staleness — a client can transiently exceed the true global limit within that sync interval — for much better utilization than static partitioning, at the cost of running an additional aggregation/sync pipeline that is itself infrastructure to build, monitor, and fail-handle. 3. **A third pattern** avoids the discrete-window problem altogether by using GCRA (Generic Cell Rate Algorithm, the algorithm behind the popular throttled and redis-cell libraries) — mathematically equivalent to token bucket but tracked as a single "theoretical arrival time" value per key rather than a counter, which composes cleanly with a centralized store and gives smooth, precise rate control without discrete window boundary artifacts, but it still requires the same centralization-vs-locality decision as any other shared-state approach. ## When the shared store is unreachable The second, orthogonal problem is what happens when the centralized store itself becomes unreachable — a Redis failover, a network partition between a region and Redis's home region, or Redis simply falling over under load. This decision must be made explicitly and in advance, because both extremes are dangerous. - **Failing closed** — rejecting all requests when the limiter can't be consulted — converts a rate-limiter dependency outage into a full API outage, which is almost always the wrong trade for a component whose entire purpose is protective, not core functionality; you've let a supporting system become as critical as the primary path. - **Failing open** — allowing all requests through unchecked — removes exactly the protection the limiter exists to provide at precisely the moment (Redis under stress, likely correlated with a broader incident) when uncontrolled traffic is most dangerous. The production-grade answer used by most large-scale gateways is neither pure extreme: wrap the Redis dependency in a **circuit breaker** with a short timeout, and on trip, fall back to degraded local-only limiting, each instance enforcing a conservative local approximation of its share of the global budget, for the duration of the outage, then resume centralized checks once Redis recovers — accepting slightly worse fairness/accuracy during the incident window in exchange for neither a full outage nor unlimited exposure. ## The deeper principle The deeper principle this scenario illustrates is that at sufficient scale, exact global rate limiting and low added latency are in direct tension, and every real design — static partitioning, periodic sync, GCRA with regional replicas, circuit-broken fallback — is a different point on that trade-off curve rather than a way to escape it, which is exactly why this question separates candidates who've only implemented a single-instance limiter from those who've operated one at scale.