API Gateway

## What it is and how a request flows through it An **API Gateway** is a server that sits at the network edge of a microservices system, positioned between external clients (web apps, mobile apps, third-party integrators) and the internal fleet of backend services. Mechanically, it is a **reverse proxy**: 1. Every inbound request first hits the gateway, which **inspects the request** (typically the URL path, HTTP method, host header, or a version prefix). 2. It **looks up a routing table** that maps that pattern to a specific backend service, port, and protocol. For example, requests to `/orders/*` might route to the `order-service`, while `/users/*` routes to the `user-service`. 3. The gateway then **forwards the request** to the resolved backend, waits for the response, and relays it back to the client, often rewriting headers or paths along the way (stripping an internal-only prefix, adding a correlation ID for tracing). ## Why it exists **Why it exists:** before gateways became standard, clients called services directly, which meant every client needed to know the network address of every service, handle many different TLS certificates, and reimplement cross-cutting logic (authentication, throttling, request logging, CORS handling) once per service. That duplication is expensive and error-prone — a bug in the auth check on one service doesn't get fixed everywhere at once. A gateway collapses this to a single well-known entry point: - one **DNS name**; - one **TLS termination point**; - one place to enforce a **security policy**. It also decouples the client-visible API surface from internal topology. Internally, an order-processing capability might be one service today and split into `order-service` and `payment-service` tomorrow; as long as the gateway's routing table is updated, no client code changes. This is the same value reverse proxies like **Nginx** have always provided, extended with API-aware features: - **JSON-aware transformations**; - **service discovery** integration; - API-specific policies like **per-route rate limits**. ## Trade-offs, in both directions Trade-offs run in both directions. - **On the upside:** centralizing auth, rate limiting, and TLS termination means those concerns are implemented and audited once instead of many times, and operational visibility improves because all traffic funnels through one observable point (a single place to add access logs, metrics, and distributed tracing headers). - **On the downside**, the gateway becomes a mandatory hop added to every request's **latency budget** — even a well-tuned gateway adds a few milliseconds of proxying overhead, and a poorly tuned one can add tens of milliseconds. - It also becomes a **single point of failure** and a **scaling bottleneck**: if the gateway goes down, every service becomes unreachable from outside even though the services themselves are healthy, so gateways are typically deployed as a horizontally scaled, stateless fleet behind a load balancer, with no single instance holding critical state. - There is also an **organizational risk**: because the gateway is shared infrastructure, changes to its routing table or policies require coordination across teams, and it can become a bottleneck for release velocity if every team must go through a central gateway team to ship a new route — a common criticism when gateways evolve into what people call a 'distributed monolith' chokepoint. ## Failure modes in production Failure modes in production show up in recognizable patterns. 1. **First, routing table drift:** a service is renamed, redeployed on a new port, or decommissioned, but the gateway's route entry isn't updated, causing 502/504 errors that look like a backend outage but are actually a stale gateway config — this is why gateways are increasingly paired with service discovery (Kubernetes Services, Consul) rather than static route tables. 2. **Second, cascading overload:** if the gateway does synchronous work per request (token introspection calls to an auth server, for instance) and that auth server slows down, the gateway's own thread/connection pool can exhaust, taking down routing to every service, not just the one whose dependency is slow — this argues for timeouts, circuit breakers, and caching validated tokens at the gateway layer. 3. **Third,** misconfigured rate limits or CORS rules at the gateway can silently break legitimate traffic in ways that are hard to debug because the failure happens before the request ever reaches application code or logs. ## Where you have seen it A concrete real-world example: **Netflix's Zuul** edge-routing layer sits in front of the streaming service's hundreds of microservices, handling dynamic routing, load shedding, and authentication for all inbound traffic from client devices, so that individual services like the recommendation engine or playback service never have to implement their own edge security or throttling. Similarly, **AWS API Gateway** is commonly placed in front of a fleet of Lambda functions or ECS services precisely so that a mobile app talks to one HTTPS endpoint instead of directly invoking dozens of individually-addressed functions.