At organizational scale, with many teams and multiple clusters or regions, how does gateway routing need to evolve beyond a single static routing table, and where does it start competing with service-mesh-based routing?
answer
- federate route ownership per team, validate centrally
- derive routes from service registry, don't hand-write them
- region selection sits above per-region service routing
- sidecar mesh avoids the shared central-gateway hop for internal traffic
basics
~20 sOne hand-edited routing table doesn't scale once many teams own routes and traffic spans regions. Routes need per-team ownership, dynamic publishing as services change, and sometimes a mesh of proxies per service instead of one central gateway.
solid answer
~50 sA single, centrally maintained routing table becomes an organizational bottleneck once many independent teams need to change routes: every change funnels through one shared config and one team's mistake can affect everyone. At scale, rules are typically generated dynamically from service discovery/registry data rather than hand-written, ownership is federated (each team's route definitions live with their service and get merged or validated centrally), and multi-region setups add a layer of routing above the per-region gateway to pick which region handles a request before that region does path-level routing. Beyond a point, some of the routing decision moves out of one central gateway into a service mesh, where a sidecar proxy next to each instance routes locally, avoiding the extra hop and blast radius of one shared choke point, at the cost of a more complex distributed system to operate.
go deeper
Not expected to have a developed answer here; a reasonable attempt might just note that a bigger system probably needs more automation than a manually edited config file.
Should recognize that routing rules can be generated rather than hand-written, and that regions add another layer above service-level routing.
Should articulate the ownership bottleneck problem and describe service-registry-driven routing plus the basic edge-gateway-vs-mesh distinction.
Should design the full layering (federated ownership, registry-driven rules, region selection above service routing, edge gateway vs. mesh split) and reason explicitly about the operational trade-offs and new failure modes each layer introduces.
## Where one routing table breaks down A single hand-maintained routing table works well for a system with a handful of services and one team owning the gateway, but it breaks down along several axes as an organization grows: - too many independent teams need to change routes for one shared file to remain safe, - a single-region gateway can't express where a request should even be sent when the system spans multiple regions, - and the fixed extra hop through one central gateway becomes an increasingly visible cost as internal service-to-service call volume dwarfs external client traffic. Addressing this is less about a single fix and more about three separate axes of evolution: **ownership**, **dynamism**, and **topology**. ## Ownership: federated authorship, central validation On ownership, a routing table maintained as one shared artifact edited by whichever team happens to need a change is a coordination bottleneck and a blast-radius risk: one team's typo in a rule can misroute traffic for a service they don't own or even know exists. At scale, this is addressed by **federating ownership**: each team defines the routes for the services it owns, typically as a small piece of declarative configuration co-located with that service's own deployment manifests, and a central process (validation tooling, a review gate, or an automated merge/aggregation step) combines these into the live routing configuration without requiring every team to touch a shared file directly. This mirrors how large organizations handle DNS zones or IAM policies: centralized enforcement of invariants (no two teams can claim overlapping routes, no route can point at an unregistered backend) with decentralized authorship. ## Dynamism: routes derived from the registry On dynamism, a hand-written routing table assumes someone remembers to update it every time a backend is deployed, scaled, or retired, which doesn't hold once services are created and destroyed frequently by many teams. The standard fix is to **generate routing rules from service discovery or registry data** rather than writing them by hand: a service registers itself (its name, version, health status, and often its intended route) with a central registry, and the gateway's routing table is continuously derived from that registry rather than manually authored. This closes the gap that causes stale-rule failures (a service is decommissioned but its rule lingers) because the rule's existence is now tied to the service's registration rather than to someone remembering a manual cleanup step, and it also lets routing react automatically to instance-level health, pulling a route out of rotation the moment its backend stops reporting healthy, rather than waiting for a human to notice and edit a rule. ## Topology: region selection above service routing On topology, multi-region or multi-cluster deployments add a layer above what a single gateway can express. A path- or header-based rule answers 'which service should handle this,' but a global system also needs to answer 'which region or cluster should handle this' first, based on factors like the client's geographic location, data residency requirements, or regional capacity/failover state. This is usually handled by a layer above the per-region gateways, DNS-based geo-routing, a global load balancer, or an anycast layer, that picks a region, after which that region's own gateway does the familiar path/header-based service routing locally. Getting this wrong shows up as cross-region latency (a client routed to a far region for no good reason) or, more seriously, **data-residency violations** if a request is routed to a region that isn't legally permitted to process it. ## Central gateway or service mesh The most consequential architectural decision at scale is whether some or all of the routing decision should move out of one central gateway entirely and into a service mesh. In a mesh architecture, each service instance runs alongside a **sidecar proxy**, and that sidecar makes routing decisions locally using policy that is centrally defined but locally enforced, rather than every call, including internal service-to-service calls, being forced through one shared gateway process. This eliminates the extra network hop and the shared blast-radius risk for internal traffic (a sidecar issue affects only the one service instance it's attached to, not every service behind a shared gateway), which matters enormously once internal call volume is an order of magnitude larger than external client traffic. The cost is operational complexity: instead of reasoning about one gateway's behavior, you're reasoning about a distributed system of many independent proxies that must all be configured consistently, and debugging a routing issue now means checking the specific sidecar involved rather than one central component. In practice, mature large-scale systems commonly run both: - a centralized gateway at the edge for external client traffic, where the decoupling and version-control benefits are highest and internal-hop cost doesn't apply, - and a service mesh for internal east-west traffic, where the sidecar's local-decision model avoids the shared-bottleneck cost that a single internal gateway would otherwise impose. Istio (built on the Envoy proxy) is the standard reference implementation of this pattern, expressing both edge gateway routing and mesh-internal routing through the same underlying proxy technology and a shared, centrally managed but locally enforced configuration model, which is precisely the ownership-plus-topology answer this scale of system needs.
- Why does deriving routing rules from a service registry reduce the stale-rule failure mode discussed elsewhere?A hand-written rule persists until someone remembers to delete it, even after its backend is gone, whereas a registry-derived rule exists only as long as the corresponding service is actively registered, so decommissioning a service and removing its rule become the same event instead of two separate steps someone can forget. It also lets the routing layer react to health-check failures automatically rather than only to explicit configuration changes.
- What's a concrete reason a large organization would keep a central gateway for external traffic but move internal traffic to a mesh?External client traffic is comparatively low-volume and benefits most from centralized version control, topology hiding from non-cooperative clients, and a single well-secured edge; internal traffic is typically much higher-volume and latency-sensitive, so forcing it through the same shared gateway multiplies both the latency cost and the blast radius of a gateway incident, which a per-instance sidecar avoids.
- What new failure mode does federating route ownership across teams introduce that a single central table didn't have?Two teams can define overlapping or conflicting routes independently without either noticing, since neither owns the full picture anymore, so the central merge/validation step has to actively detect conflicts (duplicate or shadowing routes across team boundaries) that used to be visually obvious in one shared file. Without that validation gate, federated ownership trades a coordination bottleneck for a cross-team collision risk.
It's the difference between one national postal sorting office handling every letter in a huge country versus a hierarchy: national hubs route by region, regional hubs route by city, and local carriers (the sidecars) make the last-mile decision themselves, each layer only handling the part of the decision it's best positioned to make.
saying these in an interview costs you the question
- Assumes one central routing table scales indefinitely regardless of organization size
- Doesn't distinguish region-selection routing from service-level routing within a region
- No awareness of service-mesh sidecar routing as an alternative to a central gateway for internal traffic
- Thinks federating route ownership has no new failure modes of its own
- Can't explain why routing rules should be derived from service discovery rather than hand-maintained at scale