A CDN advertises the exact same IP address, say 203.0.113.10, from data centers in Frankfurt, Singapore, and Sao Paulo simultaneously via BGP, and relies on the internet's normal routing to send each user to the nearest one. Explain how this works, and describe a failure scenario where a user's connection to that IP breaks mid-session even though all three data centers are healthy.

## How anycast works **Anycast** is a routing technique in which the exact same IP address is announced from multiple physically distinct locations at the same time, and the network itself — not any single server or a DNS lookup — decides which location a given packet actually reaches. Every anycast PoP runs a **BGP (Border Gateway Protocol)** speaker that advertises reachability for the same IP prefix into the internet's global routing system. Routers all over the internet end up with several candidate paths to that one prefix — one via Frankfurt, one via Singapore, one via Sao Paulo — and BGP's normal best-path selection algorithm (which weighs factors like AS-path length, local routing policy, and other metrics, not literal geographic distance) picks one path per router. A user in Berlin, several network hops from Frankfurt but many more from Singapore or Sao Paulo, ends up with their packets naturally converging on the Frankfurt announcement, purely as an emergent property of ordinary internet routing decisions made independently by routers along the path — the CDN itself makes no per-request steering decision at all. ## Why it exists This mechanism exists because it solves two problems at once with almost no operational overhead: 1. **Automatic proximity routing** without needing a DNS layer that has to guess a client's location (a guess that can be wrong, especially when clients use a public DNS resolver far from themselves). 2. **Near-instant failover.** If a PoP goes down — or is deliberately taken out of rotation for maintenance — it simply stops announcing its BGP route for that prefix. Upstream routers detect the withdrawal and reconverge onto the next-best remaining announcement within normal BGP convergence time, typically seconds to tens of seconds, with zero visible change from the client's point of view: same IP, same DNS record, just a different physical server now answering it. This is also why large-scale DDoS mitigation providers lean heavily on anycast — an attack's traffic gets naturally spread across every PoP announcing the prefix rather than concentrating on one location, since each attacking source's packets follow the same nearest-path logic as any other client's. ## The trade-off The trade-off against this simplicity is that anycast has no concept of a session. Compare this to DNS-based geo-routing: | Approach | How the steering decision is made | |---|---| | **Anycast** | Every packet is routed independently based on whatever path currently looks best from wherever it enters the network; the anycast PoPs themselves don't share per-connection state with each other. | | **DNS-based geo-routing** | A resolver returns a specific unicast IP for a specific PoP once, at resolution time, and the client keeps using that same server (and thus the same underlying connection state) until the DNS record's TTL expires. | That is a decision made once, up front, versus anycast's continuous, per-packet, routing-layer decision that can react to network changes but is blind to what's happening at the application layer above it. ## The failure mode That blindness is exactly the failure mode in the scenario: TCP is a stateful, connection-oriented protocol, and a long-lived TCP connection is anchored to whichever PoP happened to answer the very first SYN packet. If, mid-connection, an ISP-level routing change causes that user's packets to suddenly take a different path — one that BGP now resolves to the Sao Paulo PoP instead of the Singapore PoP the connection was originally established with — the newly-receiving PoP has no record of that TCP connection's sequence numbers or state, and the connection simply resets. This is a real, if relatively rare, operational risk for connection-oriented, long-lived traffic like live video streaming or WebSockets, but it's much less of a concern for ordinary HTTP request/response traffic, where each connection typically completes in well under a second, leaving little window for a mid-flight reroute, and a dropped connection just means the client opens a fresh one. ## Where it shows up This is why teams building latency-sensitive, connection-heavy products on top of anycast infrastructure sometimes pair it with additional mitigations — protocols like QUIC carry a connection identifier independent of the underlying IP/port tuple specifically so a mid-session network change doesn't have to mean starting over, and some providers build explicit session-affinity or state-sharing layers on top of anycast for exactly this reason. Anycast is heavily used by major CDN and DNS providers — Cloudflare's and Google's public DNS resolvers (1.1.1.1 and 8.8.8.8) are classic examples, as are most large CDNs' HTTP edge IPs — precisely because the vast majority of traffic they carry is short-lived enough that the mid-connection reroute risk rarely matters in practice.