Connecting 40 branches to a cloud IPsec VPN gateway that offers two tunnel endpoints per branch, how do you design tunnels, routing and failover, and what does each choice cost?
answer
- two endpoints, two tunnels, one decision
- let routing pick the live tunnel
- detection: IKE liveness, BGP hold, BFD
- active/active versus active/standby
- the branch is the remaining single point
basics
~10 sBuild two route-based tunnels per branch with any-to-any selectors, run BGP over each, and let route withdrawal move traffic. Then decide active/standby versus active/active, how fast failure is detected, and how routes are filtered.
solid answer
~50 sWith two gateway endpoints, each branch gets two tunnels, each with its own IKE SA and Child SA pair. Policy-based tunnels fit badly: both would carry identical selectors, an ordered SPD uses the first matching entry, and failover depends on implementation-specific peer lists. Route-based tunnels with any-to-any selectors and a BGP session on each let routing choose: withdraw on failure, converge to the other. The real decisions follow: **active/standby** (prefer one tunnel by BGP policy, simple, symmetric) or **active/active** (more capacity, but return traffic can arrive on the other tunnel and break a stateful filter); **detection** speed from the IKE liveness check, the BGP hold timer, or BFD where both ends support it; **prefix filtering** on each session, since routing is now the policy; and the branch router and uplink that a second tunnel does not protect. At 40 branches that is 80 tunnels, 160 ESP SAs and 80 BGP sessions.
go deeper
Recall that cloud VPN gateways usually expect two tunnels per site so one endpoint can fail or be serviced while the other carries traffic.
Explain why route-based tunnels with a routing protocol on each make failover a route withdrawal, and why identical selectors on two policy-based tunnels do not.
Show the operating details: failure detection by liveness check, BGP hold timer or BFD, asymmetric return paths with active/active, MTU after failover, and the counts the hub carries.
Own the trade: capacity versus symmetry, detection speed versus session load, and branch-side redundancy versus cost, then set an estate-wide standard for prefix filtering and failover tests.
## The constraints - **40 branches**, each needing reach to a set of cloud prefixes. - A **cloud VPN gateway with two endpoints** (say `203.0.113.10` and `203.0.113.20`) and a request for one tunnel to each. Two tunnels per connection is a common provider convention, not an IETF rule; the point is that either endpoint can be lost — to failure or maintenance — while the other carries traffic. - Each tunnel is its own **IKE SA** plus at least one **Child SA pair**. RFC 7296 requires Child SAs that can fail independently, without their IKE SA being able to send a delete, to be negotiated under separate IKE SAs; two tunnels to two endpoints already have separate IKE SAs. ## Why policy-based tunnels fit badly Two policy-based tunnels to two endpoints would carry the **same selectors**. RFC 4301's SPD is an **ordered** list, so the first matching entry is the one used; the second tunnel sits idle until something removes or reorders the first. That something is an implementation-specific peer-failover feature, not routing, and nothing in it tells the cloud which tunnel the branch is using. Use policy-based only if a branch device offers nothing else, and accept slow, opaque failover. ## The route-based shape 1. Two tunnel interfaces per branch, each with an any-to-any Child SA. 2. A **BGP** session over each tunnel; the branch advertises its own prefixes, the cloud its prefixes. 3. On failure, the dead tunnel's BGP session drops, its routes are withdrawn, and traffic follows the surviving routes — no IPsec renegotiation on the surviving tunnel. At this size the hub carries **80 tunnels, 80 IKE SAs, 80 Child SA pairs (160 one-way ESP SAs) and 80 BGP sessions**. ## The decisions and what each costs | Decision | Option | Gain | Cost | |---|---|---|---| | Traffic split | Active/standby, one tunnel preferred by BGP policy | Symmetric paths, simple troubleshooting | Half the tunnel capacity idle | | Traffic split | Active/active, equal-cost multipath | Both tunnels carry load | Flows may return on the other tunnel; a stateful filter at the branch can drop them | | Detection | IKE liveness check | Built into IKEv2 | Interval and retries are implementation choices; often slow | | Detection | BGP hold timer | Ties failover to routing directly | RFC 4271 suggests a default HoldTime of 90 seconds; shorter values cost more keepalives | | Detection | BFD (RFC 5880) | Fast, independent of IKE and BGP timers | Both ends must support it; another session per tunnel | | Routing policy | Prefix filters both ways | A branch can only advertise its own prefixes | Filters to maintain per branch | How one tunnel is preferred — path prepending, multi-exit discriminator or local preference — and how multipath is enabled are BGP mechanisms; the IPsec design only needs one of them chosen and applied consistently at both ends. ## What a second tunnel does not cover - **The branch router and its uplink.** Both tunnels usually leave one branch device over one internet line; losing either takes both down. A second device or a second uplink, with one tunnel on each, closes that gap at a price. - **Routing as policy.** With any-to-any selectors the IPsec inbound selector check no longer limits which inner sources a branch can send, and BGP decides what enters each tunnel. Filter the prefixes each branch may advertise and accept, and filter inner sources on each tunnel interface. - **MTU.** Both tunnels need an interface MTU that allows for the tunnel-mode and ESP overhead; a mismatch on the standby tunnel can stay hidden until failover moves large flows onto it. ## A recommended starting point 1. Route-based, any-to-any, two tunnels per branch. 2. Active/standby unless the branch path is stateless or the stateful filter sees both tunnels, then active/active. 3. BFD where both ends support it; otherwise tune BGP timers deliberately rather than relying on the IKE liveness check. 4. Prefix filters on every session and inner-source filters on every tunnel interface. 5. Test failover per branch by taking one tunnel down and watching routes and sessions move. There is no single right answer: a small estate with static subnets can live with active/standby static routes; a large one with frequent prefix changes needs BGP and the discipline that comes with it.
- Why can active/active cloud IPsec tunnels break traffic through a stateful filter at the branch?With equal-cost paths, the cloud may return a flow's replies over the other tunnel. A stateful filter that tracks connections per interface, or that sits on only one tunnel's path, then sees replies for a connection it never saw open and drops them. Active/standby, or a filter that sees both tunnels as one zone, avoids it.
- Why not rely on the IKEv2 liveness check alone to fail over between two cloud tunnels?It only says the peer has stopped answering; it does not move traffic by itself, and its interval and retry count are implementation choices that often make detection slow. In a route-based design what moves traffic is route withdrawal, so BGP's hold timer or BFD drives failover, with the liveness check tidying up the dead IKE SA.
saying these in an interview costs you the question
- Two tunnels to two cloud endpoints remove every single point of failure.
- Two policy-based tunnels with identical selectors load-share automatically.
- Active/active tunnels always outperform active/standby with no side effects.
- BGP over the tunnel replaces the need for any prefix filtering.
- IKE liveness checks alone give subsecond failover between tunnels.