skip to content

How does BGP EVPN multihome a server to two VXLAN leaves through an Ethernet segment, and what do route types 1 and 4 do?

level: seniorimportance: should knowfreq 12%

answer

  1. one link bundle, two leaves
  2. a 10-octet Ethernet Segment Identifier
  3. who sends BUM to the server
  4. V mod N over sorted VTEP addresses
  5. aliasing and mass withdrawal

basics

~20 s

Both leaves give the server's link bundle one Ethernet Segment Identifier; type-4 Ethernet Segment routes let them discover each other and elect a designated forwarder for BUM, and type-1 Ethernet A-D routes provide aliasing and mass withdrawal.

solid answer

~50 s

The server bundles its links (for example with IEEE 802.1AX LACP) and both leaves assign that **Ethernet segment** the same non-zero 10-octet **ESI**. Each leaf sends a **type-4** `Ethernet Segment` route, imported only by leaves on that segment via the ES-Import route target; after a timer (3 seconds by default in RFC 7432) they sort the originators' IPs and elect a **designated forwarder** per VLAN with `V mod N`, and only the DF forwards broadcast, multicast and flooded unknown unicast arriving from the fabric to the server. **Type-1** `Ethernet A-D` routes do two jobs: per segment, one withdrawal moves every MAC on a failed link at once (**mass withdrawal**); per EVPN instance, they let remote VTEPs load-balance to MACs only one leaf learned (**aliasing**). VXLAN has no ESI label, so RFC 8365 prevents echoing BUM back to the server with **local bias**.

code

pseudocode · 16 lines
pseudocode
# RFC 7432 section 8.5 default DF election (service carving)
on local Ethernet segment esi discovered:
    advertise EthernetSegmentRoute(esi, originator = my_vtep_ip)
    wait df_timer                      # 3 s by default
    peers = sort_numerically(originators of ES routes for esi, plus my_vtep_ip)
    N = length(peers)                  # ordinals run 0 .. N-1
    for each VLAN V on esi:
        df = peers[V mod N]
        if df == my_vtep_ip:
            forward BUM from the fabric toward esi for V
        else:
            drop BUM from the fabric toward esi for V

# peers = [10.0.0.11, 10.0.0.12]
# VLAN 100: 100 mod 2 = 0 -> 10.0.0.11
# VLAN 101: 101 mod 2 = 1 -> 10.0.0.12

go deeper

for a junior

Recall the goal: one server connects to two leaves for redundancy, and both leaves identify that link bundle with the same Ethernet segment identifier.

for a middle

Explain why only one leaf may forward broadcast arriving from the fabric to the server, and that type-4 routes let the leaves find each other and elect that designated forwarder.

for a senior

Compute a DF with V mod N over sorted VTEP addresses, separate the two type-1 jobs, and explain local bias as VXLAN's split-horizon method.

for a principal

Weigh standards-based multihoming against proprietary two-switch pairs, and the default election's uneven carving against Highest Random Weight.

## Ethernet segments and ESIs A server, or a switch, that connects to two leaves for redundancy presents those links as one **link aggregation group** (the IEEE's 802.1AX standard defines LACP for this). EVPN calls that set of links an **Ethernet segment** and gives it an **Ethernet Segment Identifier (ESI)**: a **10-octet** value whose first octet is a type. ESI **0** means single-homed, and all-ones (**MAX-ESI**) is reserved. Every leaf on the segment must use the same non-zero ESI; RFC 7432 also defines ESI types that are derived automatically, for example from the LACP system MAC. Two redundancy modes exist. In **All-Active**, every leaf on the segment forwards unicast for it; in **Single-Active**, only one does for a given VLAN. The rest of this answer assumes All-Active, the usual data-centre case. ## Route type 4: discovery and DF election Without coordination, a broadcast flooded through the fabric would reach the server once from each leaf. EVPN elects one **designated forwarder (DF)** to send BUM traffic toward the segment. RFC 7432 section 8.5 defines the default election, called **service carving**: 1. Each leaf that discovers the ESI on a local port advertises a **type-4 Ethernet Segment route** carrying the ESI, its own IP address as **Originating Router's IP Address**, and an **ES-Import route target** derived from the ESI, so only leaves on that same segment import it. 2. It starts a timer, **3 seconds by default**, to collect the other leaves' type-4 routes. 3. It sorts the originating IPs in increasing numeric order and numbers them from **ordinal 0**. 4. For each VLAN `V` on the segment, the leaf with ordinal `V mod N` is the DF, where `N` is the number of leaves. 5. The DF unblocks BUM arriving from the fabric toward the segment for that VLAN; the non-DFs drop such BUM in that direction. If a leaf's link fails or the leaf is added or removed, it withdraws or adds its type-4 route and the election runs again. ## Route type 1: two jobs | Variant | Purpose | Effect | |---|---|---| | Ethernet A-D per ES | **fast convergence (mass withdrawal)** and the split-horizon flags | when a leaf loses its link it withdraws this route, and every remote VTEP drops it as next hop for all MACs on that segment at once | | Ethernet A-D per EVI | **aliasing** (All-Active) or **backup path** (Single-Active) | a remote VTEP can send to any leaf on the segment, even if only one leaf has advertised a given MAC | Aliasing matters because the server's hashing may send all of one MAC's traffic, including its ARP, up a single link, so only one leaf learns and advertises that MAC in a type-2 route. The type-2 route carries the segment's ESI; the per-EVI type-1 routes from both leaves tell remote VTEPs that either leaf can reach that ESI, so they spread traffic across both. RFC 7432 forbids using a per-EVI route for forwarding until the matching per-ES routes have arrived. ## Loop prevention: DF filtering and local bias Suppose the server sends a broadcast up its link to leaf A, which floods it into the fabric. Leaf B, also on the segment, must not deliver it back to the server. With MPLS, RFC 7432 marks such frames with an **ESI label**. VXLAN has no label stack for that, so **RFC 8365** uses **local bias**: each VTEP tracks which remote VTEPs share segments with it and drops BUM arriving from one of them on those shared segments, judging by the tunnel's outer source IP. In return the ingress VTEP must deliver locally received BUM to its own attached segments **regardless of DF state**. RFC 8365 says the two methods MUST NOT be mixed on one segment. ## When the modulo goes wrong RFC 8584 lists the default election's problems. If every VLAN on a two-leaf segment is even, `V mod 2` is always 0 and one leaf is DF for everything. When a leaf joins or leaves, `N` changes and most VLANs move to a different DF, even those whose DF was unaffected. RFC 8584 defines a **Highest Random Weight** election that spreads VLANs evenly and moves only the affected ones, plus an option to make a leaf's DF candidacy depend on its attachment-circuit state. ## A failure walk 1. Leaves `10.0.0.11` and `10.0.0.12` share a segment carrying VLANs 100 and 101; the first is DF for 100, the second for 101. 2. The link to `10.0.0.12` fails. It withdraws its per-ES type-1 routes; remote VTEPs immediately stop using it for every MAC on the segment. 3. It withdraws its type-4 route; `10.0.0.11` re-runs the election with `N = 1` and becomes DF for both VLANs. 4. Its type-2 routes for MACs on the segment are withdrawn afterwards; traffic had already moved in step 2.

  • Why does a VXLAN EVPN fabric use local bias instead of an ESI label for split horizon?
    The ESI label is an MPLS label, and a VXLAN packet has no label stack to carry it. RFC 8365 instead has the egress VTEP look at the outer source IP: BUM from a VTEP it shares a segment with is not sent out on that segment. The ingress VTEP compensates by delivering BUM to its own local segments regardless of DF state.
  • What goes wrong with the default DF election when every VLAN on a two-leaf segment is even?
    V mod 2 is 0 for every VLAN, so the leaf at ordinal 0 is DF for all of them and the other carries no BUM duty, defeating service carving. Changing N also reshuffles most VLANs. RFC 8584 addresses both with a Highest Random Weight election.
  • What do remote VTEPs do when one leaf loses its link to the segment?
    That leaf withdraws its Ethernet A-D per ES routes, and every remote VTEP removes it as a next hop for all MACs on the segment at once, which is mass withdrawal, instead of waiting for each type-2 withdrawal. Its type-4 withdrawal re-runs the DF election among the remaining leaves.

saying these in an interview costs you the question

  • An ESI of 0 marks a segment that is multihomed to every leaf.
  • Both leaves send broadcasts to the server and the server discards duplicates.
  • The designated forwarder is simply the leaf with the highest router ID.
  • Aliasing comes from type-4 Ethernet Segment routes.
  • Every VTEP in the fabric imports every Ethernet Segment route.