skip to content

Two tenants in one VXLAN fabric both use 10.0.0.0/24; how does the fabric keep their bridging and routing apart?

level: seniorimportance: must knowfreq 28%

answer

  1. every lookup has a context
  2. segment identifier for frames
  3. per-tenant routing table
  4. distinguisher versus target
  5. what the underlay never learns

basics

~20 s

Every lookup happens inside a tenant context: frames ride in the tenant's own L2 VNI, routed packets are looked up in the tenant's own VRF and carried in its L3 VNI, so tenant A's 10.0.0.5 and tenant B's are never compared.

solid answer

~40 s

Each VTEP maps tenant A's VLAN to A's L2 VNI and tenant B's VLAN to B's, so bridged frames are scoped by VNI and duplicate MACs or IPs never meet. For routing, each subnet's gateway interface belongs to that tenant's own **VRF**, so the ingress VTEP looks up 10.0.0.5 only in the table of the VLAN it came from, and the egress VTEP picks the VRF from the L3 VNI. In the control plane, a **route distinguisher** makes the two 10.0.0.0/24 routes distinct BGP routes (RFC 4364), and **route targets** decide which VRF imports each. The underlay routes only between VTEP addresses and never learns any tenant prefix. Isolation ends where policy leaks routes between VRFs, and overlapping prefixes then need NAT or renumbering.

go deeper

for a junior

Recall that each tenant gets its own segment identifier and its own routing table, so reused addresses never meet.

for a middle

Trace a packet: ingress VLAN to VNI and VRF, lookup in that VRF only, L3 VNI across, egress VNI back to the same tenant's VRF.

for a senior

Name the failure points that break isolation: wrong VLAN-to-VNI mapping, wrong route-target import, and leaking overlapping prefixes into a shared VRF without NAT.

for a principal

Decide how shared services and internet exit are offered to tenants with overlapping space, and what that costs in NAT, address planning and per-leaf table capacity.

## The scenario Two tenants share one leaf-and-spine fabric and both chose 10.0.0.0/24, a private RFC 1918 range, for their web tier. RFC 7365's definition of **tenant separation** covers exactly this: traffic of one tenant is not visible or delivered to another except by policy, and "different tenants can use the same address space without conflict". | | Tenant A | Tenant B | |---|---|---| | VLAN on leaf 1 | 100 | 200 | | L2 VNI (10.0.0.0/24) | 10100 | 20100 | | VRF | vrf-A | vrf-B | | L3 VNI | 50100 | 50200 | Both tenants have a host at 10.0.0.5. The VTEPs' own addresses, say 172.16.0.1 and 172.16.0.2, live in the underlay. ## Bridging: the VNI scopes the MAC table - Leaf 1 maps VLAN 100 to VNI 10100 and VLAN 200 to VNI 20100. The VLAN numbers are only local; the VNI is the segment. - Every MAC lookup, and every flood of an unknown or broadcast frame, happens inside one VNI. RFC 7348 notes that overlapping MAC addresses across segments never cross over because the VNI isolates them. - So tenant A's ARP request for 10.0.0.5 is flooded only within VNI 10100 and can only be answered by tenant A's host. ## Routing: the VRF scopes the IP table 1. Tenant B's host at 10.0.0.9 sends a packet to a tenant B server in another subnet, addressing the frame to its gateway's MAC. 2. Leaf 1 receives it on VLAN 200, sees the gateway MAC on the gateway interface of that subnet, and routes. That interface belongs to **vrf-B**, so the lookup happens in vrf-B only. 3. vrf-B's entry points to leaf 2 and VNI 50200. Leaf 1 encapsulates with VNI 50200. 4. Leaf 2 maps VNI 50200 to vrf-B (RFC 9135 section 5.5: the VNI identifies the IP-VRF) and looks up the destination there. At no step does tenant A's table participate, so the identical 10.0.0.0/24 in vrf-A is irrelevant. With one shared routing table, the two prefixes would collide and one tenant's traffic would go to the other's hosts: the per-tenant VRF is what makes overlapping address space work. ## Control plane: distinguisher and target When VTEPs exchange tenant routes over BGP, two attributes keep them apart (the BGP/MPLS IP VPN mechanisms of RFC 4364, reused by EVPN): - **Route Distinguisher (RD)**: an 8-byte value prepended to the prefix. RFC 4364 says its purpose "is solely to allow one to create distinct routes to a common IPv4 address prefix", so both 10.0.0.0/24 routes survive as different routes instead of one replacing the other. - **Route Target (RT)**: an extended community that decides **which VRF imports** the route. vrf-A imports A's target, vrf-B imports B's. The RD carries no meaning about where a route goes; confusing it with the RT is a classic mistake. The control-plane route formats themselves are the EVPN discussion's subject. ## The underlay never sees tenant addresses The spine switches route only on the outer IP header, from one VTEP address to another. They never learn 10.0.0.0/24 from either tenant, so they need no per-tenant state and cannot mix tenants up. In principle the underlay could even use 10.0.0.0/24 itself, because it lives in a separate routing table, but distinct underlay addressing keeps troubleshooting sane. ## Where isolation ends - **A VLAN mapped into the wrong VNI** joins two tenants' broadcast domains on that leaf. - **A wrong route-target import** puts one tenant's routes into another's VRF. - **Deliberate sharing** (a shared services VRF, an internet exit) needs routes leaked between VRFs, and overlapping prefixes cannot both be leaked as-is: the shared side would see two 10.0.0.5. NAT on the way in, or unique addressing for shared services, resolves it. - **Hardware is shared.** VRF, MAC and ARP table capacity on each leaf is one pool that all tenants draw from. The short version for an interview: the VNI keeps frames apart, the VRF keeps routes apart, the RD keeps BGP routes apart, the RT decides who imports what, and the underlay never knows any of it.

  • Tenant A must reach a shared DNS server in a services VRF. What changes?
    The services VRF imports tenant A's prefix and tenant A's VRF imports the server's prefix, usually by adding route targets selectively. The server's address must not overlap either tenant, and if tenant B also connects, its 10.0.0.0/24 collides with A's on the services side, so one tenant's traffic is translated with NAT or the services reply cannot tell them apart.
  • Why can't the spine switches route tenant traffic wrongly between the two tenants?
    They never see tenant addresses. A spine routes only on the outer IP header between VTEP addresses, so tenant prefixes, VRFs and VNIs never enter its tables; any mix-up must happen at a VTEP, in a VLAN-to-VNI mapping or a route import.

saying these in an interview costs you the question

  • Overlapping tenant subnets always need NAT inside the fabric.
  • The VNI alone keeps routed traffic apart, so per-tenant VRFs are optional.
  • The spine routers must carry every tenant's prefixes to forward overlay traffic.
  • The route distinguisher decides which VRF imports a route.
  • Tenants must use different MAC addresses to share one fabric.