skip to content

Multitenancy and VNIs

A 24-bit VNI gives about 16 million segments against 4094 VLANs, with L2 VNIs for bridging and L3 VNIs tied to VRFs for routing. Interviewers ask how tenants stay isolated in one fabric.

on this pageshow

questions

5

Why does VXLAN identify segments with a 24-bit VNI rather than VLAN IDs, and how many segments does that allow?

level: juniorimportance: must knowfreq 42%

answer

  1. too many tenants for one tag space
  2. count the identifier bits
  3. each identifier is its own broadcast domain
  4. the VTEP adds it, the host never sees it

basics

~20 s

VXLAN carries a 24-bit VXLAN Network Identifier in its own header, giving 2^24 = 16,777,216 segment IDs against 4,094 usable 12-bit VLAN IDs, so a shared data centre can give every tenant many isolated layer 2 segments.

solid answer

~40 s

An 802.1Q VLAN ID is 12 bits, and with 0 and 4095 reserved by the IEEE only 4,094 VLANs remain for the whole switched domain. RFC 7348 calls that limit inadequate once a provider hosts many tenants, each wanting several segments and each choosing VLAN IDs and MAC addresses independently. VXLAN instead puts a 24-bit **VXLAN Network Identifier (VNI)** in its own header, so one administrative domain can hold up to 2^24 = 16,777,216 segments ("up to 16 M" in the RFC). Each VNI is a separate layer 2 broadcast domain: a frame is delivered only to hosts on the same VNI, so overlapping MAC addresses in different segments never cross over. Hosts never see the VNI; the VTEP adds it on encapsulation and removes it on decapsulation.

go deeper

for a junior

Recall the two widths and their counts: 12-bit VLAN ID with 4,094 usable values, 24-bit VNI with 16,777,216. Say that each VNI is its own broadcast domain.

for a middle

Explain who stamps the VNI and where: the ingress VTEP writes it into the VXLAN header from the local VLAN or port, and the egress VTEP delivers only within that VNI.

for a senior

Point out that the identifier space is rarely the real limit: per-switch VNI, MAC and ARP table sizes bind first, and a routed tenant consumes extra VNIs.

for a principal

Frame the VNI as one layer of tenancy: bridging isolation by VNI, routing isolation by per-tenant VRF, and the allocation plan that keeps both consistent across a fabric.

## The problem VLAN IDs could not solve A **VLAN** splits one switched Ethernet network into separate **broadcast domains**. Each frame on a trunk carries an IEEE 802.1Q tag whose **VLAN ID** field is 12 bits wide. Twelve bits give 4,096 values; the IEEE reserves 0 and 4095, which leaves **4,094 usable VLAN IDs** (the 802.1Q values are the IEEE's, not an RFC's). RFC 7348, the Informational RFC that defines VXLAN, lists why that ceiling hurt multi-tenant data centres: - **Tenant count.** A provider serving many customers, each needing several segments, runs out of 4,094 IDs quickly; the RFC says the limit is "often inadequate". - **Independent numbering.** Tenants assign their own MAC addresses and VLAN IDs, so the same values collide on the shared physical network. - **Spanning tree.** Large layer 2 domains depend on spanning tree, which blocks redundant links, and the RFC notes several data centres limit how many VLANs they use because of it. ## What the VNI is VXLAN is a **layer 2 overlay on a layer 3 network**: an Ethernet frame is wrapped in an outer IP/UDP packet and carried across a routed fabric between two **VXLAN Tunnel End Points (VTEPs)**. The wrapper includes an 8-byte VXLAN header, and the identifier that matters for tenancy sits in it: the 24-bit **VXLAN Network Identifier (VNI)**, also called the VXLAN segment ID. Each value names one **VXLAN segment**, an independent layer 2 broadcast domain. ## The arithmetic | Identifier | Field width | Values | Usable for segments | |---|---|---|---| | 802.1Q VLAN ID | 12 bits | 4,096 | 4,094 (0 and 4095 reserved by the IEEE) | | VXLAN VNI | 24 bits | 16,777,216 | "up to 16 M" per RFC 7348 | 2^24 = 16,777,216, which is 4,096 times the 12-bit space. RFC 7348 phrases the result as up to 16 million segments coexisting within the same **administrative domain**. ## How the VNI isolates segments 1. A host sends an ordinary Ethernet frame; it knows nothing about VXLAN. 2. The ingress VTEP decides which segment the frame belongs to, usually from the local VLAN or port it arrived on, and writes that segment's VNI into the header. 3. The egress VTEP reads the VNI and delivers the inner frame only to local hosts attached to the same VNI. Because every MAC lookup is made inside one VNI, RFC 7348 notes you "could have overlapping MAC addresses across segments but never have traffic cross over". Two tenants can therefore reuse the same MAC addresses, and the same VLAN numbers on their own ports, without colliding. ## What the VNI does not do on its own - **It does not make every tenant routable.** A VNI is a bridging domain. Routing between a tenant's subnets, and keeping overlapping IP prefixes apart, needs a per-tenant routing instance (a VRF), usually tied to its own VNI. - **It does not remove hardware limits.** MAC tables, ARP tables and the number of VNIs a single switch can hold are implementation limits, and they usually bind long before 16 million. - **It is not 16 million tenants.** A tenant normally uses several segments, plus one more VNI for its routing instance in a routed design. - **It does not replace VLANs at the edge.** Servers still send untagged or VLAN-tagged frames to the leaf switch; the VLAN simply becomes a local attachment detail that the VTEP maps to a VNI. ## Where it sits among overlays VXLAN is not the only overlay with a wider identifier: NVGRE (RFC 7637) carries a 24-bit Virtual Subnet Identifier in a GRE key, and Geneve (RFC 8926) carries a 24-bit VNI as well. The interview point is the same for all three: a 24-bit tenant segment identifier carried in an encapsulation header, outside the tenant's own frame, lifts the 4,094-segment ceiling that a 12-bit tag imposed.

  • Does a VXLAN fabric stop using VLANs altogether?
    No. Servers and appliances still send untagged or 802.1Q-tagged frames to the leaf. The VTEP maps the local VLAN, or the port, to a VNI and, as RFC 7348 section 6.1 recommends, strips the tag before encapsulating. The VLAN becomes a local attachment detail on one switch; the VNI is the segment's identity across the fabric.
  • Why are 16 million VNIs not the same as 16 million tenants?
    A tenant usually needs several bridged segments, and in a routed design one more VNI for its routing instance. In practice the binding limit is hardware: how many VNIs, MAC entries and ARP entries one switch can hold. The 24-bit space removes the identifier ceiling, not the resource ceiling.

saying these in an interview costs you the question

  • VXLAN widens the 802.1Q VLAN ID to 24 bits inside the same tag.
  • A 24-bit VNI means a fabric can host 16 million tenants without other limits.
  • Two tenants cannot reuse the same MAC addresses inside one VXLAN fabric.
  • Each host must be configured with the VNI of its segment.
  • VXLAN segments all share one broadcast domain separated only by IP subnets.
open as a page

Two tenants in one VXLAN fabric both use 10.0.0.0/24; how does the fabric keep their bridging and routing apart?

level: seniorimportance: must knowfreq 28%

basics

~20 s

Every lookup happens inside a tenant context: frames ride in the tenant's own L2 VNI, routed packets are looked up in the tenant's own VRF and carried in its L3 VNI, so tenant A's 10.0.0.5 and tenant B's are never compared.

open as a page

In a VXLAN fabric that routes for its tenants, what is the difference between an L2 VNI and an L3 VNI?

level: middleimportance: should knowfreq 24%

basics

~20 s

An L2 VNI names one bridged subnet, so the receiving VTEP looks up the destination MAC in that bridge table; an L3 VNI names a tenant's VRF, so the receiving VTEP looks up the destination IP in that tenant's routing table.

open as a page

On a VXLAN VTEP, how is a local VLAN mapped to a VNI, and why may two VTEPs use different VLAN IDs for one segment?

level: middleimportance: should knowfreq 30%

basics

~20 s

The VTEP holds a configured VLAN-to-VNI table; it strips the 802.1Q tag before encapsulating and maps the arriving VNI back to its own local VLAN, so with the usual domain-wide VNIs only the VNI must agree and VLAN IDs stay locally significant.

open as a page

Under RFC 9135, how do symmetric and asymmetric integrated routing and bridging differ in a VXLAN fabric, and what does each demand of every leaf?

level: seniorimportance: should knowfreq 15%

basics

~20 s

Asymmetric IRB routes only at the ingress VTEP and bridges into the destination subnet's L2 VNI, so every leaf carries every subnet; symmetric IRB routes at both VTEPs over the tenant's L3 VNI, so each leaf carries only its local subnets.

open as a page