skip to content

In VXLAN, which headers wrap the original Ethernet frame, from the outside in, and what job does each one do?

level: juniorimportance: should knowfreq 28%

answer

  1. MAC-in-UDP-in-IP
  2. outer headers cross the underlay
  3. UDP destination port 4789
  4. 8-byte header names the segment
  5. inner frame minus its FCS

basics

~20 s

VXLAN wraps the original Ethernet frame, minus its FCS, in an 8-byte VXLAN header carrying the 24-bit VNI, then UDP to port 4789, an outer IP header between the two VTEPs, and an outer Ethernet header with a new FCS.

solid answer

~40 s

VXLAN (RFC 7348, an Informational RFC) is MAC-in-UDP. From the outside in: an **outer Ethernet header** gets the packet to the next hop on each underlay link; an **outer IP header** carries it from the source VTEP's address to the destination VTEP (or to a multicast group); an **outer UDP header** to destination port `4789` tells the receiver it is VXLAN; and the **8-byte VXLAN header** carries the 24-bit VNI that names the overlay segment. Then comes the **inner Ethernet frame** - the tenant's own MAC addresses, EtherType and payload - with its original FCS removed, and a new FCS closes the outer frame. Underlay switches and routers forward on the outer headers alone; the tenant's machine only ever sees its own frame, because the VTEP adds and removes everything else.

go deeper

for a junior

Recall the order from the outside in: outer Ethernet, outer IP, UDP to port 4789, the VXLAN header with the VNI, then the original frame. Say plainly that it carries layer 2 over layer 3.

for a middle

Give each layer's size and job, who reads it, and the two changes to the inner frame: its FCS is dropped and an inner VLAN tag is normally stripped.

for a senior

Explain what each underlay device actually inspects, why the destination port is the decapsulation trigger, and how the outer source port, derived from the inner frame, gives path diversity without the underlay parsing the overlay.

for a principal

Frame the format choice: MAC-in-UDP rides existing hardware and 5-tuple tooling, a fixed header is simple to process at line rate, and the price is no extensibility, which Geneve's options later addressed.

## What VXLAN is wrapping, and why **VXLAN** (Virtual eXtensible Local Area Network) is defined in **RFC 7348**, published in 2014 as an *Informational* RFC rather than a standards-track one. It carries a whole **Ethernet frame** across an **IP network**, so that machines in different racks, or different routed subnets, behave as if they shared one layer 2 segment. The device that adds and removes the extra headers is the **VTEP** (VXLAN Tunnel End Point) - a hypervisor's virtual switch, a physical switch or a server. The short name for the format is **MAC-in-UDP**: an Ethernet frame inside a UDP datagram inside an IP packet inside another Ethernet frame. ## The stack, outside in | Layer | Size (bytes) | Key fields | Job | |---|---|---|---| | Outer Ethernet header | 14 (18 with an optional outer 802.1Q tag) | destination and source MAC, EtherType `0x0800` or `0x86DD` | Delivers the packet across one underlay link to the next hop | | Outer IP header | 20 (IPv4, no options) or 40 (IPv6) | source = sending VTEP, destination = receiving VTEP or a multicast group, protocol 17 | Carries the packet across the routed underlay | | Outer UDP header | 8 | source port chosen by the VTEP, destination port `4789` | Marks the payload as VXLAN and gives the underlay a 5-tuple | | VXLAN header | 8 | flags with the **I** bit, reserved bits, 24-bit **VNI** | Names the overlay segment the inner frame belongs to | | Inner Ethernet frame | original length minus 4 | tenant destination and source MAC, EtherType, payload | The tenant's traffic, untouched apart from its FCS | | Outer FCS | 4 | new frame check sequence | Protects the whole outer frame on each link | RFC 7348 describes the outer destination MAC as "the address of the target VTEP or of an intermediate Layer 3 router" - it changes at every routed hop, exactly as for any other IP packet. ## What happens to the inner frame The tenant's frame is carried almost exactly as it was sent, with three details worth knowing: - **Its FCS is removed.** RFC 7348's frame-format figure says it outright: the original Ethernet frame's FCS is not included, and a new FCS is computed for the outer frame. - **An inner 802.1Q tag is normally stripped.** RFC 7348 section 6.1 says the encapsulating VTEP SHOULD strip a VLAN tag before tunnelling, and a decapsulating VTEP SHOULD discard tagged inner frames, unless configured otherwise. The VNI, not the tag, identifies the segment inside the tunnel. - **The inner MAC addresses are preserved**, which is what lets the receiving side deliver the frame to the right machine on the right segment. ## Who reads which layer 1. An **underlay switch** forwards on the outer destination MAC, like any other frame. 2. An **underlay router** forwards on the outer destination IP, decrements the outer TTL or hop limit and rewrites the outer MACs. It does not need the VNI or the inner frame to do its job. 3. The **receiving VTEP** recognises UDP port 4789, reads the VNI, checks that it serves that segment, strips the four outer layers and hands the inner frame on. 4. The **tenant machine** sees an ordinary Ethernet frame. RFC 7348 is explicit that the VM never sees the VNI or the encapsulation. ## Why UDP, and the alternatives RFC 7348 puts the VXLAN header inside UDP rather than directly inside IP so that existing switches and routers can treat it as an ordinary 5-tuple flow - hashing it across paths using a source port the VTEP derives from the inner frame, and filtering it with 5-tuple ACLs, as its security section notes. Two other overlays make the contrast: - **Geneve** (RFC 8926) also rides in UDP, to port 6081, but adds variable-length options after its fixed 8-byte header. - **NVGRE** (RFC 7637) uses GRE instead of UDP and carries a 24-bit Virtual Subnet ID plus an 8-bit FlowID in the GRE key, because GRE has no ports to hash on. ## Common confusions - VXLAN carries the **Ethernet frame**, not just the IP packet inside it - that is what makes it a layer 2 overlay. - The VXLAN header sits **inside** the UDP datagram, not in front of the outer IP header. - Adding the four outer headers gives the well-known **50 bytes over IPv4**; what that number demands of the underlay is a separate design question.

  • Does a transit router in the underlay need to look at the inner frame of a VXLAN packet to forward it?
    No. To a transit router a VXLAN packet is an ordinary UDP datagram between two IP addresses: it forwards on the outer destination IP, decrements the outer TTL and rewrites the outer MAC addresses. The VNI and the inner frame are read only by the receiving VTEP, which strips the outer layers and delivers the inner frame.
  • Why does VXLAN leave out the inner frame's FCS instead of carrying it through the tunnel?
    RFC 7348's frame format omits the original FCS and appends a new FCS that covers the entire outer frame, inner bytes included, on every underlay link. Carrying the old one would duplicate that check. When the receiving VTEP transmits the inner frame onto an Ethernet link, that transmission computes a fresh FCS for it.

Like posting a sealed letter inside a courier pouch: every courier routes by the pouch label (the outer headers), a slip inside names the department (the VNI), and only the receiving mailroom opens the pouch and hands over the original letter.

saying these in an interview costs you the question

  • VXLAN wraps the inner IP packet, not the whole Ethernet frame
  • The VXLAN header sits in front of the outer IP header
  • VXLAN runs directly over IP with its own protocol number
  • Underlay routers read the VNI to decide where to forward
  • The original frame crosses the tunnel byte for byte, FCS included