skip to content

A GRE tunnel over IPv4 carries the RFC 2890 Key field across 1,500-byte links; what tunnel MTU and TCP MSS follow, and why?

level: middleimportance: should knowfreq 25%

answer

  1. count every header the encapsulator adds
  2. 20 outer, 4 base, 4 per option
  3. MSS is inner MTU minus 40
  4. RFC 2784's appendix on the DF bit

basics

~20 s

The tunnel adds 28 bytes: a 20-byte outer IPv4 header, the 4-byte GRE base header and the 4-byte Key. The inner IP MTU is 1,500 - 28 = 1,472 bytes, so the TCP MSS is 1,472 - 40 = 1,432.

solid answer

~50 s

Count what the encapsulator adds: the delivery IPv4 header (20 bytes), the GRE base header (4 bytes, RFC 2784) and the `Key` field (4 bytes, RFC 2890) — 28 bytes. A 1,500-byte underlay therefore leaves 1,472 bytes for the inner packet, and a TCP segment over inner IPv4 can carry 1,472 − 20 − 20 = 1,432 bytes, the MSS to advertise or clamp. Each further option costs 4 more: the Sequence Number, or the Checksum Present bit, which brings the `Checksum` and `Reserved1` fields. Left at 1,500, the tunnel must fragment full-size packets or drop them when DF is set on the delivery header; RFC 2784's appendix records that GRE implementations of its day neither ran PMTUD nor set DF on the delivery header, so the far endpoint reassembled. With IPsec on top, ESP's variable overhead comes too, which is why operators often pick a round margin such as 1,400.

go deeper

for a junior

Recall that a tunnel adds an outer IP header plus the GRE header, so the inner packet must be smaller than the link's 1,500 bytes.

for a middle

Compute the overhead for each combination of GRE fields over IPv4 and IPv6, and derive both the inner MTU and the TCP MSS without notes.

for a senior

Diagnose a tunnel where small traffic works and large transfers stall, explain the fragmentation and DF behaviour RFC 2784's appendix describes, and set MTU and MSS for GRE over IPsec.

for a principal

Decide whether to raise the underlay MTU or lower every tunnel's, weighing the transports you control against the internet paths you do not, and set an estate-wide margin.

## What the encapsulator adds A GRE tunnel endpoint takes an inner packet and puts two headers in front of it: the **GRE header** and a **delivery header**. Every byte of both comes out of the underlay link's **MTU** (maximum transmission unit), the largest IP packet the link carries without fragmenting. With an IPv4 delivery header and no IPv4 options, the costs are: | GRE fields present | GRE header | Plus IPv4 delivery | Inner IP MTU on a 1,500-byte link | TCP MSS for an inner IPv4 packet | |---|---|---|---|---| | Base only (RFC 2784) | 4 | 24 | 1,476 | 1,436 | | Key (RFC 2890) | 8 | 28 | 1,472 | 1,432 | | Key + Sequence Number | 12 | 32 | 1,468 | 1,428 | | Checksum + Key + Sequence Number | 16 | 36 | 1,464 | 1,424 | The **Checksum Present** bit adds 4 bytes because it brings two 2-byte fields, `Checksum` and `Reserved1`. The **Key** and **Sequence Number** fields are 4 bytes each. ## Working the numbers for the Key case 1. Delivery IPv4 header: **20** bytes. 2. GRE base header — the `C` bit, reserved bits, `Ver` and `Protocol Type`: **4** bytes. 3. The Key field: **4** bytes. Total added: 20 + 4 + 4 = **28**. 4. Inner IP MTU: 1,500 − 28 = **1,472** bytes. This is the MTU the tunnel interface should carry. 5. **TCP MSS** (maximum segment size) is the TCP payload that fits, so subtract the inner IPv4 header (20) and the TCP header without options (20): 1,472 − 40 = **1,432**. The MSS figure matters because TCP hosts on the two LANs advertise an MSS based on *their* 1,500-byte Ethernet, 1,460, and would send segments 28 bytes too big for the tunnel. Rewriting the MSS on SYNs crossing the tunnel, or lowering it on the hosts, keeps TCP inside the limit; how that clamping works in general is a separate subject. ## Why the tunnel MTU must come down If the tunnel interface is left at 1,500, a full-size inner packet becomes a 1,528-byte delivery packet on a 1,500-byte link, and one of two things happens: - **The delivery packet is fragmented.** RFC 2784's appendix records that deployed GRE implementations did not run Path MTU Discovery and **did not set the Don't Fragment bit** on the delivery header, so large packets were fragmented inside the tunnel and **reassembled at the tunnel exit**. That puts reassembly memory and CPU on the far router, and losing any fragment loses the whole packet. - **The packet is dropped.** If the entry point instead runs PMTUD on the tunnel and sets DF on the delivery header, an oversized delivery packet is dropped inside the tunnel, and the original sender learns of it only if the entry point relays the ICMP error back. RFC 2784's appendix notes that this relaying was not a GRE requirement; without it, the sender keeps retransmitting at the same size and the connection stalls after the handshake. Setting the tunnel MTU correctly moves the decision to the entry router, where it is visible and cheap. ## IPv6 as the delivery protocol RFC 7676 specifies GRE over IPv6. The IPv6 header is 40 bytes, so base GRE costs **44** bytes, leaving 1,456 for the inner packet on a 1,500-byte link: an inner IPv4 TCP MSS of 1,416, or 1,396 for inner IPv6 (1,456 − 40 − 20). RFC 7676 also requires that a GRE tunnel be able to carry a **1,280-byte IPv6 payload** from ingress to egress without fragmenting it, IPv6's minimum link MTU. ## Adding IPsec GRE over IPsec puts ESP's overhead on top of the 24 or more GRE bytes. ESP's cost is not a single number — it depends on the cipher, its initialisation vector and integrity tag, padding to the block size and the IPsec mode — so the exact figure belongs to the IPsec arithmetic. Operators commonly choose a round tunnel MTU of **1,400** with an MSS of **1,360** to leave margin for every combination; that is an operator convention, not a value any RFC sets. ## Checklist when you build one - Add up the outer header, the GRE base header and **every optional field you enabled**. - Set the tunnel interface MTU to the result, not to the default 1,500. - Bring the TCP MSS down by the same amount for traffic crossing the tunnel. - Test with large packets with DF set; small pings and TCP handshakes succeed even when the MTU is wrong. - Recompute when you add IPsec, IPv6 delivery or an underlay with a smaller MTU.

  • Why not leave the GRE tunnel at 1,500 and let routers fragment?
    Fragmenting the delivery packet moves reassembly to the far tunnel endpoint, costing it memory and CPU, and the loss of one fragment loses the whole packet. If DF is set on the delivery header instead, oversized packets are dropped inside the tunnel, and unless the entry point relays the ICMP error to the original sender, large transfers stall.
  • Does the GRE Sequence Number cost anything beyond 4 bytes?
    Yes. Under RFC 2890 the receiver treats a packet numbered at or below the last one it decapsulated as out of sequence and should silently discard it, so reordering in the underlay can become loss. RFC 2890 advises against using it for protocols that already handle ordering, such as TCP.
  • How does the arithmetic change with an IPv6 delivery header?
    The outer header is 40 bytes instead of 20, so base GRE costs 44 and leaves 1,456 bytes on a 1,500-byte link. RFC 7676 also requires the tunnel to carry a 1,280-byte IPv6 payload without fragmenting it.

saying these in an interview costs you the question

  • GRE adds only 4 bytes, so a tunnel MTU of 1,496 is enough.
  • The TCP MSS on a tunnel is its MTU minus 20 bytes.
  • Optional GRE fields such as the Key are free; only the base header counts.
  • ESP overhead is one fixed number, so GRE over IPsec has one exact correct MTU.
  • If pings and TCP handshakes work, the GRE tunnel MTU must be right.