skip to content

Why does IPsec transport mode through a NAT leave inner TCP and UDP checksums wrong, and how does the receiver repair them?

level: seniorimportance: nice to knowfreq 10%

answer

  1. the pseudo-header includes addresses
  2. the checksum travels encrypted
  3. original addresses from IKE
  4. Traffic Selectors with one address
  5. tunnel mode avoids it

basics

~20 s

TCP and UDP checksums cover the IP addresses through the pseudo-header, and in transport mode they travel encrypted inside ESP, so the NAT cannot fix them. The receiver learns the original addresses through IKE and fixes the checksum after decryption.

solid answer

~40 s

In IPsec transport mode the original IP header stays outside and the TCP or UDP header is encrypted inside ESP. The TCP and UDP checksums include a **pseudo-header** with the source and destination addresses, so when a NAT rewrites the outer addresses the checksum inside no longer matches, and the NAT cannot correct it because it cannot read or modify the protected payload. RFC 3948 §3.1.2 makes the receiver repair it after decryption: incrementally adjust the checksum using the peer's original addresses, recompute it, or, for UDP, zero it. IKEv2 supplies the original addresses through the **Traffic Selectors**, which in transport-mode NAT traversal MUST hold exactly one IP address each; IKEv1 used `NAT-OA` payloads. Tunnel mode avoids the issue, because the inner header is never translated.

go deeper

for a junior

Recall that TCP and UDP checksums include the IP addresses, and that in transport mode those checksums are hidden inside encrypted ESP.

for a middle

Explain the pseudo-header dependency, why the NAT cannot fix an encrypted checksum, and why tunnel mode does not have the problem.

for a senior

Explain the receiver's repair options, where IKEv2 gets the original addresses, the address substitution before the SPD lookup, and the overlapping-selector problem with several clients behind one NAT.

for a principal

Decide when transport mode behind NAT is worth its constraints against moving remote clients to tunnel mode with assigned addresses, given the policy limits RFC 3948 places on mixed clients.

## The checksum depends on addresses the NAT changes TCP and UDP checksums are computed over a **pseudo-header** that includes the IP source and destination addresses, plus the transport header and data. Any device that rewrites an address must therefore adjust the checksum too, and an ordinary NAT does exactly that for unprotected traffic. In **IPsec transport mode** the packet looks like `IP header | ESP | TCP | data | trailer | ICV`. The original IP header is the outer header, and the TCP header is encrypted and integrity-protected inside ESP. With NAT traversal a UDP header sits between the IP header and ESP: 1. The client at `192.168.1.10` builds TCP with a checksum over `192.168.1.10` and the server's address. 2. ESP encrypts the TCP segment; a UDP header for port `4500` is inserted. 3. The NAT rewrites the outer source to `198.51.100.7` and fixes the **outer UDP** header, which it can see. 4. The server decrypts and finds a TCP checksum computed over `192.168.1.10`, but the IP header it now holds says `198.51.100.7`. The checksum fails. RFC 7296 §2.23 puts it plainly: the NAT cannot correct the checksums because they are cryptographically protected. RFC 3715 lists this as an intrinsic NAT incompatibility. ## What the receiver is told to do RFC 3948 §3.1.2, the **transport mode decapsulation NAT procedure**, requires one of these, by local policy: | Option | How it works | Limit | |---|---|---| | Incremental fix-up | Subtract the received source and destination addresses from the checksum and add the original ones learned through IKE | Needs the original addresses | | Full recompute | Recompute the TCP or UDP checksum from scratch | Costs a pass over the data | | Skip the check | Set a UDP checksum to zero; for TCP, tell the stack not to verify it, if the stack allows | Only for transport mode, and only when the packet is integrity-protected | Skipping is defensible because ESP's integrity check has already authenticated the segment with a key; RFC 3715 notes the checksum then guards only against internal processing errors. If the original and received address are the same for one direction, that half of the fix cancels out. RFC 3948 also lets the receiver repair other protocols inside the packet that the NAT broke, such as ones carrying embedded addresses, since the NAT cannot see into ESP to do it. ## Where the original addresses come from - **IKEv2 (RFC 7296):** from the **Traffic Selectors** of the exchange. In transport-mode NAT traversal, each TSi and TSr MUST carry exactly one IP address, which serves as the original address. The responder stores those addresses for the checksum fix-up, then substitutes the addresses it actually sees, for example `198.51.100.7` in place of `192.168.1.10`, before its SPD lookup, so its policy and SA entries use addresses as its own stack sees them. - **IKEv1 (RFC 3947, deprecated by RFC 9395):** from `NAT-OA` (NAT original address) payloads sent in Quick Mode. - The `NAT_DETECTION` hashes cannot supply them: a hash shows that addresses changed, not what they were. ## Why tunnel mode is not affected In tunnel mode the inner IP header travels encrypted along with the TCP segment. The NAT rewrites only the new outer header, the inner addresses and checksum still agree on arrival, and RFC 3948 says tunnel-mode TCP checksums MUST be verified. That is one reason RFC 3948's appendix lists, as an option, an initiator asking for a **tunnel-mode** SA instead once it detects a NAT. ## The neighbouring problem: several clients behind one NAT Transport mode has a second NAT issue, in RFC 3948 §5.2. Two laptops behind one NAT appear to the server with the same address, separated only by their UDP encapsulation ports. If their traffic descriptions overlap, a simple selector lookup cannot choose the right SA, and implementations MUST handle it, for example by refusing conflicting connections. The server also cannot require security from one client behind that NAT while accepting clear text from another: a misbehaving client's clear text is indistinguishable, so for security guarantees that mix MUST NOT be allowed. - Transport mode behind a NAT therefore suits one client per NAT and narrow, specific selectors. - Tunnel mode with an address assigned through the tunnel is the usual answer for many remote users.

  • Why does the IKEv2 responder replace the client's private address in the Traffic Selectors before looking up its SPD?
    Its SPD and SAD work on the addresses its own stack sees. The client's `192.168.1.10` means nothing at the server, so RFC 7296 has it store the original address for checksum fix-up and then substitute the translated `198.51.100.7` before the lookup and the narrowing that follows.
  • Two laptops behind one NAT open transport-mode SAs to the same server for overlapping TCP traffic; why is that a problem?
    To the server both appear as the NAT's address, so their SAs differ only by UDP encapsulation port. A selector lookup on address and protocol cannot pick between them. RFC 3948 §5.2 says implementations MUST handle it, for example by disallowing conflicting connections.

saying these in an interview costs you the question

  • The NAT can recompute the TCP checksum inside a transport-mode ESP packet.
  • TCP checksums do not cover IP addresses, so NAT cannot affect them.
  • IKEv2 recovers the original addresses from the NAT_DETECTION hashes.
  • Tunnel mode has the same checksum problem as transport mode behind NAT.
  • A server can require IPsec from some clients behind a NAT and allow clear text from others.