skip to content

A TLS-based VPN forced onto TCP 443 crawls over a lossy hotel link, while the same tunnel over UDP is fine — why, and what do you change?

level: seniorimportance: must knowfreq 35%

answer

  1. two layers of reliability
  2. loss becomes delay
  3. stacked retransmission timers
  4. UDP first, TCP as fallback

basics

~20 s

Over TCP the tunnel retransmits every loss itself, so inner TCP connections see long, bursty delays, fire their own retransmission timeouts and cut their rates: TCP-in-TCP meltdown. Carry the tunnel on UDP and keep TCP 443 only as a fallback.

solid answer

~50 s

On a lossy link the **outer** TCP connection carrying the tunnel hides each loss by retransmitting it, and because it delivers in order, everything behind the loss waits. The **inner** TCP connections see that as delay, not loss: their round-trip estimates swing, a long stall fires their retransmission timeout, they resend segments that were never lost and shrink their windows. RFC 9329 (IKE and ESP over TCP) describes the same effect in Section 9.1 and names the worst case **TCP meltdown**: inner retransmissions add load that overruns the outer connection, so the mechanism meant to avoid loss increases it. All flows also share one head-of-line queue, so a single loss stalls every application. Over UDP a loss is just a loss and each inner connection recovers it once, end to end. The fix: offer UDP first and fall back to TCP 443 only on networks that block UDP.

go deeper

for a junior

Recall that VPN tunnels prefer UDP and that carrying TCP inside TCP makes losses turn into long delays.

for a middle

Explain why the outer TCP hides losses as delay and why one in-order stream makes every flow in the tunnel wait.

for a senior

Walk the stacked-timer feedback loop, explain why timer tuning is out of the operator's reach, and design UDP-first with a measured TCP fallback.

for a principal

Weigh reachability against throughput: how many users the TCP 443 fallback rescues, what it costs them, and whether a UDP port that guest networks pass reduces the fallback population.

## The scenario A user in a hotel connects to the company's **TLS-based VPN**. The hotel network blocks most UDP, so the client falls back to **TCP port 443**, which most guest networks pass. The Wi-Fi drops a small share of packets. Web pages crawl, file copies stall and then burst, and a voice call breaks up — yet on the home network, over UDP, the same tunnel is fast. Nothing is wrong with the encryption; the problem is that two TCP stacks are now stacked on top of each other. ## Two retransmission timers, stacked TCP assumes it runs over an unreliable datagram layer and treats delay and loss as signals about the path. RFC 9329 Section 9.1 (written for IKE and ESP over TCP, the same layering) traces what happens when the layer beneath it is another TCP: 1. The Wi-Fi drops one outer segment. 2. The **outer** TCP — the tunnel's own connection — retransmits it after its own timer or duplicate acknowledgements. 3. Because the outer TCP delivers in order, every tunnelled packet behind the lost one waits. The loss is "interpreted only as delay by inner connections". 4. The **inner** TCP connections, from the applications in the tunnel, see a sudden stall followed by a burst. Their round-trip estimates are distorted and a long enough stall fires their **retransmission timeout**. 5. Each inner sender retransmits segments that were never lost and cuts its sending rate. If those retransmissions are not detected as spurious in time, the rate stays low. 6. The extra retransmissions add load to the outer connection. RFC 9329 calls the worst case **TCP meltdown**: losses below cause timeouts above, whose retransmissions cause more loss below — "the very mechanism intended to avoid loss (retransmission) interacts between the two layers to increase loss". The RFC cites a 2005 study for the effect and gives **no loss-rate threshold** at which it starts; it is a qualitative warning, not a number. ## Why every application suffers at once - **One queue for all flows.** The outer connection is a single byte stream, so a loss on any packet stalls web, mail, file and voice traffic alike. RFC 9329 notes the negative effects "will be shared among all flows going through the outer TCP connection". - **Real-time traffic gets retransmission it never wanted.** RFC 9329 Section 9.2: protocols that prefer loss to delay, such as voice, are hurt when the tunnel retransmits their packets. - **Burstiness.** Packets released together after a repair arrive as a burst, which inner congestion control misreads. ## UDP and TCP carriage compared | Property | Tunnel over UDP | Tunnel over TCP | |---|---|---| | Outer loss seen by inner TCP as | Loss, recovered once end to end | Delay, then spurious timeouts | | Ordering across flows | Independent packets | One in-order stream | | Voice and video | Late or lost packets dropped | Retransmitted, arriving late | | Passes restrictive guest networks | Often blocked | Usually, on TCP 443 | ## What the operator changes - **Prefer UDP, fall back to TCP.** Configure clients to try UDP first and use TCP 443 only where UDP cannot get through. RFC 9329 states the same principle for IPsec: implementations "MUST favor using direct ESP or UDP encapsulation over TCP encapsulation whenever possible". - **Offer UDP on a port guest networks tend to pass**, so fewer users fall back at all. - **Keep segments whole.** Size the tunnel MTU and clamp the inner maximum segment size so tunnelled packets do not fragment; RFC 9329 Section 9.4 asks the encapsulating TCP connection to negotiate its MSS. - **Do not count on timer tuning.** RFC 9329 suggests giving the upper TCP a much longer timeout than the lower one, but the inner TCP stacks belong to users' hosts and the servers they reach, outside the VPN's control. - **Measure the fallback rate.** If many sessions land on TCP, the meltdown is a design problem, not a hotel problem. ## How to explain it in one line TCP over UDP has one retransmission timer per connection; TCP over TCP has two that react to each other, and on a lossy link they amplify instead of cancel.

  • RFC 9329 suggests giving the upper TCP a much longer timeout than the lower one. Why does that rarely help a VPN operator?
    The upper TCP connections are the applications' own, running on users' laptops and on the servers they talk to. Their retransmission timers are set by those hosts' operating systems, not by the VPN. The operator controls only the outer connection, so the practical fix is to avoid the outer TCP — carry the tunnel on UDP — rather than to tune timers it cannot reach.
  • Why does a voice call over the TCP-carried tunnel break up even when file transfers seem to work?
    Voice prefers a lost packet to a late one. Over the outer TCP every loss is repaired by retransmission, and every packet behind it waits in the same in-order queue, so voice frames arrive late and in bursts — past their playout time. RFC 9329 Section 9.2 warns that real-time protocols are hurt when a TCP tunnel adds reliability they never asked for.

saying these in an interview costs you the question

  • TCP is more reliable, so a VPN over TCP is the safer default on a bad link.
  • The slowdown comes from TLS encryption overhead, not from the transport.
  • Over UDP the tunnel loses data, so inner applications see corrupted transfers.
  • An RFC defines the loss rate at which TCP-in-TCP collapses.
  • Raising the outer TCP window cures TCP-in-TCP meltdown.