For a remote-access estate, how do you choose between a TLS-based VPN and IPsec, weighing user-space processing, firewall traversal and interoperability?
answer
- kernel path or user-space path
- one port versus IKE plus ESP
- who else speaks the protocol
- standards track versus one implementation
basics
~20 sIPsec runs in the kernel and any standard gateway speaks it, but needs IKE's UDP ports and ESP to pass; a TLS-based VPN needs one UDP or TCP port, at the cost of user-space copies and one implementation at both ends.
solid answer
~50 sThe choice turns on three axes. **Interoperability**: IPsec is specified in RFCs (RFC 4301, RFC 4303 for ESP, RFC 7296 for IKEv2), so partners' firewalls and cloud gateways terminate it; the TLS-control-channel design has no RFC, so both ends run the same implementation. **Firewall traversal**: IPsec needs IKE on UDP 500 and 4500 and ESP as IP protocol 50 or UDP-encapsulated, with TCP encapsulation (RFC 9329, TCP 4500) as a last resort; a TLS-based VPN needs one port and can sit on TCP 443, paying TCP-in-TCP costs when it does. **Data-path cost**: a user-space tunnel copies every packet between kernel and user space; one benchmark in the WireGuard paper put a TLS-based VPN in UDP mode at 258 Mbit/s against 881 for IPsec on the same machines. A common answer: IPsec for site-to-site and high throughput, a TLS-based VPN for roaming laptops on hostile networks — UDP first.
go deeper
Recall that IPsec is a standard spoken by many vendors' gateways, while a TLS-based VPN rides one port and is easier to get through firewalls.
Explain what each needs from the network — IKE ports and ESP versus one UDP or TCP port — and why user-space processing costs throughput.
Diagnose where each fails in practice: blocked ESP or IKE on guest networks, TCP-in-TCP on the fallback, concentrator CPU limits from user-space copies.
Own the estate design: which tunnels must interoperate with third parties, where throughput dominates, and whether running two VPN systems is worth reaching every user.
## What is being compared Both families encrypt a tunnel between a client or branch and a gateway. They differ in how they are specified, where packets are processed and what they need from the network in between. | Aspect | IPsec | TLS-based VPN | |---|---|---| | Specification | RFCs: architecture RFC 4301, ESP RFC 4303, IKEv2 RFC 7296 | No RFC; defined by its implementations | | Key agreement | IKEv2 | A TLS session on a control channel | | Data path | Usually in the operating system kernel | Usually a user-space process | | What the network must pass | IKE on UDP 500/4500, plus ESP (IP protocol 50) or ESP in UDP 4500 | One UDP or TCP port, often TCP 443 as a fallback | | Interoperability | Independent implementations interoperate | Both ends run the same implementation | ## Firewall traversal - **IPsec** needs more from the path. IKE "normally listens and sends on UDP port 500" (RFC 7296), with UDP 4500 used once encapsulation is in play (RFC 3948); raw ESP is its own IP protocol, 50, which many guest networks do not pass. RFC 9329 adds IKE and ESP over TCP — implementations "MUST support TCP encapsulation on TCP port 4500" and may listen on alternate ports — but it states that implementations "MUST favor using direct ESP or UDP encapsulation over TCP encapsulation whenever possible", and every TCP stream begins with the `IKETCP` prefix, which the RFC itself notes an operator can filter on. - **A TLS-based VPN** needs one port. On TCP 443 it uses the port most firewalls leave open for web traffic, so it reaches users that IPsec cannot — though deep inspection can still tell its framing from ordinary HTTPS. The price is TCP-in-TCP: over a lossy link, stacked retransmission timers slow everything in the tunnel. Running UDP first and TCP 443 only as a fallback keeps that price to the users who need it. RFC 9329's introduction lists proprietary "SSL VPNs", which "often run on TCP port 443", among the earlier ways of carrying IPsec-style traffic over TCP. ## Data-path cost A kernel IPsec data path encrypts packets where they are routed. A user-space tunnel reads each packet from a virtual interface into a process, encrypts it and writes it back to a socket — copies between kernel and user space and a scheduler hop for every packet. The WireGuard paper (a whitepaper, not an RFC) measured one setup: the TLS-based VPN in UDP mode with AES-256 and HMAC-SHA2-256 reached **258 Mbit/s**, IPsec with AES-256-GCM **881 Mbit/s**, and WireGuard **1,011 Mbit/s**, with ping times of 1.541, 0.508 and 0.403 ms. It is one benchmark on one pair of machines, not a law — but the direction is the expected one, and the gap matters for a concentrator serving thousands of users or a branch moving bulk data. ## Security surface and operations - Both rely on certificates or keys and a PKI to run well; neither removes that work. - TLS brings a large, general-purpose library into the VPN path; IKE brings its own complex negotiation. Each needs patching discipline. - A TLS-based VPN can put a pre-shared HMAC in front of TLS so strangers never reach the TLS code. - IPsec's configuration space — proposals, policies, traffic selectors — is a frequent source of mismatches between vendors. ## A decision framework 1. **Must a third party's gateway terminate the tunnel?** A partner's firewall or a cloud provider's VPN gateway speaks IPsec; choose IPsec. 2. **Is throughput or concentrator scale the constraint?** Prefer a kernel data path. 3. **Do users connect from networks you do not control?** A TLS-based VPN, UDP first with TCP 443 as fallback, reaches the most of them. 4. **Can you run one client everywhere?** A TLS-based VPN needs its implementation on every endpoint; IPsec clients are often built into operating systems. Many estates answer "both": IPsec between sites and to cloud gateways, a TLS-based VPN for roaming laptops — and accept two systems to operate as the cost.
- Why is a TLS-based VPN usually the wrong choice for a tunnel to a partner's firewall?The TLS-control-channel design has no RFC, so only its own implementation speaks it; the partner's firewall almost certainly speaks IKEv2 and ESP, which independent vendors implement from RFC 7296 and RFC 4303. Asking a partner to install and operate your VPN software at their edge is rarely acceptable, so standards-based IPsec wins by default.
- IPsec can now run over TCP. Does that erase the TLS-based VPN's firewall advantage?Partly. RFC 9329 requires support on TCP 4500 and allows alternate ports, so IPsec can reach some networks it could not. But every stream starts with the `IKETCP` prefix, which the RFC notes operators can filter on, and the RFC makes TCP a last resort because it inherits the same TCP-in-TCP penalties. A TLS-based VPN on TCP 443 still uses the port guest networks are least likely to block.
saying these in an interview costs you the question
- A TLS-based VPN is an IETF standard, so any vendor's gateway can terminate it.
- IPsec can only run as raw ESP, so it never passes a strict firewall.
- User-space packet processing costs nothing measurable on modern hardware.
- Choosing a TLS-based VPN removes the need for certificates or a PKI.
- IPsec over TCP avoids the TCP-in-TCP problems of a TLS-based VPN on TCP.