In RDP, how does Standard RDP Security differ from TLS and CredSSP, and why is it the layer to retire?
answer
- three layers, one is native
- RC4 keys from two randoms
- who authenticates the server?
- external protocol does the work
- legacy kept for old clients
basics
~20 sStandard RDP Security is RDP's own legacy layer: RC4 session keys and no server authentication in the sequence itself. TLS and CredSSP hand protection and server authentication to an external protocol, so the legacy layer is the one to retire.
solid answer
~40 sRDP can secure a session three ways. Standard RDP Security (`PROTOCOL_RDP`, 0x00000000) is RDP's native layer: RC4 session keys (40, 56 or 128 bit) built from two 32-byte random values, or FIPS-validated methods at the FIPS level. TLS (`PROTOCOL_SSL`) and CredSSP (`PROTOCOL_HYBRID`, which runs a TLS handshake first and then authenticates the user inside it) are Enhanced RDP Security: an external protocol does the encryption, the integrity checks and the server authentication. The specification states that the base connection sequence has no mechanism that authenticates the server, and its own glossary notes RC4's weaknesses. The legacy layer survives for compatibility with older clients; the control is to require TLS or CredSSP on the server and plan the retirement of clients that cannot offer them.
go deeper
Name the three layers, say that Standard RDP Security is RDP's own RC4-based layer, and say TLS and CredSSP rely on an external protocol.
Explain where the session keys come from in each layer, that the server's method and level are zero under Enhanced RDP Security, and that the flags are PROTOCOL_RDP, PROTOCOL_SSL and PROTOCOL_HYBRID.
Argue from the specification's own statement that the legacy sequence cannot authenticate the server, and say which server setting turns that into a refusal.
Frame legacy-layer retirement as a compatibility debt with an owner and a date, not as a one-off hardening step.
## The layers RDP can negotiate When an RDP client connects, the first exchange (the X.224 Connection Request and Connection Confirm) settles which **security layer** protects everything after it. The Remote Desktop Protocol specification, MS-RDPBCGR, splits the layers into two families. | Layer | Flag in the negotiation | Family | What protects the traffic | |---|---|---|---| | Standard RDP Security | `PROTOCOL_RDP` (0x00000000) | Native, section 5.3 | RDP's own RC4 encryption, or FIPS-validated methods | | TLS | `PROTOCOL_SSL` (0x00000001) | Enhanced RDP Security, section 5.4 | TLS, an external protocol | | CredSSP | `PROTOCOL_HYBRID` (0x00000002), and `PROTOCOL_HYBRID_EX` (0x00000008) | Enhanced RDP Security | TLS first, then Kerberos or NTLM inside it | The specification lists further external protocols, among them RDSTLS, which are outside this answer. The flag table names TLS 1.0, 1.1 and 1.2 for `PROTOCOL_SSL`, while the later text of section 5.4.5.1 says TLS 1.0 to 1.3. ## What Standard RDP Security does Standard RDP Security is implemented inside RDP itself. - **Cipher.** Session keys are used with RC4. The key sizes the specification names are 40, 56 and 128 bit (`ENCRYPTION_METHOD_40BIT`, `_56BIT`, `_128BIT`), plus `ENCRYPTION_METHOD_FIPS` for FIPS-validated methods. - **Keys.** The server sends its public key and a 32-byte random value in the Server Security Data; the client returns its own 32-byte random, encrypted with that public key, in the Security Exchange PDU. The two random values produce the session keys. - **Level.** The server is configured with an encryption level (`ENCRYPTION_LEVEL_NONE`, `LOW`, `CLIENT_COMPATIBLE`, `HIGH`, `FIPS`, values 0 to 4) and tells the client the method and level it selected. - **Server identity.** Section 1.3.1.2 states that the RDP connection sequence does not provide any mechanism that ensures the identity of the server is authenticated, and that it is therefore vulnerable to man-in-the-middle attacks. ## What the Enhanced layers change With Enhanced RDP Security, section 5.4 says all security operations (encryption and decryption, integrity checks, server authentication) are implemented by the external protocol. The server's security data then carries `ENCRYPTION_METHOD_NONE` and `ENCRYPTION_LEVEL_NONE` (0), and the Security Exchange PDU is not sent, because RDP no longer derives its own session keys. Section 5.4.5.1 says TLS was added to RDP to enable authentication of the remote computer's identity, mitigating man-in-the-middle attacks. CredSSP always runs the TLS handshake first; Kerberos or NTLM then authenticates the user inside the TLS channel. How that credential exchange works is a separate topic. ## Why the legacy layer is the one to retire 1. **No server authentication from the sequence itself.** A client cannot tell the server it meant to reach from something else relaying the connection. 2. **Aged cryptography.** The specification's glossary says RC4 weaknesses make it vulnerable to various cryptographic attacks, and the specification still defines the 40- and 56-bit key sizes and the Low level (which, per the product documentation, protects only client-to-server data). 3. **It is the compatibility layer.** The negotiation-based approach exists, in the specification's words, to provide backward compatibility with previous RDP implementations. Allowing it keeps old clients working and keeps the weakest option available. 4. **Microsoft's own policy page** describes native RDP encryption, as opposed to SSL encryption, as not recommended. ## How to say it in an interview 1. Name the three layers and their flags. 2. Say who does the cryptography: RDP itself for the legacy layer, an external protocol for the other two. 3. Say what the legacy layer cannot do: authenticate the server. 4. Say what you do about it: require TLS or CredSSP on the server, and treat clients that cannot offer them as a retirement project. ## A worked reading A hospital audit lists a kiosk fleet that "still negotiates Standard RDP Security". Read it in three steps. 1. The finding names the layer, so the cipher is RDP's own RC4 (or FIPS-validated methods at the FIPS level), and the server's identity is not authenticated by the sequence. 2. The level in force is a server setting, so the question to ask is whether the server allows the legacy layer at all, not which level it uses. 3. The fix is a server requirement for TLS or CredSSP; the kiosk clients that cannot meet it become the migration list. Two confusions to avoid: - **Layer against level.** The layers are three (native, TLS, CredSSP); the levels (Low, Client Compatible, High, FIPS) apply inside the native layer only. - **Encrypted against authenticated.** A legacy session is encrypted; what it lacks is a way for the client to check who the server is. ## What it leaves open Requiring an Enhanced layer does not by itself decide which TLS versions are acceptable, whether the certificate is trusted, or how the user is authenticated; those are separate settings and separate topics. It also does not fix a client that cannot offer anything but `PROTOCOL_RDP`; that client is refused, which is the intended outcome and also the migration cost.
- If the legacy layer is the weak one, why does the specification keep it at all?Backward compatibility. The negotiation-based approach lets a client list the protocols it supports and a server pick one, so older RDP implementations that know only Standard RDP Security still connect. If the server's response carries no negotiation data at all, the client assumes the server does not support Enhanced RDP Security and that `PROTOCOL_RDP` is in force. The compatibility is the cost: a server that still allows it keeps the weakest option available.
- Does choosing TLS remove the need to set an RDP encryption level?Yes for the level as a protection. With Enhanced RDP Security the server's security data carries encryption method and level 0 (`NONE`), and Microsoft's policy page says the client connection encryption level setting applies only to native RDP encryption and does not apply to SSL encryption. The level still matters on any listener that allows the legacy layer, which is why leaving it high is not evidence that TLS is in use.
- Says RDP traffic is always TLS-protected by default everywhere.
- Treats the encryption level setting as controlling TLS sessions.
- Believes FIPS level makes Standard RDP Security equal to TLS.
- Claims CredSSP replaces TLS rather than running a TLS handshake first.
- Says 128-bit RC4 makes the legacy layer safe to leave on.
- Confuses the three layers with the three encryption levels.