Why does a DTLS 1.2 server answer the first ClientHello with HelloVerifyRequest instead of starting the handshake?

seniormust knowfreq 52%

answer

the address is a claim, not a fact
small in, large out
prove you can receive first
recompute rather than remember
a different message in 1.3

basics

~20 s

A datagram source address is unverified, so an unchecked server is both an amplifier and a state-exhaustion target. HelloVerifyRequest returns a cookie the client must echo, proving it can receive at the address it claimed, and the server stores nothing while it waits.

solid answer

~40 s

Over datagrams there is no handshake beneath DTLS that proves the source address is real. Answer a bare ClientHello and a server does two dangerous things at once: it sends a large flight — hello plus certificate chain — to whatever address the packet claimed, which makes it an amplifier pointed at a victim, and it allocates handshake state for a peer that may not exist. HelloVerifyRequest carries cookie<0..2^8-1> and closes both. The suggested construction is Cookie = HMAC(Secret, Client-IP, Client-Parameters), so the server recomputes and compares rather than remembering: it allocates nothing and can forget the exchange entirely. The client retransmits its ClientHello, otherwise identical, with the cookie attached, and only then does the server do work. The cookie proves return routability, not identity.

Over datagrams there is no handshake beneath DTLS that proves the source address is real. Answer a bare `ClientHello` and a server does two dangerous things at once: it sends a large flight — hello plus certificate chain — to whatever address the packet claimed, which makes it an amplifier pointed at a victim, and it allocates handshake state for a peer that may not exist. `HelloVerifyRequest` carries `cookie<0..2^8-1>` and closes both. The suggested construction is `Cookie = HMAC(Secret, Client-IP, Client-Parameters)`, so the server **recomputes and compares** rather than remembering: it allocates nothing and can forget the exchange entirely. The client retransmits its `ClientHello`, otherwise identical, with the cookie attached, and only then does the server do work. The cookie proves return routability, not identity.

code

pseudocode · 12 lines

pseudocode

a ClientHello arrives with no cookie:
    cookie = HMAC(server_secret, client_address, client_hello_parameters)
    send HelloVerifyRequest carrying cookie      -- at most 255 bytes
    allocate nothing, remember nothing, set no timer

a ClientHello arrives carrying a cookie:
    expected = HMAC(server_secret, client_address, client_hello_parameters)
    if cookie is not equal to expected:
        discard the datagram silently
    otherwise:
        allocate handshake state
        send the server flight

go deeper

for a junior

Know the shape: the server asks the client to echo back a cookie before it does any real work, because a datagram's source address has not been checked by anything.

for a middle

Explain the two attacks separately — amplification of a small hello into a large flight, and state exhaustion — and how a recomputed cookie closes both without the server storing anything.

for a senior

Show the design consequences: the exchange stays outside the transcript in DTLS 1.2, the server never retransmits its own challenge, and the cookie buys return routability only, so rate limiting and authentication remain separate jobs.

for a principal

Weigh the round trip against the exposure. On a long-latency fleet the extra trip is a real battery and latency cost per fresh handshake, which is what makes association lifetime and resumption policy a design decision rather than a tuning detail.

An address nobody checked

A datagram carries a source address that no one has verified. Anyone able to emit packets can put a victim's address in them, and the reply goes to the victim. That is a property of the transport, not a bug in DTLS — but DTLS is the thing that decides how much work to do before the address means anything.

Two distinct attacks follow, and a good answer names both:

Amplification. A ClientHello is small. The server's answering flight is large, because it carries a certificate chain. Spoof the source address, and the server becomes a device that turns a small packet into a much larger one aimed at somebody else.
State exhaustion. Every accepted hello costs the server memory, a timer and partially computed key material. A flood of spoofed hellos from addresses that will never answer exhausts that budget without the attacker ever receiving a byte.

The stateless answer

HelloVerifyRequest is a return-routability check. The server answers the first hello with a small message carrying an opaque cookie<0..2^8-1> — at most 255 bytes — and then forgets everything. The client retransmits its ClientHello, identical except that it now carries the cookie, and the server checks it.

The statelessness is the whole point, and it is achieved by construction rather than storage. RFC 6347's suggested form is:

Cookie = HMAC(Secret, Client-IP, Client-Parameters)

When the second hello arrives the server recomputes that value from its own secret, the address the packet came from and the client's own parameters, and compares. A match means the cookie was issued to this address, so somebody at that address received the earlier message and echoed it back. A mismatch means the datagram is discarded, silently and cheaply.

Two details follow from the design:

Neither the initial ClientHello nor the HelloVerifyRequest is included in the handshake transcript. The transcript starts from the second hello, because a stateless server cannot remember what it said in a message it kept no record of.
The server does not retransmit HelloVerifyRequest. It holds no timer for a client it has not committed to. If the message is lost, the client's own retransmission of the ClientHello produces the same cookie again.

What it proves, and what it does not

This is the most common wrong answer in the whole subject. The cookie proves exactly one thing: something at the claimed address received the server's message. It is not authentication, it says nothing about who the peer is, it is not a defence against an attacker who is on the path or who controls a real address, and it does not limit how many handshakes a legitimately reachable client can start. Rate limiting and authentication are separate jobs.

How strong is the requirement?

Stated at its real strength: a server SHOULD perform the cookie exchange for a new handshake, and MAY be configured not to where amplification is not a concern — for example where source addresses are already verified by the network. A client, on the other hand, must be prepared to do it on every handshake, since it cannot know the server's configuration in advance.

DTLS 1.3 does the same job with a different message

HelloVerifyRequest does not exist in DTLS 1.3. The role is taken over by HelloRetryRequest, the TLS 1.3 message, carrying the cookie in the cookie(44) extension. The properties are deliberately preserved:

	DTLS 1.2	DTLS 1.3
Message	`HelloVerifyRequest`	`HelloRetryRequest` with `cookie(44)`
Server state before the cookie returns	none	none
Retransmitted by the server?	no	no, deliberately
In the handshake transcript?	no	yes, through the message-hash mechanism TLS 1.3 defines
Client obligation	echo the cookie in a second hello	echo the cookie in a second hello

The row that trips people is the retransmission one: the server deliberately does not retransmit its retry, because doing so would require remembering that it sent one, which is exactly the state the design refuses to hold. Recovery is the client's timer, not the server's.

In the worked setting

For a buoy fleet the exchange is one extra round trip on a link whose round trip is already long, paid on every fresh handshake. That is a real cost on a duty-cycled radio, and it is the reason resumption and long-lived associations matter so much here — but the alternative is a public service that can be pointed at a third party and used to flood it, so the round trip is bought rather than avoided.

## An address nobody checked A datagram carries a source address that no one has verified. Anyone able to emit packets can put a victim's address in them, and the reply goes to the victim. That is a property of the transport, not a bug in DTLS — but DTLS is the thing that decides how much work to do before the address means anything. Two distinct attacks follow, and a good answer names both: - **Amplification.** A `ClientHello` is small. The server's answering flight is large, because it carries a certificate chain. Spoof the source address, and the server becomes a device that turns a small packet into a much larger one aimed at somebody else. - **State exhaustion.** Every accepted hello costs the server memory, a timer and partially computed key material. A flood of spoofed hellos from addresses that will never answer exhausts that budget without the attacker ever receiving a byte. ## The stateless answer `HelloVerifyRequest` is a return-routability check. The server answers the first hello with a small message carrying an opaque `cookie<0..2^8-1>` — at most 255 bytes — and then **forgets everything**. The client retransmits its `ClientHello`, identical except that it now carries the cookie, and the server checks it. The statelessness is the whole point, and it is achieved by construction rather than storage. RFC 6347's suggested form is: `Cookie = HMAC(Secret, Client-IP, Client-Parameters)` When the second hello arrives the server recomputes that value from its own secret, the address the packet came from and the client's own parameters, and compares. A match means the cookie was issued to **this** address, so somebody at that address received the earlier message and echoed it back. A mismatch means the datagram is discarded, silently and cheaply. Two details follow from the design: - **Neither the initial `ClientHello` nor the `HelloVerifyRequest` is included in the handshake transcript.** The transcript starts from the second hello, because a stateless server cannot remember what it said in a message it kept no record of. - **The server does not retransmit `HelloVerifyRequest`.** It holds no timer for a client it has not committed to. If the message is lost, the client's own retransmission of the `ClientHello` produces the same cookie again. ## What it proves, and what it does not This is the most common wrong answer in the whole subject. The cookie proves exactly one thing: **something at the claimed address received the server's message.** It is not authentication, it says nothing about who the peer is, it is not a defence against an attacker who is on the path or who controls a real address, and it does not limit how many handshakes a legitimately reachable client can start. Rate limiting and authentication are separate jobs. ## How strong is the requirement? Stated at its real strength: a server **SHOULD** perform the cookie exchange for a new handshake, and **MAY** be configured not to where amplification is not a concern — for example where source addresses are already verified by the network. A client, on the other hand, **must be prepared to do it on every handshake**, since it cannot know the server's configuration in advance. ## DTLS 1.3 does the same job with a different message `HelloVerifyRequest` does not exist in DTLS 1.3. The role is taken over by `HelloRetryRequest`, the TLS 1.3 message, carrying the cookie in the `cookie(44)` extension. The properties are deliberately preserved: | | DTLS 1.2 | DTLS 1.3 | |---|---|---| | Message | `HelloVerifyRequest` | `HelloRetryRequest` with `cookie(44)` | | Server state before the cookie returns | none | none | | Retransmitted by the server? | no | no, deliberately | | In the handshake transcript? | no | yes, through the message-hash mechanism TLS 1.3 defines | | Client obligation | echo the cookie in a second hello | echo the cookie in a second hello | The row that trips people is the retransmission one: **the server deliberately does not retransmit its retry**, because doing so would require remembering that it sent one, which is exactly the state the design refuses to hold. Recovery is the client's timer, not the server's. ## In the worked setting For a buoy fleet the exchange is one extra round trip on a link whose round trip is already long, paid on every fresh handshake. That is a real cost on a duty-cycled radio, and it is the reason resumption and long-lived associations matter so much here — but the alternative is a public service that can be pointed at a third party and used to flood it, so the round trip is bought rather than avoided.

What does the cookie prove, and what does it not?
It proves return routability: something at the address in the packet received the server's message and echoed the value back. It proves nothing about identity, nothing about intent, and nothing against an attacker who is on the path or who controls a genuine address. Authentication happens later in the handshake, and abuse from reachable clients still needs rate limiting.
Why does the server never retransmit HelloVerifyRequest or the DTLS 1.3 retry that replaced it?
Because retransmitting requires remembering that you sent something and running a timer for it, which is precisely the state the exchange exists to avoid. Recovery is pushed onto the client: its own retransmission of the ClientHello arrives, the server recomputes the same cookie from its secret and the client's parameters, and sends the same answer again.
Is the cookie exchange mandatory?
For the server it is a SHOULD, with an explicit MAY to skip it where amplification is not a concern, such as a network in which source addresses are already verified. For the client it is effectively unavoidable: it must be prepared to perform the exchange on any handshake, because it cannot know how the server is configured.

A venue that takes a booking by callback: it does not hold a table when you phone, it gives you a reference and rings the number you gave. Only an answer at that number turns the request into a reservation.

saying these in an interview costs you the question

Says the cookie authenticates the client
Claims the server stores issued cookies in a table
Thinks HelloVerifyRequest still exists in DTLS 1.3
Believes the cookie stops floods from genuinely reachable addresses
Includes the first ClientHello and the verify request in the transcript
Says the server retransmits its verify message on a timer

Why does a DTLS 1.2 server answer the first ClientHello with HelloVerifyRequest instead of starting the handshake?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

An address nobody checked

The stateless answer

What it proves, and what it does not

How strong is the requirement?

DTLS 1.3 does the same job with a different message

In the worked setting

▾common follow-ups3 questions

▾analogy~20 s

▾red flags6 flags

An address nobody checked

The stateless answer

What it proves, and what it does not

How strong is the requirement?

DTLS 1.3 does the same job with a different message

In the worked setting

answer

go deeper