skip to content

An agency switches a client's DNSSEC-signed zone to Cloudflare's nameservers, and validating resolvers now return SERVFAIL; why, and how should the move have run?

level: seniorimportance: should knowfreq 30%

answer

  1. the parent still vouches for old keys
  2. some resolvers fine, validators fail
  3. wait out the DS TTL first
  4. or multi-signer: swap the ZSKs
  5. 1.5 times the old DS TTL

basics

~20 s

The registrar still publishes the old provider's DS record, which matches none of Cloudflare's keys, so validating resolvers reject every answer. Remove the DS and let its TTL expire before switching, or use Cloudflare's multi-signer migration to stay signed throughout.

solid answer

~50 s

The DS record at the parent still vouches for the old provider's key-signing key. Cloudflare either serves the zone unsigned or signs it with its own keys, and neither matches, so validating resolvers return SERVFAIL while non-validating ones resolve normally; the zone can also sit in **Pending Nameserver Update**. `dig DS example.com` shows the stale record. The repair is to make the parent agree with Cloudflare: remove the old DS, or replace it with Cloudflare's DS if DNSSEC is enabled there, and wait for cached copies to expire. Done properly, the move takes one of two paths: remove the DS, wait for its TTL to expire fully, switch nameservers, then enable DNSSEC on Cloudflare and add its DS; or, if the old provider accepts external `DNSKEY` records, run Cloudflare's multi-signer migration so the zone never goes unsigned.

code

bash · 6 lines
bash
# What the parent zone publishes for the domain (TTL and key tag)
dig DS example.com +noall +answer

# The keys the new and the old provider serve (flag 256 = ZSK, 257 = KSK)
dig example.com DNSKEY @<assigned-cloudflare-nameserver> +noall +answer
dig example.com DNSKEY @<previous-provider-nameserver> +noall +answer

go deeper

for a junior

Remember that the DS record lives at the registrar and must be dealt with before a signed zone changes nameservers.

for a middle

Explain why a DS for the old provider's key fails validation against Cloudflare's answers, and why non-validating resolvers hide the outage.

for a senior

Diagnose with dig DS and DNSKEY, repair by making the parent agree with Cloudflare, and run either the unsigned-window path or the multi-signer migration in the right order.

for a principal

Weigh a short unsigned window against an advanced key exchange for each client, and set who holds registrar access and the go or no-go call.

## Why the switch broke validation A signed zone is trusted through a chain. The parent zone publishes a **DS record** that names one of the zone's **key-signing keys**. A validating resolver fetches the zone's `DNSKEY` set, checks that a key matches the DS, and then checks every answer's `RRSIG` against those keys. The DS is published by the registrar, not the DNS provider, so changing nameservers does not touch it. After the switch the DS still names the old provider's key. Cloudflare's answers either carry no signatures, because DNSSEC is not enabled on the new zone, or carry signatures from Cloudflare's own keys, which no DS names. Either way a validating resolver cannot build the chain, marks the answers bogus and returns **SERVFAIL**. Resolvers that do not validate never look at the DS and resolve the zone normally, which is why the outage looks partial: one network reports it down, another does not. A stale DS can also keep the new zone in **Pending Nameserver Update**. ## Confirming it ```bash dig DS example.com +noall +answer dig DNSKEY example.com @<assigned-cloudflare-nameserver> +noall +answer ``` - If the DS is present and Cloudflare's DNSSEC is off, the parent vouches for a zone that is no longer signed. - `dig example.com +cd` through a validating resolver turns validation off for one query: a proper answer there and SERVFAIL without it marks a DNSSEC failure rather than a missing record. - If both are present, compare the DS key tag with the keys Cloudflare serves. Cloudflare's own DS always carries key tag **2371**, so a DS with any other tag came from the previous provider. ## Repairing the live outage 1. **Make the parent agree with Cloudflare.** Remove the stale DS at the registrar. If you already enabled DNSSEC on Cloudflare, you can instead replace the old DS with Cloudflare's. 2. **Wait for cached copies to expire.** Resolvers that cached the old DS keep failing until its TTL runs out, commonly 24 to 48 hours for most top-level domains. 3. **Re-enable DNSSEC on Cloudflare** afterwards if you removed it, and add Cloudflare's DS. ## Path 1: a short unsigned window This path works with any previous provider. 1. **Remove the DS** at the registrar. 2. **Wait for the DS TTL to expire fully** at the parent; confirm with `dig DS example.com`. 3. **Change the nameservers** to Cloudflare's assigned pair. 4. **Wait for the previous `NS` TTL to expire**, typically an hour or less. 5. **Enable DNSSEC** on Cloudflare and add its DS at the registrar. Changing the nameservers before step 2 finishes reproduces the SERVFAIL above, because cached DS records still point at the old keys. ## Path 2: multi-signer migration, never unsigned Cloudflare documents a zero-downtime path based on **multi-signer DNSSEC** (RFC 8901). It needs one thing from the old provider: it must let you add `DNSKEY` records at the apex and serve them in answers. 1. **Set up Cloudflare.** Add the zone, review or import the records, select **Enable DNSSEC**, and turn on **Multi-signer DNSSEC** (`dnssec_multi_signer: true` through the API). 2. **Cross-import the zone-signing keys.** Add the old provider's ZSK to the Cloudflare zone as a `DNSKEY` record with flag `256`, and add Cloudflare's ZSK to the old provider. Check that both providers now return both ZSKs. 3. **Add Cloudflare at the registrar**: its DS record and its nameservers, beside the old ones. 4. **Remove the old provider**: its DS, then its nameservers. 5. **Clean up.** After at least **one and a half times** the old DS TTL, delete the old provider's ZSK `DNSKEY` record from the Cloudflare zone. Because each provider serves both zone-signing keys, a resolver holding either provider's cached key set can verify answers signed by the other. ## Choosing between them | | Path 1 | Path 2 | | --- | --- | --- | | Needs from the old provider | Nothing | External `DNSKEY` records at the apex | | DNSSEC protection | Absent for about a DS TTL plus an `NS` TTL | Continuous | | Effort | Low, mostly waiting | Advanced: key exchange and API steps | | Main risk | Switching before the DS TTL expires | A key missing from one provider's set |

  • Why is it not enough, without multi-signer, to add Cloudflare's DS beside the old one and then switch nameservers?
    The DS records anchor each provider's key-signing key, but a resolver that cached the old provider's `DNSKEY` set cannot verify answers signed with Cloudflare's zone-signing key, which is not in that set. Multi-signer DNSSEC fixes this by putting both providers' ZSKs in both key sets before the switch, so either cached set verifies either provider's signatures.
  • In the multi-signer migration, why wait one and a half times the old DS TTL before deleting the old provider's ZSK from Cloudflare?
    Resolvers may still hold the old DS and key data, or answers signed by the old provider, until their TTLs run out. Keeping the old ZSK in Cloudflare's key set until well after those copies expire means no cached signature is left without a key to check it. The extra half TTL is a safety margin over the strict minimum.

  • Changing nameservers carries the DNSSEC setup over to the new provider.
  • If some users can resolve the domain, the SERVFAIL reports must be a client problem.
  • Removing the stale DS fixes every resolver the moment the registrar saves it.
  • Cloudflare can re-sign the zone with the old provider's private keys.
  • Multi-signer migration works with any previous DNS provider.
  • The old provider's ZSK can come out of Cloudflare as soon as the zone is Active.