skip to content

After you create a Route 53 private hosted zone named example.com and associate it with a VPC, instances in that VPC stop resolving public example.com names such as www.example.com. What is happening, and how do you fix it?

level: seniorimportance: should knowfreq 48%

answer

  1. split horizon, not an overlay
  2. most specific matching zone wins
  3. authoritative NXDOMAIN, no fallback
  4. the fix is naming, not more records
  5. only VPCs you associated are affected

basics

~20 s

The private hosted zone becomes authoritative for the whole example.com namespace inside that VPC. Route 53 Resolver answers from it and returns NXDOMAIN for names it does not contain — it never falls back to public DNS. Scope the private zone to a dedicated internal subdomain instead.

solid answer

~40 s

This is split-horizon DNS working exactly as designed, and it surprises everyone once. Route 53 Resolver matches a query against the most specific private hosted zone associated with the querying VPC; once `example.com` matches, that zone is authoritative for every name under it. If `www.example.com` isn't in the private zone, the instance gets NXDOMAIN — Resolver does **not** fall through to the public `example.com` zone. Two fixes exist. The tactical one is to copy the public names you still need into the private zone, which works but creates two sources of truth that drift. The right one is to stop overlapping: give the private zone a dedicated internal name such as `internal.example.com` or a separate corporate suffix, so only genuinely internal names are shadowed and everything else resolves publicly as usual.

code

bash · 5 lines
bash
aws route53 create-hosted-zone \
  --name internal.example.com \
  --caller-reference "internal-zone-$(date +%s)" \
  --hosted-zone-config Comment="internal names only",PrivateZone=true \
  --vpc VPCRegion=us-east-1,VPCId=vpc-0abc123def4567890

go deeper

for a junior

Know that a private hosted zone answers only inside VPCs you associate with it, and that the zone name determines which names it takes over.

for a middle

Explain the selection rule: Resolver picks the most specific matching private zone and then answers authoritatively, returning NXDOMAIN rather than consulting public DNS.

for a senior

Diagnose it as split-horizon shadowing, weigh replicating records against renaming the zone, and check enableDnsSupport, enableDnsHostnames and the DHCP option set before touching records.

for a principal

Set the naming convention that prevents this org-wide — reserve an internal suffix — and account for the fact that one shared private zone associated with dozens of VPCs makes a single record change a multi-environment event.

## What a private hosted zone actually is A private hosted zone is a Route 53 zone that answers only for VPCs you explicitly associate with it. It is not reachable from the internet and has no delegation from any parent zone — there are no NS records at a registrar pointing at it. The only way a query reaches it is through the Amazon-provided DNS resolver inside an associated VPC (the VPC's CIDR base address plus two, also reachable at `169.254.169.253`). Two VPC attributes must be true for this to work at all: `enableDnsSupport`, which turns on the Amazon resolver, and `enableDnsHostnames`. A private hosted zone associated with a VPC that has either attribute off simply never answers, and the symptom looks identical to a missing record. ## How Resolver chooses which zone answers When an instance asks for a name, Route 53 Resolver checks the private hosted zones associated with that VPC and picks the **most specific** matching zone name. A query for `db.app.example.com` prefers a private zone named `app.example.com` over one named `example.com`. Once a zone has been selected, that zone is authoritative for the name: - If the record exists, the instance gets it. - If the record does not exist, the instance gets an authoritative NXDOMAIN. The second bullet is the whole question. Resolver does not treat the private zone as an overlay on top of public DNS. There is no "not found here, try the internet" step. By creating a private zone at the apex `example.com` you told AWS that inside this VPC, you own the entire `example.com` namespace — and you evidently only put a handful of records in it. So `www.example.com`, `mail.example.com`, the SaaS webhook at `hooks.example.com`, and the SES verification TXT record all vanish for anything running in that VPC. Applications that were happily calling your own public API by name start failing with DNS errors, which is why this incident usually presents as "the deploy broke everything" rather than as a DNS change. ## Fixing it **Tactical: replicate.** Add the public names you need as records inside the private zone. This is legitimate and sometimes deliberate — split-horizon exists so internal callers can reach a service by its public name over a private path, hitting an internal ALB instead of going out through NAT and back in. The cost is that every public record you care about now exists twice, and nothing keeps the copies in sync. Every future change to the public zone is a change someone must remember to mirror. **Structural: don't overlap.** Give the private zone a name that has no public counterpart — `internal.example.com`, or a distinct suffix reserved for internal use. Now Resolver only shadows names under that suffix; everything else falls through to public DNS untouched because no private zone matched at all. This is the design to argue for in an interview, because it removes an entire class of surprise rather than managing it. Whichever you choose, the blast radius follows the *association*, not the account: only VPCs you associated see the private zone. A second VPC that skipped the association keeps resolving `example.com` publicly, which is why this kind of failure so often looks environment-specific. ## The associations you should know about A private hosted zone can be associated with several VPCs, including VPCs in other accounts and other regions. Cross-account association is a two-step handshake: the zone owner calls `CreateVPCAssociationAuthorization`, and the VPC owner then calls `AssociateVPCWithHostedZone`. At scale, a single shared private zone associated with many VPCs is a common pattern — and it means one bad record change is visible in every one of them simultaneously. ## Related failure modes worth recognising - **Nothing resolves at all, private or public.** Check `enableDnsSupport` / `enableDnsHostnames` on the VPC before blaming the zone. - **A custom DHCP option set.** If the VPC's DHCP options replace `domain-name-servers` with your own servers instead of `AmazonProvidedDNS`, instances never talk to the Route 53 Resolver, so private hosted zones are bypassed entirely. - **On-premises hosts can't see it.** The VPC resolver only answers queries that originate inside the VPC. Reaching a private hosted zone from another network needs a Route 53 Resolver inbound endpoint. - **Overlapping zones.** Two private zones, `example.com` and `app.example.com`, both associated with the same VPC: the more specific one wins for everything under `app`, including names that only exist in the broader zone.

  • When would you deliberately want a private hosted zone that shadows a public domain?
    When internal callers should reach a service by its public name but over a private path — resolving `api.example.com` to an internal ALB instead of leaving through NAT and re-entering from the internet. It saves data-transfer cost and keeps traffic off the public path. The price is duplicated records, so keep the overlapping set small and generated, not hand-edited.
  • Instances resolve nothing at all after you associate a private hosted zone. Where do you look first?
    At the VPC, not the zone. `enableDnsSupport` and `enableDnsHostnames` must both be true, and the VPC's DHCP option set must still point `domain-name-servers` at AmazonProvidedDNS. If a custom resolver was substituted there, instances never reach the Route 53 Resolver and no private hosted zone can answer.
  • How does a VPC in another AWS account get to use this private hosted zone?
    With an explicit two-sided handshake: the zone's owner calls `CreateVPCAssociationAuthorization` for that VPC, then the VPC's owner calls `AssociateVPCWithHostedZone`. Neither side can do it alone. Remember that every associated VPC sees the same records, so a shared zone concentrates blast radius.

saying these in an interview costs you the question

  • Assumes Route 53 falls back to public DNS on a miss
  • Thinks the private zone only adds names, never hides them
  • Believes on-prem hosts can query the VPC resolver directly
  • Names the private zone after the public domain by default
  • Blames the record instead of checking the VPC's DNS attributes

context

resolving links…