skip to content

A Cloudflare Pages project builds a preview for every pull request of an unreleased site; how do you keep those previews private and unindexed?

level: seniorimportance: should knowfreq 36%

answer

  1. previews are public by default
  2. a project-level access policy
  3. which hostnames it covers
  4. noindex is not access control
  5. separate preview variables

basics

~20 s

Enable the Pages project's access policy, which puts Cloudflare Access in front of every *.<project>.pages.dev preview, and rely on the X-Robots-Tag: noindex header Pages adds there. Neither covers <project>.pages.dev or custom domains, so handle those separately.

solid answer

~40 s

Preview deployments are **public by default**: anyone with a `<hash>.<project>.pages.dev` or branch-alias URL can open them. In the project's **Settings > General**, *Enable access policy* creates a Cloudflare Access application for `*.<project>.pages.dev`, so previews require login, by default limited to your Cloudflare account's members. It does **not** cover `<project>.pages.dev` or any custom domain, including one pointed at a preview branch; those need their own Access application. For search engines, Pages already sends `X-Robots-Tag: noindex` on preview deployments; `noindex` keeps pages out of results but stops nobody from reading them. Two more controls close the gaps: give the **Preview** environment its own variables and keys, because whatever a build inlines is readable by anyone who reaches the site, and limit which branches build previews with branch deployment controls.

code

plaintext · 5 lines
plaintext
https://:project.pages.dev/*
  X-Robots-Tag: noindex

https://staging.example.com/*
  X-Robots-Tag: noindex

go deeper

for a junior

Recall that Pages previews are public URLs by default and that the project settings have an option to put them behind Cloudflare Access.

for a middle

Explain which hostnames the access policy covers, that Pages adds noindex to previews, and why production and preview variables are kept apart.

for a senior

Close the gaps: custom domains on preview branches, the production pages.dev subdomain, inlined build-time secrets, and verifying headers with curl.

for a principal

Set the default for every project, previews protected and noindexed from day one with test keys in Preview, and decide who may view them.

## The starting point: previews are public In **Cloudflare Pages**, every push to a non-production branch creates a **preview deployment** at `<hash>.<project>.pages.dev`, plus a branch alias such as `redesign.<project>.pages.dev`. By default these URLs are **public**: anyone who has, guesses or finds the link can view the site. Hash URLs also stay reachable for old builds. For an unreleased marketing site, that means a leak of the launch page, the pricing and the copy, and a duplicate of the site that search engines could index. Privacy and indexing are different problems with different tools: | Concern | Tool | Stops a person reading it? | |---|---|---| | Who can open a preview | Cloudflare Access via the project's access policy | Yes | | Whether search engines list it | `X-Robots-Tag: noindex` | No | | What a preview build knows | Separate Preview environment variables | Limits what can leak | | Which branches produce previews | Branch deployment controls | Reduces exposure | ## Lock previews behind Access In the Pages project, go to **Settings > General** and select **Enable access policy**. Pages creates a **Cloudflare Access** application for `*.<project>.pages.dev`, which takes in every hash preview and every branch alias. Visitors must then authenticate, and by default only members of your Cloudflare account get in; you widen it to your organisation's identity provider or named people with Access policies (configuring those policies is covered in the lesson on Access). What it does **not** protect: - `<project>.pages.dev`, the production subdomain; - any **custom domain**, including a custom domain you attached to a preview branch such as `staging.example.com`. Protecting those takes extra Access applications for those hostnames. If you do protect `<project>.pages.dev`, you must also protect each custom domain; otherwise custom-domain visitors see an Access login that does not work. One ordering trap when a preview branch gets its own custom domain: Pages cannot add a custom domain that already has an Access policy enabled on it, and the domain verifies only if Cloudflare can complete the certificate check on `/.well-known/acme-challenge/*`. Attach the domain to the project first, then put Access in front of it. ## Keep previews out of search results Pages adds `X-Robots-Tag: noindex` to responses on preview deployments by default, so crawlers that honour the header leave them out. You can confirm it with: ```bash curl -sI https://redesign.marketing-site.pages.dev | grep -i x-robots-tag ``` The default applies to the preview hostnames under `pages.dev`. Two hostnames need your own header in the `_headers` file: - the production subdomain, when the site's canonical home is a custom domain; - a custom domain pointed at a preview branch, which is not a `pages.dev` preview hostname, so check it with `curl -I` rather than assuming. ```plaintext https://:project.pages.dev/* X-Robots-Tag: noindex https://staging.example.com/* X-Robots-Tag: noindex ``` Rules in `_headers` apply to static asset responses only; responses produced by Pages Functions set their own headers. ## Limit what previews know Pages keeps **separate environment variables for Production and Preview**, both at build time and at runtime. A static site generator often inlines values into the HTML and JavaScript it outputs, so: - never give the Preview environment production keys or tokens; - treat anything a build can read as public to whoever can open the deployment; - use `CF_PAGES_BRANCH` at build time if a page should render differently outside production, for example a visible "preview" banner. ## Limit which branches build - **Branch deployment controls** can restrict previews to *Custom branches*, for example include `release/*` and `feat/*` only, or turn them off with *None*. - Pull requests from **forked repositories** do not create previews, so an outside contributor's pull request never gets a public preview URL. - Old previews can be removed with `wrangler pages deployment delete <id> --project-name <name>`; the latest deployment of a branch cannot be deleted, and an aliased one needs `--force`. ## A checklist for this site 1. Enable the access policy before the first pull request is opened. 2. Add Access applications for any custom domain that serves a preview branch. 3. Confirm `x-robots-tag: noindex` on preview and staging hostnames with `curl -I`. 4. Split production and preview variables, with test keys in Preview. 5. Restrict preview branches and prune old deployments.

  • Why is `X-Robots-Tag: noindex` not enough to keep an unreleased Pages site private?
    It is an instruction to well-behaved crawlers about search results, not a barrier. Anyone holding the preview URL, from a pull request, a chat message or browser history, can still load it. Privacy needs Cloudflare Access in front of the preview hostnames, which the Pages project's access policy sets up.
  • The team attached `staging.example.com` to the `staging` branch. Is it covered by the project's preview access policy?
    No. The policy covers `*.<project>.pages.dev` only. A custom domain, even one serving a preview branch, needs its own Access application, and its own `X-Robots-Tag` rule in `_headers` if it must stay out of search results.

  • Preview deployments are private to the account until you share them.
  • A noindex header stops people from viewing a preview.
  • The preview access policy also protects the custom domains.
  • Preview builds can safely share production's API keys behind Access.
  • Pull requests from forks build previews that need locking down too.