A Cloudflare Pages project builds a preview for every pull request of an unreleased site; how do you keep those previews private and unindexed?
answer
- previews are public by default
- a project-level access policy
- which hostnames it covers
- noindex is not access control
- separate preview variables
basics
~20 sEnable the Pages project's access policy, which puts Cloudflare Access in front of every *.<project>.pages.dev preview, and rely on the X-Robots-Tag: noindex header Pages adds there. Neither covers <project>.pages.dev or custom domains, so handle those separately.
solid answer
~40 sPreview deployments are **public by default**: anyone with a `<hash>.<project>.pages.dev` or branch-alias URL can open them. In the project's **Settings > General**, *Enable access policy* creates a Cloudflare Access application for `*.<project>.pages.dev`, so previews require login, by default limited to your Cloudflare account's members. It does **not** cover `<project>.pages.dev` or any custom domain, including one pointed at a preview branch; those need their own Access application. For search engines, Pages already sends `X-Robots-Tag: noindex` on preview deployments; `noindex` keeps pages out of results but stops nobody from reading them. Two more controls close the gaps: give the **Preview** environment its own variables and keys, because whatever a build inlines is readable by anyone who reaches the site, and limit which branches build previews with branch deployment controls.
code
plaintext · 5 lineshttps://:project.pages.dev/*
X-Robots-Tag: noindex
https://staging.example.com/*
X-Robots-Tag: noindexgo deeper
Recall that Pages previews are public URLs by default and that the project settings have an option to put them behind Cloudflare Access.
Explain which hostnames the access policy covers, that Pages adds noindex to previews, and why production and preview variables are kept apart.
Close the gaps: custom domains on preview branches, the production pages.dev subdomain, inlined build-time secrets, and verifying headers with curl.
Set the default for every project, previews protected and noindexed from day one with test keys in Preview, and decide who may view them.
## The starting point: previews are public In **Cloudflare Pages**, every push to a non-production branch creates a **preview deployment** at `<hash>.<project>.pages.dev`, plus a branch alias such as `redesign.<project>.pages.dev`. By default these URLs are **public**: anyone who has, guesses or finds the link can view the site. Hash URLs also stay reachable for old builds. For an unreleased marketing site, that means a leak of the launch page, the pricing and the copy, and a duplicate of the site that search engines could index. Privacy and indexing are different problems with different tools: | Concern | Tool | Stops a person reading it? | |---|---|---| | Who can open a preview | Cloudflare Access via the project's access policy | Yes | | Whether search engines list it | `X-Robots-Tag: noindex` | No | | What a preview build knows | Separate Preview environment variables | Limits what can leak | | Which branches produce previews | Branch deployment controls | Reduces exposure | ## Lock previews behind Access In the Pages project, go to **Settings > General** and select **Enable access policy**. Pages creates a **Cloudflare Access** application for `*.<project>.pages.dev`, which takes in every hash preview and every branch alias. Visitors must then authenticate, and by default only members of your Cloudflare account get in; you widen it to your organisation's identity provider or named people with Access policies (configuring those policies is covered in the lesson on Access). What it does **not** protect: - `<project>.pages.dev`, the production subdomain; - any **custom domain**, including a custom domain you attached to a preview branch such as `staging.example.com`. Protecting those takes extra Access applications for those hostnames. If you do protect `<project>.pages.dev`, you must also protect each custom domain; otherwise custom-domain visitors see an Access login that does not work. One ordering trap when a preview branch gets its own custom domain: Pages cannot add a custom domain that already has an Access policy enabled on it, and the domain verifies only if Cloudflare can complete the certificate check on `/.well-known/acme-challenge/*`. Attach the domain to the project first, then put Access in front of it. ## Keep previews out of search results Pages adds `X-Robots-Tag: noindex` to responses on preview deployments by default, so crawlers that honour the header leave them out. You can confirm it with: ```bash curl -sI https://redesign.marketing-site.pages.dev | grep -i x-robots-tag ``` The default applies to the preview hostnames under `pages.dev`. Two hostnames need your own header in the `_headers` file: - the production subdomain, when the site's canonical home is a custom domain; - a custom domain pointed at a preview branch, which is not a `pages.dev` preview hostname, so check it with `curl -I` rather than assuming. ```plaintext https://:project.pages.dev/* X-Robots-Tag: noindex https://staging.example.com/* X-Robots-Tag: noindex ``` Rules in `_headers` apply to static asset responses only; responses produced by Pages Functions set their own headers. ## Limit what previews know Pages keeps **separate environment variables for Production and Preview**, both at build time and at runtime. A static site generator often inlines values into the HTML and JavaScript it outputs, so: - never give the Preview environment production keys or tokens; - treat anything a build can read as public to whoever can open the deployment; - use `CF_PAGES_BRANCH` at build time if a page should render differently outside production, for example a visible "preview" banner. ## Limit which branches build - **Branch deployment controls** can restrict previews to *Custom branches*, for example include `release/*` and `feat/*` only, or turn them off with *None*. - Pull requests from **forked repositories** do not create previews, so an outside contributor's pull request never gets a public preview URL. - Old previews can be removed with `wrangler pages deployment delete <id> --project-name <name>`; the latest deployment of a branch cannot be deleted, and an aliased one needs `--force`. ## A checklist for this site 1. Enable the access policy before the first pull request is opened. 2. Add Access applications for any custom domain that serves a preview branch. 3. Confirm `x-robots-tag: noindex` on preview and staging hostnames with `curl -I`. 4. Split production and preview variables, with test keys in Preview. 5. Restrict preview branches and prune old deployments.
- Why is `X-Robots-Tag: noindex` not enough to keep an unreleased Pages site private?It is an instruction to well-behaved crawlers about search results, not a barrier. Anyone holding the preview URL, from a pull request, a chat message or browser history, can still load it. Privacy needs Cloudflare Access in front of the preview hostnames, which the Pages project's access policy sets up.
- The team attached `staging.example.com` to the `staging` branch. Is it covered by the project's preview access policy?No. The policy covers `*.<project>.pages.dev` only. A custom domain, even one serving a preview branch, needs its own Access application, and its own `X-Robots-Tag` rule in `_headers` if it must stay out of search results.
- Preview deployments are private to the account until you share them.
- A noindex header stops people from viewing a preview.
- The preview access policy also protects the custom domains.
- Preview builds can safely share production's API keys behind Access.
- Pull requests from forks build previews that need locking down too.