With Cloudflare Load Balancing steering set to Off, how does traffic move between pools as endpoints fail, and when does the fallback pool take over?
answer
- pool order is failover order
- Health Threshold per pool
- Degraded versus Critical
- fallback health is ignored
- 530 with 1016
basics
~20 sOff steering sends traffic to the first healthy pool in list order. A pool below its Health Threshold goes Critical and the next pool takes over; when every pool is Critical, the fallback pool serves without any health check.
solid answer
~50 sWith **Off - Failover**, pool order is the failover order. Monitors mark endpoints healthy or unhealthy; a pool stays in rotation while it has at least its **Health Threshold** of healthy endpoints (status **Degraded** if some have failed), and goes **Critical** below it. A Critical pool gets no traffic, so the load balancer moves to the next healthy pool in order. When every pool is Critical, traffic goes to the **fallback pool**, the pool of last resort whose health is never checked (shown as **No health**); by default it is the last pool listed. If the fallback pool is disabled too, a proxied hostname returns a 530 with error 1016, and a DNS-only one answers with the SOA record. When the first pool recovers, traffic fails back to it automatically, except for clients that session affinity keeps elsewhere.
go deeper
Recall that pool order decides failover under Off steering, and that the fallback pool is the last resort when every pool is down.
Explain the Health Threshold, the Degraded and Critical statuses, automatic failback, and why the fallback pool's health is ignored.
Show you would size the threshold from survivor capacity, keep the fallback pool able to serve, and read 530 with 1016 as an exhausted fallback.
Weigh active-passive failover against active-active spreading: idle standby capacity and slower requests against tolerance to a whole region failing.
## Three layers of health Cloudflare Load Balancing decides where traffic goes from health at three levels, each computed from the one below. 1. **Endpoint health.** A monitor attached to the pool probes each endpoint from three data centers in each selected region. A region counts as healthy when most of its data centers pass, and the endpoint counts as healthy when most regions do. The API's `consecutive_down` and `consecutive_up` fields make an endpoint fail or pass several probes in a row before its status flips. 2. **Pool health.** Each pool has a **Health Threshold** (`minimum_origins` in the API): the number of healthy endpoints it needs to count as healthy. 3. **Load balancer health.** This is derived from its pools and shown in the dashboard. The pool statuses are the ones you read during an incident: | Pool status | Meaning | Receives traffic? | |---|---|---| | **Healthy** | every endpoint passes | yes | | **Degraded** | at least one endpoint has failed, but the pool still meets its Health Threshold | yes | | **Critical** | fewer healthy endpoints than the Health Threshold | no, unless it is the fallback pool | | **Health unknown** | no monitor attached, or the first results are not in yet | depends on whether a monitor is attached | | **No health** | reserved for the fallback pool, whose health is ignored | as the last resort | One subtlety: once a monitor is attached, an endpoint whose status is still unknown is treated as down. With no monitor at all, health is not considered, and Off steering sends everything to the first pool. ## Off steering: pool order is the failover order **Off - Failover** is one of the two standard steering policies, the other being **Random**. The pool list is a priority list. Take an API with three pools ordered `cloud-east`, `cloud-west`, `onprem-dc`: - While `cloud-east` meets its Health Threshold, it gets all traffic, even when Degraded. - When `cloud-east` goes Critical, traffic moves to `cloud-west`. - If `cloud-west` also goes Critical, traffic moves to `onprem-dc`. - If a pool lower in the order goes Critical, nothing visible happens; it is simply skipped if failover ever reaches it. This is the classic **active-passive** layout. Choosing any other steering policy over two pools gives an active-active one instead. **Failback** is automatic: when `cloud-east` is healthy again, new traffic returns to it. Clients that **session affinity** has pinned to an endpoint in `cloud-west` stay there until their affinity expires, so recovery is gradual rather than instant. ## The fallback pool Traffic can keep arriving even when every pool is disabled or Critical, and the load balancer has to send it somewhere. That somewhere is the **fallback pool**: - Its health is never taken into account, which is why the dashboard reports it as **No health**. - By default it is the last pool in the list; you can choose another in the dashboard or with the API's `fallback_pool`. - When the load balancer reaches **Critical** (all pools unhealthy), traffic flows to the fallback pool. Load Balancing analytics labels that traffic with the pool name followed by `(Fallback)`, so you can tell last-resort routing from normal steering. - If the fallback pool is also disabled, a **proxied** hostname returns **HTTP 530 with error 1016** (Origin DNS failure), and a **DNS-only** hostname answers with the zone's **SOA record**. Pick a fallback pool that can really serve: since nobody checks it, a dead fallback turns a regional outage into errors for everyone. ## What happens between two probes Monitors run on an interval, so an endpoint can die between probes. **Adaptive routing** covers that gap on proxied load balancers: - **Zero-Downtime Failover** makes one retry against another healthy endpoint in the same pool, and only for Cloudflare errors 521, 522, 523, 525 or 526. - **Failover across pools** carries that retry into the next pools in failover order when the current pool has no healthy endpoint left. Neither changes pool status; they rescue individual requests until the monitor catches up. ## Tuning the threshold for the scenario For a pool of three endpoints in `cloud-east`, a Health Threshold of 1 keeps the region serving on a single surviving server, while 2 moves traffic to `cloud-west` as soon as a second server fails. A higher threshold makes the pool more reliable as a destination but more likely to be marked unhealthy, so set it from how much load one region's survivors can really carry.
- Why might a small share of traffic reach the second pool while the first pool still shows Healthy?Health is decided by a majority of regions and data centers, but each Cloudflare data center acts on its own view. If probes fail from one data center while the endpoint is fine elsewhere, that data center can steer a few requests to a pool it considers healthy. The global status stays Healthy, and analytics shows the requests as steered away.
- How does a disabled fallback pool change what users see when every pool is Critical?With a working fallback pool, requests still reach an origin. With the fallback pool disabled as well, there is nowhere left: a proxied hostname returns HTTP 530 with error 1016, and a DNS-only hostname answers with the SOA record instead of an address.
- When would you raise a pool's Health Threshold above 1?When the survivors of a partly failed pool cannot carry its traffic. With a threshold of 2 on a three-server region, losing a second server moves traffic to the next pool instead of overloading the last server. The cost is earlier failover, so the next pool needs the capacity to absorb it.
- A pool stops receiving traffic as soon as one endpoint fails
- The fallback pool is used only if its own monitor passes
- Traffic stays on the backup pool until someone fails back by hand
- With no monitor attached, Off steering still moves traffic to the next pool
- The fallback pool must be a separate pool outside the main list