A design review claims a branch site reaches four nines because it has two WAN circuits at 99.9% each; how do you evaluate that claim and decide what to change?

## Recompute the claim The arithmetic in the claim is not wrong; its scope is. Two circuits at 99.9% each, failing independently, are down together `0.001 × 0.001 = 0.000001` of the time, so the **pair** is about 99.9999%, roughly 32 seconds a year. But the site's users do not experience the pair; they experience the **whole path** from their access switch to the far end, and that path contains elements that are not duplicated. ## Hunt the series elements Walk the path physically and logically and list everything traffic depends on that exists only once: - **The edge router.** Two circuits into one router make the router a series element. - **Power.** One building feed, one UPS, one rack power strip. - **The physical route.** Both circuits through one duct, one building entry or one street; a single excavation cuts both. - **The upstream.** Two circuits from the same provider exchange, or two carriers who lease the same fibre. - **Software and change.** Two devices on the same release share every defect in it; one change pushed to both can break both. - **The LAN side.** If hosts point at one gateway address with no gateway-redundancy protocol, a second router does nothing for them. ## Model it Give each element an availability — from vendor data, provider history or your own incident records — and combine them: shared causes in series, independent alternatives in parallel. With illustrative figures: | Element | Unavailability | Downtime per year | |---|---|---| | Single edge router at 99.95% | 0.0005 | about 263 min | | Shared duct for both circuits | 0.0002 | about 105 min | | Circuit pair, independent part | 0.000001 | under 1 min | | **Site path** (about 99.93%) | **about 0.0007** | **about 6.1 hours** | The circuits, the subject of the claim, contribute under a minute. The router and the duct contribute almost everything. ## Where the figures come from Every number in such a model is an estimate, and the review should say how good each one is: - **Device figures** come from published MTBF and your own MTTR. The MTTR is usually the weaker number — a branch two hours' drive from the nearest spare has a very different repair time from one with a spare on the shelf. - **Circuit figures** come from the provider's service terms and, better, from your own outage history with that provider at that site type. - **Shared-cause figures** — a duct cut, a building power event, a release defect — are the hardest to estimate and the most important, because they sit in series. Use incident history across the estate, and when in doubt run the model with a pessimistic and an optimistic value. If the conclusion flips between the two values, the design is too close to the target to claim it; if it holds either way, the number is good enough to act on. ## Decide what to change Rank fixes by the downtime they remove per unit of cost. 1. **Second edge router, each on its own circuit and building entry.** Each path is now router × circuit in series, `0.9995 × 0.999 ≈ 0.9985`, and the two paths are in parallel: `1 − 0.0015² ≈ 99.9998%`. Paths must be computed this way — series inside each path, then parallel across paths — unless the routers are cross-connected to both circuits. 2. **Verify the diversity.** Ask for route records showing separate entries, ducts and upstream exchanges; two contracts with two carriers do not prove two physical paths. 3. **Remove the next series term.** With the paths fixed, a single building power feed at an assumed 99.99% becomes the cap: `0.9999 × 0.999998 ≈ 99.9898%`, about 53.7 minutes a year — still just short of four nines' 52.6-minute allowance. A second feed or a battery runtime long enough to cover typical outages is the next spend. 4. **Cut MTTR.** Monitoring, spares on site and rehearsed procedures shrink every remaining term at once. 5. **Separate the software.** Upgrade the two routers at different times, so a defective release shows itself on one router before it reaches the other. ## Question the target Four nines at a branch can cost far more than three nines. The review should end with numbers the business can price: what each option costs, the downtime it removes, and what that downtime costs. Sometimes the right answer is to accept about 99.95% and spend the money on faster repair. Whatever is chosen, test it: fail each element on purpose and measure, because a model with a hidden shared cause is only as good as the trace behind it.