skip to content

As networks keep attaching to a transit hub, which ceiling bites before bandwidth does, and how does it show up?

level: seniorimportance: nice to knowfreq 30%

answer

  1. entries, not bandwidth
  2. networks times prefixes per network
  3. propagation stops installing quietly
  4. partial reachability on the newest attachment
  5. summarise, or segment the hub tables

basics

~20 s

The route-table entry limit bites first. Entries scale with networks multiplied by the prefixes each announces, not with traffic, so a hub and its spokes hit a ceiling while bandwidth is still idle — and the symptom is partial reachability for the newest ranges.

solid answer

~50 s

Bandwidth on a hub scales with what you pay for; **route-table entries** do not. Every range an attached network announces has to occupy an entry in the hub route table that serves it, and usually another entry in each spoke's own table pointing back at the hub. That count grows with networks multiplied by prefixes per network, so an estate that keeps carving small subnets hits the ceiling long before traffic is interesting. The failure is nasty because it is partial: propagation for the newest attachment stops installing, so some ranges resolve and forward while others silently do not, which reads exactly like a filtering bug. The levers are summarising each network's ranges into fewer prefixes, segmenting attachments into separate hub route tables so no single table carries everything, and accepting that a per-attachment standing charge makes 'attach everything' a decision rather than a default.

code

pseudocode · 11 lines
pseudocode
# Invented estate: 40 attached networks, 6 prefixes each
networks = 40
prefixesAnnounced = 6

hubTableEntries = networks * prefixesAnnounced        # 240
spokeEntriesUnsummarised = (networks - 1) * prefixesAnnounced   # 234
spokeEntriesSummarised = 1   # one broad prefix pointed at the hub

# nothing here depends on traffic volume
if hubTableEntries > hubTableCeiling:
    newAttachmentRanges = "partially installed"

go deeper

for a junior

Take away one fact: a hub is limited by how many route entries its tables hold, and that count comes from how many networks attach and how many ranges each one announces.

for a middle

Explain the multiplier: networks times prefixes per network, on the hub side and again on each spoke unless a summary covers the estate. Say why traffic graphs show nothing.

for a senior

Recognise the signature in an incident: newest attachment, partial ranges, asymmetric. Separate 'never worked' from 'stopped working' before anyone touches a rule set.

for a principal

Set the budget up front: track entries alongside attachments, decide whether the estate's allocation can be summarised, and make hub route-table segmentation a standard rather than a rescue.

## What actually consumes the ceiling A transit hub feels like it removes limits, because it replaces a growing tangle of links with one attachment per network. It does remove the link-count problem. It does not remove the **route-table entry limit**, and that is the constraint that a growing estate meets first. The accounting is simple and worth doing on paper before an estate is built: - Every **prefix** an attached network announces needs an entry in whichever hub route table serves the attachments that must reach it. - Every attached network usually needs entries in its **own** route table for the remote ranges it must reach through the hub — unless those ranges can be covered by a summary. - The count therefore grows with **networks multiplied by prefixes per network**. It does not grow with traffic, with request rate, or with anything you would see on a bandwidth graph. That last point is what makes it a surprise. Every other limit in a network shows up as pressure you can watch climbing. This one is invisible until an attachment is added and something quietly fails to install. ## The symptom, and why it is misdiagnosed When the ceiling is reached, the platform does not stop the hub. It stops installing new entries. The effects, in the order people notice them: 1. The most recently attached network is reachable on some ranges and not others, because the propagation for its remaining prefixes had nowhere to go. 2. The failure is one-directional in places, because the hub side and the spoke side hit their ceilings independently. 3. Every test from an established network to another established network passes, so the hub "is fine". That combination — partial, asymmetric, newest-thing-affected — is exactly the signature of a badly ordered filtering rule, and teams burn hours in the rule sets before anyone reads the entry count. The tell is that a range which never worked is different from a range that stopped working: an entry that was never installed leaves no trace in the filtering layer at all. ## An estate on paper Assume an invented but realistic shape: forty attached networks, each announcing six prefixes because subnets were carved per zone and per tier. | Table | Entries needed | Grows with | |---|---|---| | Hub route table serving all attachments | 40 x 6 = 240 | networks x prefixes | | One spoke, unsummarised remote ranges | 39 x 6 = 234 | networks x prefixes | | One spoke, estate summarised into one prefix | 1 | nothing, until the summary breaks | | Full mesh instead of a hub, per network | 234 plus a target per link | pairs | The interesting row is the third one. A single summarised entry pointing the whole estate's address space at the hub collapses the spoke side to nothing — but only if the estate's ranges are contiguous enough to summarise, which is a property of how addresses were allocated in the first place and not something you can retrofit cheaply. ## The levers, in the order they are usually available 1. **Summarise on the spoke side.** Point a single broad prefix covering the estate at the hub, instead of one entry per remote network. Cheapest lever when the allocation permits it. 2. **Segment the hub's route tables.** Not every attachment needs to reach every other. Associating groups of attachments with separate hub route tables means no single table has to carry the whole estate, and it is also the mechanism that keeps two business units from reaching each other by accident. 3. **Reduce prefixes per network.** Six ranges per network is a choice, not a law; fewer, larger ranges per network cut the multiplier directly. 4. **Stop attaching networks that do not need it.** Each attachment carries a standing charge that accrues whether or not a byte moves, so an attachment created "to be ready" is a monthly line item and an entry cost with no return. ## Why it is always found late The first ten attachments work perfectly, and nothing on any dashboard is trending toward a cliff. The estate grows by one network at a time, each addition looks identical to the last, and the ceiling is a property of a table nobody looks at. The discipline is to make the entry count a number you track alongside the attachment count, and to know in advance which of the four levers you would pull — because the moment you need one, something is already partially unreachable.

  • Why does hitting this ceiling look like a filtering fault?
    Because it is partial and asymmetric. Some ranges of the newest attachment work and others do not, and the two directions fail independently, which is the classic signature of a misordered rule. The distinguishing question is whether the range ever worked: a filtering change breaks something that used to work, while a missing entry was never installed at all.
  • What does segmenting the hub into several route tables buy besides entries?
    Deliberate reachability. Attachments associated with different hub route tables do not reach each other, so a shared services network can be reachable from everything while two business units stay separated. It converts 'everyone can reach everyone because they are all attached' into an explicit decision per group.
  • Is a per-attachment charge a reason to attach fewer networks?
    It is one input. An attachment accrues a standing charge whether or not traffic flows, so attachments created speculatively cost money and entries for nothing. The stronger argument is usually the entry budget and the reachability that an attachment silently grants, but the standing charge is what makes the waste visible on a bill.

saying these in an interview costs you the question

  • Assumes a hub removes every scaling limit, not just the link count
  • Thinks route entries grow with traffic rather than with networks and prefixes
  • Expects a loud failure instead of partially installed routes
  • Believes every attachment must share one hub route table
  • Treats an idle attachment as free because no traffic crosses it