How would you choose between an encapsulated overlay and a natively routed workload network for a large estate?
answer
- an organisational question first
- who can change the fabric
- addresses against packet headroom
- where visibility and enforcement land
- reversing it re-addresses the fleet
basics
~20 sDecide on control of the network, the address budget and what you need to see - not on benchmarks. An overlay needs no cooperation and lets pools repeat per cluster; native routing costs real addresses and agreement, and returns headroom and workload-level visibility.
solid answer
~50 sThe first question is not performance, it is whether you can get per-host workload ranges into the surrounding network's routing at all. If the fabric is not yours to change, native routing is off the table and the decision is made. If it is, the trade is real: routing consumes addresses from the site's own space that must stay unique across the routed domain, and in exchange every cross-host packet crosses as itself - no wrapper bytes, no reduced packet size to maintain, and the network can see and act on workload addresses. An overlay costs those three things and buys independence from the network's owners plus the ability to reuse one private pool in every cluster. I would default to routing on a large estate where the address budget allows it, because the daily savings are in visibility and in never re-living a packet-size incident, and use overlays where address reuse is the constraint or the fabric belongs to someone else.
go deeper
Know that the two designs exist: one wraps cross-host packets in host-to-host traffic, the other has the network route workload addresses directly.
Name the concrete trade - wrapper bytes and a reduced packet size against real addresses and cooperation from the network - and say which side each cost lands on.
Argue it from operations: where enforcement and visibility end up, which team debugs a failure, and what the packet-size obligation costs over the life of the fleet.
Take a position, state the evidence behind it, and name what would change it. Then account for reversibility: a change of design re-addresses the whole fleet and runs both models for a while.
## Start with who owns the network This reads like a technology comparison and is really an organisational one. A natively routed workload network requires that each host's range is known to the surrounding fabric, so that a packet addressed to a workload can be delivered as itself. That is a standing commitment from whoever operates that fabric: they carry your ranges, they re-carry them when the fleet grows, and they are on the call when a route is missing. If that commitment is not available - a rented estate, a provider's network, a team with different priorities - the decision is already made, and every other axis is a footnote. ## The axes that actually decide it 1. **Control of the fabric.** Can per-host ranges be routed at all, and by whom, and how fast when the plan changes? 2. **Address budget.** Routed ranges must be unique across the whole routed domain, so they come out of the site's real address space. An overlay hides workload addresses from every router, which means independent clusters can carve workloads out of the *same* private pool - a large saving where addresses are scarce. 3. **Packet budget.** A wrapper costs bytes on every packet and forces a reduced packet size on every workload interface, which must then be maintained forever and re-derived if anything ever adds a second header. 4. **Visibility and the enforcement point.** Routed, the fabric sees workload addresses and can count, filter and balance on them. Encapsulated, it sees host-to-host traffic only, and every one of those capabilities has to be rebuilt on the hosts. Some teams want exactly that; others lose a traffic picture they were relying on. 5. **Reach between estates.** Workloads that other estates must address directly are far simpler routed. Overlapping private pools in two clusters become a translation problem the moment they have to talk. 6. **Who carries it at three in the morning.** Routed failures are network failures diagnosed with network tooling; overlay failures are host failures diagnosed with platform tooling. Pick the one your team can actually debug. | | encapsulated overlay | natively routed | |---|---|---| | needs the fabric to carry workload ranges | no | yes | | addresses consumed from the site's space | host addresses only | every workload range | | pool may repeat between clusters | yes | no | | per-packet cost and reduced packet size | yes | no | | fabric can see workload addresses | no | yes | | who debugs it | the platform team | the network team, with you | ## How I would actually decide If the ranges can be routed and the address budget covers projected growth with room to spare, I would route natively on a large estate. The reasons are unglamorous and they compound daily: no packet-size trap to carry forever, no wrapper tax on a fleet whose packet rate is its cost, and workload addresses visible to the tooling the organisation already has. I would choose an overlay where the address budget is the binding constraint and clusters must reuse one pool, where the fabric belongs to someone else, or where the platform team needs to move faster than the network change process allows - and I would then write the reduced packet size into the platform's own configuration, with a test that proves it, rather than trusting anyone to remember. ## What would change my mind - A measured wrapper cost that is negligible against this fleet's actual packet profile weakens the packet-budget argument considerably. - A network organisation that can carry new ranges in hours rather than weeks makes routing far cheaper than its reputation. - A growth projection that exhausts the routable budget inside the plan's horizon argues for the overlay's pool reuse regardless of everything else. - Enforcement that has already moved onto the hosts removes most of the visibility argument for routing. ## Why the decision deserves this much care Because it is not cheaply reversible. Changing designs re-addresses every workload on every host, touches whatever was written against those addresses, and passes through a period where both models are live. It is a fleet-wide migration, not a setting - which is the strongest argument for deciding it deliberately, writing down what would change the answer, and revisiting it on evidence rather than on the next incident.
- What single fact settles this decision fastest?Whether per-host workload ranges can be put into the surrounding fabric's routing, and how quickly they can be changed afterwards. If the network is not yours to change, native routing is not available and the rest of the analysis is academic. If it is yours and the change process is measured in hours, routing becomes much cheaper than most teams assume.
- When would you deliberately run both designs in the same estate?When address reuse and direct reachability pull in opposite directions: overlays inside clusters, so each one carves workloads out of the same private pool, and native routing for the one cluster whose workloads other estates must address directly. The price is two operating models, two failure modes and two sets of people who have to understand the boundary between them - so it is worth it only when the address budget genuinely forces it.
saying these in an interview costs you the question
- Frames the choice purely as a throughput benchmark.
- Assumes the network's owners will simply accept the routes.
- Ignores that routed ranges consume the site's real address space.
- Treats the decision as reversible without a fleet-wide migration.
- Believes an overlay hides the choice from everyone downstream.