skip to content

Your preferred machine family is unavailable in a zone while a smaller profile launches there — what does that reveal about capacity pools?

level: middleimportance: should knowfreq 46%

answer

  1. the provider is rarely out of everything
  2. scarcity has an address
  3. which dimensions can the workload vary
  4. per zone, per family, per size
  5. an ordered list beats one pinned shape

basics

~20 s

Capacity is not one fleet-wide reservoir but many narrow pools, roughly one per zone per machine family and size. Scarcity therefore has an address, and a smaller or differently weighted shape often launches where the preferred one will not.

solid answer

~40 s

Free capacity is held in narrow pools — roughly one per zone, per family, per size — so `the provider is out of machines` is almost never true, while `this shape in this zone is` frequently is. The predictably thin pools are the accelerator-equipped families, the very largest sizes and the newest generation, because physical supply is smallest and demand is most concentrated there. The practical consequence is that a workload pinned to exactly one shape in exactly one zone has the worst possible launch odds. Express the requirement as a resource envelope instead of a catalogue name, derive an ordered preference list from it — acceptable families, sizes, zones — and let the launcher walk it. Several smaller machines are frequently obtainable when one large one is not, provided the workload can spread.

code

pseudocode · 22 lines
pseudocode
// ordered from most preferred to most acceptable
candidates = [
    { family: 'processor-weighted', size: 'large',  zone: 'primary',   units: 1 },
    { family: 'processor-weighted', size: 'medium', zone: 'primary',   units: 2 },
    { family: 'balanced',           size: 'large',  zone: 'primary',   units: 1 },
    { family: 'processor-weighted', size: 'large',  zone: 'secondary', units: 1 }
]

needed = 4
placed = 0

for each candidate in candidates:
    while placed < needed:
        result = requestMachine(candidate)
        if result.refusedBecause == 'no capacity':
            break                  // this pool is empty; move to the next candidate
        if result.refusedBecause != none:
            return reportUnexpectedRefusal(result)
        placed = placed + candidate.units

if placed < needed:
    return serveDegraded(placed, needed)

go deeper

for a junior

Recall that free machines are tracked separately for each zone and each machine family and size, so one shape being unavailable says nothing at all about the others.

for a middle

Explain which pools are predictably thin and how to express a workload's needs as a resource envelope, so a launcher can substitute a different family, size or zone for the preferred one.

for a senior

Demonstrate that the fallback path is tested rather than theoretical, and that you know what each substitution costs in performance, in price, and in cross-zone traffic.

for a principal

Weigh how much shape flexibility the organisation should require of its workloads against the engineering cost of making every service placeable anywhere.

## There is no single pool It is tempting to picture a provider's spare capacity as one reservoir per region that drains and refills. It behaves nothing like that. Free capacity is physical hardware sitting in a particular building, and what can be placed on it is constrained by what is installed there. The working model is **many narrow pools**: roughly one per zone, per machine family, per size — sometimes narrower still, because a very large shape may need a whole host while small shapes fit into fragments of one. That model explains the observation exactly. A refusal for one family in a zone while another family places immediately in the same zone is not a contradiction; it is what per-shape pools look like from the outside. ## Which pools are predictably thin - **Accelerator-equipped families.** Fewer hosts carry the hardware, it is the most contended thing a provider sells, and supply is built out slowly. - **The largest sizes.** A shape consuming most of a host can only land where most of a host is free; fragmentation alone refuses it while smaller shapes still fit. - **The newest generation.** Demand concentrates on it long before the installed base is large. The generation before it is usually the easy one to get. - **Smaller or older zones.** Zones inside a region are not all the same size, and the oldest zone in a popular region is often the one everybody's automation names first. - **Any pool at a correlated moment.** Scarcity is worst when many tenants want the same thing in the same minutes. ## What varying each dimension buys and costs | What you vary | Why it usually helps | What it costs | |---|---|---| | Size down, count up | small units fit fragments a large shape cannot | the workload must spread horizontally; per-machine overhead and licence counts rise | | An older generation | demand has moved on to the newer one | less performance per unit, usually worse price-performance | | A differently weighted family | the pools are independent of each other | you pay for a resource the workload does not use | | Another zone in the region | pools are per zone | cross-zone traffic is charged and adds latency, and the placement may undo a spread you wanted | ## Making a workload placeable The root cause of an unplaceable workload is usually that its requirement was written as a **shape name** rather than as a **resource envelope**. `it runs on this family at this size` is a sentence about a catalogue. `it needs this much memory per process, this much processor headroom at peak, and local disk it can afford to lose` is a sentence a launcher can satisfy several different ways. 1. Express the requirement as an envelope, and record the measurements it came from. 2. Derive an **ordered preference list** of concrete candidates from that envelope, most preferred first, and keep it in configuration so it can change without a code release. 3. Make the entries fail **independently** — different families, more than one generation, more than one zone, at least one smaller size at a higher count. Four entries that are all the same family in the same zone are one entry wearing four hats. 4. Exercise the fallback path deliberately, by forcing the first candidate to be skipped, so the day it runs is not the first day it has ever run. 5. Decide in advance what happens when the list is exhausted: wait, degrade, or shed. Something will happen; choose which. ## The limits of flexibility Substitution improves odds; it does not guarantee a launch, and there are two honest limits. The first is that flexibility is **shared**. At a correlated moment the obvious second and third choices are everyone's second and third choices, and they empty in roughly the order that every team's list agrees on. An unusual entry — an older generation, a differently weighted family, the less fashionable surviving zone — is worth more in that hour than another entry from near the top of the consensus. The second is that some workloads genuinely cannot be split. A process whose working set does not fit the smaller size, a licence counted per machine, or a component that must be a single writer will not accept two machines in place of one. For those, the list can vary only generation and zone, which is a shorter list and a worse position — and it is the strongest argument for holding an allocation instead of hoping to place one. ## What this does not fix Nothing here makes a launch certain. It converts a single point of refusal into several chances that fail independently, which is a large improvement when scarcity is ordinary and a much smaller one when scarcity is correlated. Where a launch genuinely must not fail, the answer is capacity that already exists or is already set aside, and the preference list becomes what covers the capacity above that line.

  • Why are accelerator-equipped and very large shapes the first to run out?
    Physical supply per zone is smallest for them: fewer hosts carry accelerators, and the largest sizes consume most of a host, so there are fewer places a request can land. Demand is also concentrated, because everyone wants the newest accelerator at the same time. Small shapes fit into the fragments that released capacity leaves behind, which larger shapes cannot use.
  • Your fallback list has four entries and all four are empty during a regional demand spike. What went wrong with the list?
    The entries were correlated. If they are the same family in the same zone, or simply the four choices everyone else also lists first, they empty together. A list earns its keep when its entries fail independently: different families, more than one generation, more than one zone, and a smaller size that fits the fragments.

saying these in an interview costs you the question

  • Says the provider has run out of machines generally
  • Pins the workload to one shape and calls it a requirement
  • Assumes the newest generation is always the easiest to obtain
  • Claims two smaller machines are never a substitute for one large one
  • Believes a fallback list removes the need for held capacity