Why can an optimisation that halves a run's machine-time bill leave a bytes-read bill untouched?
answer
- two meters, not one price
- time held against input touched
- same run, different lever
- faster is not automatically cheaper
- reading less moves both meters
basics
~20 sMachine-time billing charges for capacity held multiplied by how long it was held, so it falls when a run finishes sooner or holds fewer machines. Bytes-read billing charges only for input touched, which falls only when the run reads less.
solid answer
~50 sA run's money cost comes from one of two meters, and they reward opposite behaviour. **Metered machine-time** charges for worker processes held times how long they were held — one worker process being one process on one machine that runs pieces of the job and owns their memory — so it falls when the run finishes sooner, holds fewer or cheaper machines, or hands capacity back as its tail drains. **Metered bytes read** charges for the volume of input the run touched and is blind to machines, duration and idle time. So packing work better, shortening a long tail, or taking cheaper interruptible machines moves the first meter and not one byte of the second. Reading a narrower slice of the input is the one lever that moves both, which makes it the safer default when you are not sure which meter you are on.
code
python · 23 lines# One run, priced by two meters. Only the second change moves both.
workers_held = 40 # worker processes the run held
seconds_held = 1800 # how long it held them
bytes_read = 4_000_000 # thousands of input bytes the run touched
def machine_time_bill(workers, seconds, rate):
return workers * seconds * rate
def bytes_read_bill(nbytes, rate):
return nbytes * rate
base_time = machine_time_bill(workers_held, seconds_held, 1) # 72_000
base_bytes = bytes_read_bill(bytes_read, 1) # 4_000_000
# Change A: run twice as wide and finish in half the time.
# Ideal scaling conserves the product; real runs pay extra for
# coordination and for moving records between more workers.
a_time = machine_time_bill(80, 900, 1) # 72_000 at best, higher in practice
a_bytes = base_bytes # unchanged: input is divided, not copied
# Change B: select half the input, everything else equal.
b_time = machine_time_bill(workers_held, seconds_held // 2, 1) # 36_000
b_bytes = bytes_read_bill(bytes_read // 2, 1) # 2_000_000go deeper
Recall that a run can be charged for how long machines were held or for how much input it read, and that these are different numbers. Knowing which one applies before trying to make something cheaper is most of the answer.
Explain the mechanics: capacity times duration against input volume. Show why running wider conserves the first product and never touches the second, and why narrowing the input is the one change that moves both meters.
Demonstrate that you measure both numbers for every run and keep them after the workers are gone. Talk about the tail that holds capacity idle, and about which platforms let you hand it back mid-run and which do not.
Argue the trade: which meter the organisation should want its workloads on, given that one rewards engineering the runtime and the other rewards engineering what gets read. Say what you would stop optimising once it stopped moving the invoice.
## Two meters, one run A **job run** — one submission of one program to a cluster, from start to its last written output — can be priced two entirely different ways, and which one applies is a property of how the compute was supplied rather than of the program you wrote. - **Metered machine-time.** You are charged for capacity held: the number of machines, or of **worker processes** (one process on one machine that runs pieces of the job and owns the memory they use), multiplied by how long they were held. The meter runs whether those workers are busy or idle. - **Metered bytes read.** You are charged for the volume of input the run touched, and nothing else: not how many machines served it, not how long it took, not whether most of them sat idle. Both shapes exist across this class of system. Where you rent machines and hand them a program, you are usually on machine-time. Where you hand work to a service that shows you no machines at all, you are usually on bytes read, or on an abstract per-run unit derived from it. Some platforms bill both lines at once, and some add a floor per run. The same logical workload can therefore sit on either meter depending only on where it was run, which is why *make this job cheaper* is not a single instruction. ## What each meter rewards | Change to the run | Machine-time bill | Bytes-read bill | |---|---|---| | Finish sooner on the same capacity | falls | unchanged | | Run twice as wide, finish in half the time | roughly flat, often slightly worse | unchanged | | Release workers as their pieces finish | falls, where the platform allows it | unchanged | | Accept interruptible machines the provider may reclaim | falls | unchanged | | Select a narrower slice of the input | usually falls | falls in proportion | | Avoid a second pass over the same input | falls | falls | | Write a much larger output | rises, from time spent writing | unchanged | Two rows carry the whole lesson. **Widening buys latency, not money.** Machines multiplied by seconds is conserved under perfect scaling, so running twice as wide for half as long leaves a machine-time bill roughly where it was — and in practice slightly higher, because a step that needs records from every other worker, a **wide step**, gets more expensive as the width grows, and because more workers means more coordination. The bytes meter does not move at all: the input is divided among workers, not handed to each of them. **Reading less is the only lever on the bytes meter, and it usually moves the other one too.** Less input means less to fetch, decode and pass along, so duration normally falls with it. That asymmetry is worth stating plainly: the set of changes that help a bytes-read bill is roughly a subset of the changes that help a machine-time bill. Whether the stored data is arranged so a reader is *allowed* to skip most of it is a storage-layout question and not this one; here the point is only that the meter counts what was read, so a filter the run applies after the bytes are already in memory has already been charged for them. ## Where the two meters put idle time Machine-time meters *held*, not *busy*. A run whose last few pieces grind for an hour while four hundred others finished in seconds is charged for the capacity it still holds for that whole hour. Platforms differ sharply in what you can do about it: some release workers as the work drains, some hold the shape for the life of the run, and some never showed you a worker to release. A bytes-read meter is entirely blind to that hour. The same tail is the dominant line on one invoice and invisible on the other. ## Which changes move only one meter 1. **Packing work more evenly across workers** — shortens the tail, cuts machine-time, changes no bytes read. 2. **Cheaper capacity** — interruptible machines or a cheaper machine shape cut machine-time; the bytes meter exposes no price dimension you control. 3. **Narrower selection of input** — cuts bytes read directly and machine-time indirectly. 4. **Fewer, larger output files** — barely moves either meter for this run, but changes what the next run pays to read the result. ## What to record per run Whichever meter applies, capture the same small set for every run so the two can even be compared: capacity held times duration, input bytes read, bytes moved between workers, output bytes written. Engines differ in what they publish — some expose a run-level bytes-read total, others only per-step input figures you have to sum, and a continuous job built from repeated small finite runs reports per slice, so a figure for a billing period must be aggregated. Capture these as numbers that outlive the run rather than reading them off a live screen: once the workers are gone, so is the evidence.
- Under machine-time billing, what is a run paying for while most of its workers have nothing left to do?For capacity held, not capacity used. If four hundred pieces finish in seconds and three grind for an hour, the run is billed for whatever it still holds during that hour. How much you can do about it depends on the platform: some let capacity be handed back as the work drains, some hold the run's shape until it ends, and some expose no machines to release at all.
- Which meter does a much larger output move?Neither directly on a bytes-read meter, which counts input touched. It moves machine-time, because writing takes time the run is charged for, and it adds a storage charge, which is a separate line from the run's compute. It also raises what the next run pays to read that output, which is where an oversized result usually costs the most.
- If you do not know which meter a workload sits on, what do you optimise first?Input read. It is the only change that reduces a bytes-read bill at all, and it almost always reduces machine-time too, because there is less to fetch, decode and pass between workers. Optimisations aimed at duration alone — more width, cheaper machines, better packing — can be worth doing, but they are a bet on being on the machine-time meter.
A taxi metered by the minute and a courier charged by parcel weight quote the same delivery on different terms. Taking a faster road cuts the taxi fare and nothing else; leaving half the parcel behind cuts the courier's charge and, incidentally, the taxi's too.
saying these in an interview costs you the question
- Assumes any speed-up reduces the bill on every meter
- Thinks doubling the worker count halves the money cost of a run
- Believes a bytes-read meter prices the output the run writes
- Calls a run cheap because it finished fast, ignoring capacity held
- Ranks optimisations without first asking which meter applies