skip to content

When a pipeline stage hands a 32 MiB chunk buffer onward, how do moving it and deep-copying it differ in cost?

level: middleimportance: should knowfreq 47%

answer

  1. constant versus linear in the payload
  2. no allocator call versus one
  3. one live block versus two
  4. copy cost multiplies by pipeline depth
  5. copy buys an independently owned second version

basics

~20 s

Moving costs a few word stores and no allocation, whatever the chunk size. A deep copy costs one allocation plus 32 MiB of memory traffic per handoff, and leaves two blocks live and two releases to run.

solid answer

~40 s

A move copies only the owning record — pointer, length, capacity — so its cost is constant in the payload and the allocator is never consulted. A deep copy is linear in the payload: reserve a second block, walk 32 MiB of bytes, and now two blocks are live, each with its own release. On top of the raw copy you pay allocator pressure, page faults on freshly reserved memory, and cache displacement that slows unrelated work. With 8 chunks in flight, moving keeps the payload footprint at roughly 256 MiB; deep-copying at every one of three handoffs adds 96 MiB of copy traffic per chunk. The copy earns its keep only when the stage genuinely needs a second, independently owned version of the bytes.

code

pseudocode · 9 lines
pseudocode
// deep copy: linear in the payload, one allocation
new_block = allocate(source.length)        // reserve a second 32 MiB block
for i in 0 .. source.length - 1:
    new_block[i] = source.block[i]         // 32 MiB read, 32 MiB written
dest = handle(new_block, source.length)    // two blocks live, two releases due

// move: constant, no allocation
dest = handle(source.block, source.length) // same block, same address
source.block = none                        // source stops owning; one release due

go deeper

for a junior

Remember the two shapes: handing over ownership is a fixed small cost, while duplicating the contents grows with the amount of data being duplicated.

for a middle

Quantify it — constant versus linear, zero allocations versus one, one live block versus two — and name the second-order costs a raw byte count hides.

for a senior

Bring the pipeline arithmetic: copy cost multiplies by depth and footprint by how many stages hold a version at once, which is what turns a tidy handoff into a memory incident.

for a principal

Decide where copies are allowed at all. A move-by-default contract makes footprint a function of in-flight items rather than pipeline depth, and that is the property a capacity model needs.

## Three things you can hand on When a stage in an upload pipeline finishes with a chunk, it has three options, and they differ by an order of magnitude in cost: 1. **Move the owning handle.** The release duty goes to the next stage, the block does not move, and the source stops owning it. 2. **Hand out a non-owning view.** The duty stays with the current owner, which must then outlive every reader — a different mechanism with its own rules. 3. **Deep-copy the payload.** A second block is reserved and 32 MiB are copied into it; both the original and the copy are independently owned and independently released. This leaf is about the first and the third, and about the price tag that separates them. ## What each one actually costs | | Move the handle | Deep copy the payload | |---|---|---| | Bytes touched | a few machine words | 32 MiB read plus 32 MiB written | | Allocator calls | none | one reserve, and later one extra release | | Live blocks after | one | two | | Cost as chunk size grows | flat | linear | | Source usable afterwards | no | yes | The raw byte count understates the copy. Reserving 32 MiB usually means first-touch page faults as the pages are actually mapped; writing 32 MiB evicts whatever was in the cache, so unrelated code slows down afterwards; and the allocation and the eventual release are two more excursions into the allocator's bookkeeping, which is contended when several pipeline workers run at once. ## A number for the pipeline Take a pipeline holding **8 chunks of 32 MiB** in flight, each passing through **three handoffs** on its way to the uploader. - **Move everywhere:** payload footprint is 8 x 32 MiB = **256 MiB**, and the handoffs contribute no memory traffic at all. - **Copy at every handoff:** each chunk is copied 3 x 32 MiB = **96 MiB**, so the batch of 8 moves **768 MiB** through the memory system for no change in the bytes' content, plus 24 extra allocations and releases. Footprint also rises: whenever a stage still holds its own version while the next one holds the copy, two blocks per chunk are live. None of that work changes a single byte of the payload. It exists purely because the source wanted to remain an owner. ## When the copy is the honest answer A copy is not a defect — it is a purchase, and sometimes it is what you need: - the stage must **retry from the original bytes** after a downstream failure, and the downstream already consumed what it was given; - the stage must **hold the bytes for a different lifetime** — an audit trail, a checksum recomputed later, a cache — that does not match the pipeline's; - the payload is **small enough that the copy is noise**: duplicating a 200-byte header is not worth a design discussion, duplicating 32 MiB is; - the destination needs a **different representation** anyway, so bytes have to be walked regardless — then the copy is not an extra cost, it is the work itself. Outside those cases, a deep copy in a hot handoff usually means someone was unsure who was allowed to release the block and bought certainty with bandwidth. ## Reading the cost off the design The useful habit is to price a handoff before writing it: - **What is the payload size?** Constant-cost moves and linear-cost copies converge as the payload shrinks; at a few dozen bytes the distinction stops mattering. - **How many handoffs per item?** Copy cost multiplies by pipeline depth; move cost does not. - **How many items in flight?** Footprint multiplies by concurrency in both schemes, but a copy scheme also multiplies by how many stages hold a version simultaneously. - **Does anyone genuinely need the source afterwards?** If not, the copy is pure overhead, and the move both removes it and removes the ambiguity about who releases. An interviewer is listening for the shape of the answer, not exact figures: constant versus linear, no allocation versus one allocation, one live block versus two — and then a reason, not a reflex, whenever the copy is chosen.

  • Beyond the bytes copied, what second-order costs does a deep copy of a large chunk impose on the rest of the process?
    First-touch page faults as the new block is actually mapped, cache displacement that slows unrelated code afterwards, and extra traffic through the allocator's bookkeeping, which is contended when several workers allocate at once. It also raises peak footprint while both versions are live.
  • Does copying always mean an allocation?
    No. If the destination already owns a block of sufficient capacity, the bytes can be copied into it and no allocator call is needed — the copy is still linear in the payload. The allocation appears when the destination has no suitable block to reuse.
  • Why does the gap between moving and copying shrink for small payloads?
    A move is a fixed few word stores and a copy is that plus work proportional to the payload. At a few dozen bytes the proportional part is comparable to the fixed part, so the choice stops being a performance question and becomes purely a question of who should own the value.

saying these in an interview costs you the question

  • Prices a handoff by the payload size regardless of scheme
  • Counts only the bytes copied, ignoring allocation and page faults
  • Claims a copy is safer, without saying safer against what
  • Thinks copying avoids deciding who releases the block
  • Forgets that a copy leaves two blocks to release, not one