skip to content

How does a resumable multipart upload let a phone on a flaky network finish a 4 GB video without restarting from zero?

level: middleimportance: must knowfreq 68%

answer

  1. one session, many parts
  2. failure costs only in-flight parts
  3. persist the session ID locally
  4. ask which parts already landed
  5. ceil(file size / part size)

basics

~20 s

The file is split into numbered parts uploaded independently under one session; each part is retried on its own, the client resumes by asking which parts already landed, and a final complete call assembles them into one object.

solid answer

~50 s

The client opens an **upload session** and gets an ID, cuts the file into fixed-size parts (with 8 MiB parts a 4 GiB video is 512 parts), and uploads a few in parallel, each tagged with its part number and a checksum. The store acknowledges each part with a token such as an `ETag` or digest. A failure costs only the parts in flight, which are retried with backoff. The client persists the session ID and acknowledged parts, so after a crash or network switch it sends only the parts the store lacks. When every part is in, a **complete** call lists the parts in order with their tokens; the store assembles them in one step, and the object is not readable before that. Part size is a trade-off: smaller parts resend less on a bad link, larger parts cut request overhead, and store limits on part count and minimum size bound the choice.

code

pseudocode · 17 lines
pseudocode
session = loadSession(file) or startUpload(file)      # returns uploadId
done = listUploadedParts(session.uploadId)            # map partNo -> token
for partNo in 1..ceil(file.size / PART_SIZE):
    if partNo in done: continue
    bytes = file.read(offset = (partNo - 1) * PART_SIZE, length = PART_SIZE)
    token = null
    for attempt in 1..5:
        try:
            token = putPart(session.uploadId, partNo, bytes, sha256(bytes))
            break
        catch NetworkError:
            sleep(backoffWithJitter(attempt))
    if token is null:
        fail("part " + partNo + " failed; session kept for a later resume")
    done[partNo] = token
    saveProgress(session, done)
completeUpload(session.uploadId, done sorted by partNo)

go deeper

for a junior

Remember the shape: split the file into numbered parts, upload them separately, retry only the part that failed, then send one final call that joins them into a single file.

for a middle

Explain the full session lifecycle, including where progress is persisted, how the client discovers which parts landed, and why the object only appears at completion. Do the part-count arithmetic out loud.

for a senior

Show production judgment: bounded parallelism with backoff and jitter, credential refresh on resume, detection of a file that changed mid-upload, and cleanup of sessions that are never completed.

for a principal

Discuss the part-size and concurrency policy as a fleet decision: request cost versus rework on bad links, store part limits for the largest files you accept, and how the policy differs by client network class.

## Why a single request fails on phones A single `PUT` of a 4 GiB video is all-or-nothing. Phones lose connectivity when they switch from Wi-Fi to cellular, enter a lift, or get suspended by the operating system. If the connection drops at 90%, a single-request upload throws away roughly 3.6 GiB (0.9 x 4,096 MiB = 3,686 MiB) and starts over, and on a sufficiently flaky link it may never finish at all. A **resumable multipart upload** turns one fragile transfer into many small, independently retryable ones. ## The session lifecycle 1. **Initiate.** The client asks for a new upload session and receives an `uploadId`. The session, not a connection, is what persists. 2. **Split.** The client divides the file into numbered **parts** of a fixed size; only the last part may be smaller. 3. **Upload parts.** Each part is sent with its part number (and usually its own checksum), often several in parallel. The store stores it and returns a **part token**, such as an `ETag` or a digest. 4. **Record progress.** The client durably saves the `uploadId` and the tokens it has received, so a crash or app restart does not lose the session. 5. **Resume.** After an interruption, the client asks the store (or the backend) which parts it already holds and uploads only the missing ones. 6. **Complete.** The client sends the ordered list of part numbers and tokens. The store stitches the parts into one object in a single step; before this, no readable object exists at the key. 7. **Abort or expire.** An abandoned session holds stored parts that still consume space until someone aborts it, so a cleanup rule is part of the design. ```pseudocode session = loadSession(file) or startUpload(file) done = listUploadedParts(session.uploadId) # map partNo -> token for partNo in 1..ceil(file.size / PART_SIZE): if partNo in done: continue bytes = file.read(offset = (partNo - 1) * PART_SIZE, length = PART_SIZE) token = putPartWithRetry(session.uploadId, partNo, bytes, sha256(bytes)) done[partNo] = token saveProgress(session, done) completeUpload(session.uploadId, done sorted by partNo) ``` ## Choosing a part size Part size is **arithmetic plus a trade-off**. The arithmetic: parts = ceil(file size / part size). Using binary units, a 4 GiB file is 4,096 MiB, so: | Part size | Parts for 4 GiB | Data resent if one part fails | |---|---|---| | 5 MiB | 820 (819.2 rounded up) | up to 5 MiB | | 8 MiB | 512 | up to 8 MiB | | 16 MiB | 256 | up to 16 MiB | | 64 MiB | 64 | up to 64 MiB | The trade-off: - **Smaller parts** waste less on each failure and suit slow, lossy links, but add requests, per-request latency, and bookkeeping. - **Larger parts** use fast links efficiently with fewer round trips, but a drop near the end of a part discards more data. - **Store limits** bound the choice. Object stores typically set a minimum part size and a maximum number of parts per upload; the exact values differ between systems. With an illustrative cap of 10,000 parts, a 100 GiB file (102,400 MiB) needs parts of at least 10.24 MiB, so a client would round up to 16 MiB and send 6,400 parts. A common heuristic is to choose a size where one part finishes in tens of seconds on the expected link. ## Retry and parallelism - **Per-part retry** with exponential backoff and jitter; a part is idempotent because re-uploading the same part number simply replaces it. - **Bounded parallelism.** A few concurrent parts raise throughput on good links; with 4 parts of 8 MiB in flight, a dropped connection loses at most 32 MiB. More parallelism is not always faster: a congested uplink just splits the same bandwidth and increases timeouts. - **Adapt concurrency, not size.** Many stores fix or constrain part size once a session starts, so clients usually adapt by changing how many parts run at once. - **Refresh credentials.** If each part uses a short-lived signed URL, a resumed session requests fresh URLs rather than reusing expired ones. ## An offset-based alternative Some resumable protocols use one logical stream instead of numbered parts: the client asks the server for the current **offset** of the upload and continues sending from that byte. It is simpler for sequential uploads but gives up parallel parts. Both designs share the same core idea: the server remembers progress, so the client never restarts from zero. ## What interviewers listen for Session ID persisted on the client, per-part retry, a completion step that makes the object appear atomically, sizing arithmetic that respects store limits, and a plan for sessions that are never completed.

  • The phone app is killed mid-upload and reopened an hour later. What does a well-built client do?
    It loads the persisted `uploadId` and part tokens, checks that the local file is unchanged (size, modification time or hash), and confirms the session is still alive. It then lists the parts the store holds, requests fresh per-part URLs if the old ones expired, and uploads only what is missing. If the session expired or the file changed, it aborts and starts a new session.
  • How should a client adapt a multipart upload to changing network conditions?
    Because many stores fix or constrain part size once a session begins, the main lever is concurrency: measure throughput and error rate, add parallel parts on a fast, stable link, and drop to one or two on a lossy one. Part size is chosen up front from the expected link and file size, staying above the store's minimum and keeping the part count under its cap.

saying these in an interview costs you the question

  • On any network error, restart the whole file upload.
  • More parallel parts always means a faster upload.
  • Smaller parts are always better because retries are cheaper.
  • Each part becomes readable in the object as soon as it lands.
  • The client does not need to persist the upload session ID.
  • Abandoned multipart sessions cost nothing and clean themselves up.