In a Worker that proxies R2 multipart video uploads with createMultipartUpload() and uploadPart(), why would complete() fail for some clients?
answer
- parts must match in size
- a 5 MiB floor except the last
- resume does no validation
- a seven-day default abort
basics
~20 sR2 requires non-final parts to share one size of at least 5 MiB, and complete() needs every part's partNumber and etag. Uploads are aborted seven days after creation by default, and each part must fit the Worker's request body limit.
solid answer
~50 sWith the binding, `createMultipartUpload(key)` returns an `uploadId`, each `uploadPart(partNumber, body)` returns an `R2UploadedPart` (`partNumber`, `etag`), and `complete(parts)` assembles them. `complete()` fails when non-final parts differ in size (error 10048), when a non-final part is under 5 MiB (10011, EntityTooSmall), or when a listed part is missing or carries the wrong etag (10025). A stale or invented `uploadId` surfaces only later: `resumeMultipartUpload()` is synchronous and checks nothing, so the next call fails with 10024, NoSuchUpload, which is also what a client sees after the bucket's default rule aborts the upload seven days after it began. Because each part travels through the Worker, it must also fit the plan's request body limit, 100 MB on Free and Pro. Fix it by having the Worker dictate one part size, persisting the uploadId and part list, and wrapping every call in error handling.
code
javascript · 39 linesexport default {
async fetch(request, env) {
const url = new URL(request.url);
const key = url.pathname.slice(1);
const action = url.searchParams.get("action");
const uploadId = url.searchParams.get("uploadId");
try {
if (request.method === "POST" && action === "create") {
const upload = await env.UPLOADS.createMultipartUpload(key, {
httpMetadata: { contentType: "video/mp4" },
});
return Response.json({ key: upload.key, uploadId: upload.uploadId });
}
if (request.method === "PUT" && action === "part" && uploadId && request.body) {
const partNumber = Number(url.searchParams.get("partNumber"));
const upload = env.UPLOADS.resumeMultipartUpload(key, uploadId);
const part = await upload.uploadPart(partNumber, request.body);
return Response.json(part);
}
if (request.method === "POST" && action === "complete" && uploadId) {
const { parts } = await request.json();
const upload = env.UPLOADS.resumeMultipartUpload(key, uploadId);
const object = await upload.complete(parts);
return new Response(null, { status: 201, headers: { etag: object.httpEtag } });
}
if (request.method === "DELETE" && action === "abort" && uploadId) {
await env.UPLOADS.resumeMultipartUpload(key, uploadId).abort();
return new Response(null, { status: 204 });
}
} catch (err) {
return new Response(String(err), { status: 400 });
}
return new Response("Bad request", { status: 400 });
},
};go deeper
Recall the three steps, create, upload parts and complete, and that complete() needs every part's number and etag.
Explain the part rules: uniform size, a 5 MiB floor except the last, at most 10,000 parts, and the seven-day default abort.
Diagnose from the error code, then fix the design: the Worker picks one part size under the request limit, records parts, and handles a vanished upload.
Weigh proxying parts through a Worker, which allows per-part checks, against clients using temporary credentials, which removes the request limit and the Worker's cost.
## How a Worker-driven multipart upload runs A **multipart upload** stores one object from many separately uploaded **parts**, so a client can upload in parallel and retry a single failed part instead of the whole file. Through the R2 binding it runs in three calls: 1. `env.UPLOADS.createMultipartUpload(key, options)` starts the upload and resolves to an `R2MultipartUpload` with `key` and `uploadId`. `options` sets `httpMetadata`, `customMetadata` and `storageClass` here, because `complete()` takes only the parts list. 2. For each part, a request arrives with the `uploadId` and a `partNumber`; the Worker calls `env.UPLOADS.resumeMultipartUpload(key, uploadId).uploadPart(partNumber, request.body)` and returns the resulting `R2UploadedPart` (`partNumber`, `etag`) to the client. 3. The client sends back the full list of parts, and the Worker calls `complete(parts)`, which resolves to the finished `R2Object`. From that moment the object is readable everywhere. `abort()` cancels an upload, and aborting is a free operation. Every `uploadPart()` and the create and complete calls are Class A operations. ## The part rules R2 enforces | Error | S3 code | Cause | |---|---|---| | 10048 | `InvalidPart` | Non-final parts differ in size | | 10011 | `EntityTooSmall` | A non-final part is smaller than 5 MiB | | 10025 | `InvalidPart` | A listed part was never uploaded, or its etag is wrong | | 10024 | `NoSuchUpload` | The `uploadId` is unknown, completed or aborted | The **uniform size** rule surprises teams most: a client that slices by time, or that sends a slightly larger part after a retry, produces parts of 5 MiB and 6 MiB and fails only at `complete()`. Other limits frame the design: - Parts range from 5 MiB to 5 GiB; only the last may be smaller. - An upload holds at most 10,000 parts, and the finished object at most about 5 TiB. - The finished object's etag is not the file's MD5: it is the MD5 of the parts' MD5s followed by `-` and the part count, such as `...-3`. ## Constraints that come from the Worker - **Request body limit.** Each part arrives as a request to the Worker, so it must fit the Cloudflare plan's limit: 100 MB on Free and Pro, 200 MB on Business. A 128 MiB part sent to a Pro-plan zone is refused with 413 before your code runs. - **No validation on resume.** `resumeMultipartUpload()` returns an object immediately, without checking that the upload exists, to save a round trip. A wrong `uploadId` fails on the first `uploadPart()` or `complete()`. - **Concurrent control.** Any invocation of your Worker, or any S3 API client, can complete or abort the same upload, so every call on an `R2MultipartUpload` needs `try`/`catch`. - **No part listing in the binding.** `R2MultipartUpload` exposes `uploadPart()`, `abort()` and `complete()` only. Either the client keeps the parts list or the Worker records it in a database; the S3 API's `ListParts` is the other way to recover it. ## Choosing a part size Let the Worker, not the client, choose a single part size and return it from the create call. With a 100 MB request ceiling, a part of 64 MiB leaves headroom and reaches 10,000 x 64 MiB, about 625 GiB, far above any user video. Smaller parts mean more Class A operations and more requests; larger ones mean longer retries on flaky connections. Two behaviours make retries safe. A client may upload several parts **in parallel** and in any order, as long as the list passed to `complete()` is in ascending `partNumber` order. And re-uploading a part under the same `partNumber` replaces the earlier copy, last write winning, so a timed-out part is simply sent again; its new etag must go into the list, because the old one now fails with 10025. If the Worker adds nothing per part, such as authorisation or a virus check, it can step aside entirely: a client holding R2 temporary credentials can run the multipart upload against the S3 API with an S3-compatible SDK, free of the Worker's request limit. ## Cleaning up abandoned uploads Users close tabs mid-upload. Every bucket has a **default lifecycle rule that aborts multipart uploads seven days after initiation**, and a client returning after that gets 10024 for every call. You can shorten the window for a prefix with a lifecycle rule, for example `wrangler r2 bucket lifecycle add <bucket> <rule-name> videos/ --abort-multipart-days 1`, and you should call `abort()` yourself when a client cancels.
- How does a thumbnail consumer learn that a multipart video upload into R2 has finished?Add an event notification rule: `wrangler r2 bucket notification create <bucket> --event-type object-create --queue <queue> --prefix videos/`. `object-create` fires on `PutObject`, `CopyObject` and `CompleteMultipartUpload`, not on each part, and the queue message carries `action`, `bucket`, `object.key`, `object.size` and `object.eTag`. Queues delivers at least once, so the consumer must tolerate a repeat.
- Why does a completed R2 multipart object's etag differ from the file's MD5 checksum?R2 computes a multipart object's etag from the parts, not from the whole file: the MD5 of the concatenated binary part MD5s, followed by a hyphen and the number of parts, such as `...-3`. To verify the whole file, have the client send its own SHA-256, store it in `customMetadata` at create time, and check it downstream.
- A client lost its list of uploaded parts halfway through an R2 multipart upload. Can the Worker binding recover it?No. The binding's `R2MultipartUpload` has `uploadPart()`, `abort()` and `complete()` but no way to list parts. Record each `R2UploadedPart` in a database as the Worker returns it, or call the S3 API's `ListParts` with credentials. Without either, the client re-uploads the missing parts under the same numbers.
- Parts may be any size as long as each one is at least 5 MiB
- resumeMultipartUpload() verifies the uploadId and throws if it is unknown
- An unfinished R2 multipart upload stays open until someone aborts it
- A Pro-plan Worker can receive a 1 GB part in one request
- complete() needs only the uploadId because R2 remembers the parts