What does a byte-range GET against an Amazon S3 object do, and when would you use one instead of downloading the whole object?
answer
- one header on GetObject
- the answer comes back 206, not 200
- upload has parts, download has spans
- read the footer, skip the file
- every span is its own billed request
basics
~20 sA byte-range GET sends a Range header on GetObject so S3 returns only those bytes, with HTTP 206 Partial Content. Use it to download one object in parallel chunks, to resume an interrupted download, or to read a small region such as a file footer.
solid answer
~40 sYou add a `Range` header to `GetObject` and S3 returns just that byte span with status 206 Partial Content instead of 200. Three uses matter. First, parallel download: issue many range GETs for disjoint spans of a large object across threads and reassemble locally, which is exactly what the transfer managers do and the read-side mirror of multipart upload. Second, resume: after a failed download, request only from the byte you reached. Third, selective reads — pulling the footer of a large Parquet file, or a header, without transferring gigabytes you will discard. Ranges of roughly 8 to 16 MB are the usual choice for parallel fetches. Note each range GET is a separate billed request, and a range past the end of the object returns `InvalidRange`.
go deeper
Know that adding a Range header to GetObject returns only part of the object, and that this is how a download resumes after a failure.
Explain the 206 Partial Content response, parallel ranged download as the mirror of multipart upload, and why 8–16 MB is a sensible chunk size.
Bring the production angle: conditional If-Match so a resumed download cannot splice two object generations, request-count cost of over-splitting, and footer reads for columnar formats.
Frame it as an access-pattern decision — whether object layout and format let readers fetch only what they need, and what that implies for request cost and query latency at scale.
## The mechanism S3's `GetObject` honours the HTTP `Range` request header. Ask for `bytes=0-1048575` and S3 returns the first mebibyte of the object with status **206 Partial Content** and a `Content-Range` response header describing the span it actually sent, rather than the whole object with 200 OK. That is the entire feature; everything else is what you build on top of it. A range that lies entirely beyond the end of the object is rejected with an `InvalidRange` error (HTTP 416). A range that merely overshoots the end — `bytes=0-999999999` on a 10 MB object — is satisfied by returning what exists. S3 also accepts a `PartNumber` parameter on `GetObject`, which retrieves exactly the byte span that a given part occupied when the object was uploaded as a multipart upload. That is convenient when you want your download chunks to align exactly with the upload's part boundaries. ## Use one: parallel download This is the read-side mirror of multipart upload. A single HTTP stream is often the bottleneck for a large download, because one connection cannot fill a wide pipe. Split the object into disjoint ranges, fetch them concurrently, and write each into the right offset of the destination file: ```bash aws s3api get-object --bucket my-bucket --key big.bin \ --range bytes=0-16777215 part-000 aws s3api get-object --bucket my-bucket --key big.bin \ --range bytes=16777216-33554431 part-001 ``` Ranges in the region of **8 MB to 16 MB** are the commonly recommended size for this: large enough that per-request overhead is amortized, small enough that a failed chunk is cheap to refetch and concurrency stays high. Just as with upload part size, the practical ceiling is memory — range size multiplied by concurrency — and the practical floor is that tiny ranges turn one download into a request-count problem. ## Use two: resume If a download fails after 700 MB of a 2 GB object, you do not restart. You request `bytes=734003200-` and continue. Combined with a local record of progress, this makes long downloads over unreliable links tractable. Pair it with a conditional header such as `If-Match` on the object's ETag so that you notice if the object was replaced between attempts and do not silently splice two different generations of the object together. ## Use three: selective reads The most valuable use is reading a small, known region of a large object: - **Columnar file formats.** Parquet and ORC put their metadata footer at the end of the file. A reader fetches the last few kilobytes, learns the layout, then issues further ranges for exactly the column chunks and row groups the query needs. This is why analytics engines can query terabytes in S3 without downloading terabytes, and it is the single best illustration of why range GETs matter. - **Media.** Serving a seek into the middle of a large video, or reading a container's index. - **Cheap inspection.** Reading the first bytes to sniff a magic number or a header before deciding whether the object is worth processing. ## The costs and the traps - **Each range GET is a billed request.** Splitting one download into 200 ranges means 200 GET requests. Data transfer charges are unchanged (you move the same bytes), but request cost and per-request latency are not. Very small ranges are a false economy. - **You are responsible for reassembly.** Correct offsets, correct handling of the final partial range, and detection of an object that changed mid-download. - **Byte ranges are byte ranges, not records.** A range boundary will land in the middle of a line, a record or a multi-byte character. Any code that ranges over text has to stitch across boundaries. - **Server-side encryption is transparent** for SSE-S3 and SSE-KMS: ranges work normally. With SSE-C you must supply the customer key on the range request as on any other GET. - **Ranges do not reduce what you pay for retrieval** in storage classes that charge per-GB retrieval — you pay for what you retrieve, so a small range is cheap, but the request itself still counts. ## Relationship to multipart upload It is worth stating the symmetry explicitly, because interviewers often ask the two together: multipart upload parallelizes and makes resumable the way bytes go **in**; byte-range GET parallelizes and makes resumable the way they come **out**. Neither is exotic, both are what the SDK transfer managers do for you automatically, and both are ultimately about not letting a single serial HTTP stream define your throughput or your failure blast radius.
- How do you make sure a resumed range download does not splice together two different versions of the object?Capture the object's ETag (or version id, in a versioned bucket) on the first request and send `If-Match` with that ETag on subsequent range requests. If the object was replaced, the conditional fails instead of returning bytes from the new generation, and you restart cleanly rather than producing a corrupt file.
- Why can a query engine scan a huge Parquet file in S3 without downloading it?Parquet stores its metadata footer at the end. The reader issues a small range GET for the tail, learns where each row group and column chunk lives, then issues further ranges for only the columns and row groups the query touches. Bytes never needed are never transferred.
- Is splitting a download into as many small ranges as possible a good idea?No. Each range is a separately billed GET with its own latency, so very small ranges trade transfer time for request count and overhead. Something in the 8–16 MB region is the usual balance, and total memory is range size times concurrency, so the two must be tuned together.
saying these in an interview costs you the question
- Thinks a range GET returns 200 rather than 206
- Assumes S3 charges less data transfer for ranged reads
- Believes range boundaries align with lines or records
- Splits downloads into thousands of tiny ranges
- Thinks ranges require the object to be multipart-uploaded