In Go, how do you cap how much data a gzip.Reader can produce from an untrusted .gz file?

## The threat DEFLATE can compress a highly repetitive input by a ratio of roughly a thousand to one, and nested or crafted streams do far better. A few hundred kilobytes of `.gz` that arrived from somewhere you do not control can expand into many gigabytes. If your code does `io.ReadAll(zr)` or `io.Copy(buf, zr)` with no ceiling, the process allocates until the kernel kills it — or, on a node with a memory limit, takes the workload next to it down with it. This is a decompression bomb, and the defence is a hard ceiling on **output** bytes. ## Why the limit goes on the outside of the decompressor Capping the compressed input does not bound the output at all; that is the whole point of the attack. The reader you must limit is the `*gzip.Reader` itself, because that is the one whose `Read` produces expanded bytes: ```go const maxDecompressed = 64 << 20 // 64 MiB zr, err := gzip.NewReader(src) if err != nil { return err } defer zr.Close() n, err := io.Copy(dst, io.LimitReader(zr, maxDecompressed+1)) if err != nil { return err } if n > maxDecompressed { return errors.New("gzip stream exceeds the 64 MiB decompressed limit") } ``` A limited reader stops the copy at the ceiling, so memory and time stay bounded whatever the input claims. ## The +1, and why it is not a stylistic detail `io.LimitReader(r, n)` returns a reader that yields at most `n` bytes and then reports `io.EOF`. It does **not** return a distinguishable error. If you limit at exactly your budget and the copy stops there, you cannot tell 'the file happened to be exactly 64 MiB and is fine' from 'the file is 40 GiB and I truncated it', and silently truncating attacker-supplied data is usually worse than rejecting it — you end up storing a half-parsed record and calling it success. Reading `max+1` makes the test unambiguous: any count above `max` means there was more. `io.CopyN` with an explicit `io.EOF` check is an equivalent formulation. ## What not to trust The gzip trailer carries ISIZE, the uncompressed length modulo 2^32. It is tempting as a pre-flight check and it is useless as one: the producer writes it, so an attacker sets it to whatever is convenient; it wraps at 4 GiB, so a 5 GiB stream can declare 1 GiB; and it is physically at the end of the stream, so you would have to decompress everything to read it. Go's `gzip.Reader` does verify it — together with the CRC32 — but only when the caller reads all the way to `io.EOF`, and by then the damage would already be done. Its job is integrity, not admission control. The same reasoning applies to any size hint in an envelope around the data. ## Choosing the ceiling Pick it from what the legitimate producer actually emits, with headroom, and make it configuration rather than a constant so it can be raised at 3 a.m. without a release. Log the rejection with the stream's identity and the byte count so the on-call engineer can tell a bomb from a genuinely oversized batch — those two need opposite responses, and an error that says only 'limit exceeded' forces a guess. ## Related hardening in the same code path - Close the `gzip.Reader`. It does not close the underlying reader, but it releases the decompressor state, and pairing it with a fully consumed stream is what lets the checksum verification actually run. - If the file legitimately contains several concatenated members, the limit must cover the whole thing, since Go's reader concatenates them transparently by default. - Bound concurrency as well as size. A 64 MiB ceiling per stream and unbounded parallel decompressions is still an unbounded memory footprint; the two limits are separate decisions. - Stream to a file or a bounded consumer rather than into memory where you can, so the ceiling protects against latency and disk as well as heap. ## The one-sentence version Limit the reader that produces expanded bytes, budget one byte more than you will accept so the ceiling is detectable, and never let the stream describe its own size.