skip to content

In Go's cipher.AEAD, which overlap between Seal's dst and plaintext is allowed?

level: seniorimportance: should knowfreq 30%

answer

  1. one array, two views
  2. where does the output start
  3. only exact overlap is legal
  4. plaintext[:0] is the allowed form
  5. the library panics rather than corrupt

basics

~20 s

Only exact overlap: passing plaintext[:0] as dst, so the output starts at the plaintext's first byte, is the documented in-place form. Any other overlap between the destination's output region and the plaintext makes Go's crypto/cipher panic rather than corrupt the data.

solid answer

~50 s

Go's `crypto/cipher` allows exactly one overlap: `Seal(plaintext[:0], nonce, plaintext, additionalData)`, where the output region starts at the same address as the plaintext. That is the documented way to encrypt in place. Any *inexact* overlap — a destination whose appended output would run across the plaintext at a different offset — is detected and panics with an invalid buffer overlap message, because the encryption would otherwise overwrite input it has not read yet. This bites when a job reuses one big `[]byte` per chunk: reading the chunk into `buf[12:]` and then sealing into `buf[:0]` to write the nonce in front looks like a free optimisation and panics on the first call. The fixes are to seal into a `nil` dst, to use the exact in-place form and write the nonce separately, or to keep the input and output in different arrays. `Open` has the same rule with `ciphertext[:0]`.

code

go · 10 lines
go
plain := buf[12 : 12+n] // chunk was read in at offset 12

// Panics: the output starts at buf[0] and runs across plain.
record := gcm.Seal(buf[:0], nonce, plain, nil)

// Safe: exact overlap, the documented in-place form.
ct := gcm.Seal(plain[:0], nonce, plain, nil)

// Safe: no overlap at all.
record = gcm.Seal(nil, nonce, plain, nil)

go deeper

for a junior

Remember the safe default: pass nil as Seal's dst and let it allocate. Reusing the plaintext's own buffer is an optimisation with a rule attached, not the normal way to call it.

for a middle

Explain why exact overlap is safe and inexact overlap is not — the cipher reads and writes the same window, so a shifted output destroys input it has not consumed — and name plaintext[:0] as the one legal in-place form.

for a senior

Diagnose it from the panic, then show the test that would have caught it: a round trip over the real reused buffer at several chunk sizes, not a fresh allocation per case. Say which of the three fixes you would take and why.

for a principal

Decide whether the team writes this by hand at all. An in-place AEAD helper with the aliasing rule encoded once, tested once, and imported everywhere is usually worth more than the allocation each caller might save rediscovering it.

## The rule `crypto/cipher` documents one and only one legal aliasing between a `Seal` call's destination and its plaintext: > To reuse the plaintext's storage for the encrypted output, use `plaintext[:0]` as `dst`. Otherwise, the remaining capacity of `dst` must not overlap the plaintext. The distinction is between **exact** and **inexact** overlap. Exact means the region `Seal` will write begins at the very same address as the plaintext it will read — the two slices march forward together, and GCM's counter mode reads each byte before it writes over it, so this is safe and is the intended in-place idiom. Inexact means the write region and the read region share an array but start at different offsets. There, `Seal` would clobber plaintext bytes it has not consumed yet, silently producing a record that decrypts to garbage. Go does not leave that to chance. The `crypto/cipher` implementations check for inexact overlap and **panic** — `crypto/cipher: invalid buffer overlap` — rather than emit a corrupt ciphertext. A panic in a batch job is recoverable; an archive full of records that no longer decrypt is not. ## How the bug is actually written The shape that trips people up comes from optimising, not from carelessness. A job encrypting archive chunks on their way to object storage reuses one large buffer per chunk to keep allocations flat. It reads the chunk into the buffer at an offset, leaving room at the front for the 12-byte nonce, and then tries to seal from the front so the whole record is contiguous and can be written in one call: ```go buf := make([]byte, 12+chunkSize+gcm.Overhead()) n, _ := io.ReadFull(src, buf[12:12+chunkSize]) plain := buf[12 : 12+n] record := gcm.Seal(buf[:0], nonce, plain, nil) // panics ``` `buf[:0]` has capacity for the whole record, so `Seal` will not allocate; it will write starting at `buf[0]`. The plaintext lives at `buf[12:]`. The write region and the read region overlap, at different offsets, and the call panics. The reason it feels correct is that the same trick with the `append` builtin *is* fine. `append` copies bytes forward one at a time and callers rarely notice; a cipher reads and writes the same window under a keystream and cannot. Aliasing rules that hold for slices in general do not carry over to `crypto/cipher`. ## The three fixes 1. **Let `Seal` allocate.** `gcm.Seal(nil, nonce, plain, nil)` is always correct and is the right default until a profile says otherwise. One allocation per chunk on a job that is doing AES on megabytes is very rarely the bottleneck. 2. **Use the exact in-place form.** `gcm.Seal(plain[:0], nonce, plain, nil)` encrypts into the plaintext's own storage — no allocation, no overlap violation. The record's nonce prefix then has to be dealt with separately: write the nonce to the sink first and the ciphertext second, or keep a small separate 12-byte header slice. 3. **Separate the arrays.** Read the plaintext into one buffer and seal into another. Two reused buffers still means zero allocations per chunk and no aliasing question at all. ## Open has the same rule, plus one more `Open`'s documented in-place form is `ciphertext[:0]` as `dst`, and any other overlap panics for the same reason. It carries an extra warning worth knowing: **even when `Open` returns an error, the contents of `dst` up to its capacity may have been overwritten.** So a decrypt path that reuses a buffer cannot assume the buffer still holds anything meaningful after a failed verification, and must not hand a partially written destination to anything downstream. ## Proving it, and proving the optimisation was worth it This defect has an unusually clean test. A round-trip unit test that seals one chunk and opens it again catches the panic immediately — but only if the test exercises the **real** buffer strategy. A test that calls `make` fresh per case has no aliasing and passes happily while production panics. Write the test against the same reused buffer the job uses, across several chunk sizes including one that fills the buffer exactly and one shorter than it, so the offsets vary. And since the whole reason to alias buffers is speed, the second half of the work is measuring it: a benchmark run with `-benchmem` tells you whether the in-place form actually removed allocations per operation, or whether you have taken on an aliasing hazard for a change that AES throughput swallows entirely. "Make it faster without changing behaviour" means the benchmark, not the intuition, decides whether the risky form stays.

  • Why does crypto/cipher panic here instead of returning an error?
    Because it is a programming mistake in the caller, not a runtime condition, and the alternative is worse: silently sealing over unread plaintext produces a record that looks fine and decrypts to garbage, possibly discovered months later when the archive is read. A panic fails on the first call, in the first test, and points at the exact line. `Seal` has no error return to use in any case.
  • Does the same rule apply to Open?
    Yes — the in-place form is `ciphertext[:0]` as `dst`, and any other overlap panics. `Open` adds one hazard: even when it returns an error, the contents of `dst` up to its capacity may already have been overwritten. So a reused decrypt buffer holds nothing trustworthy after a failed verification, and must not be forwarded on the error path.
  • Why did a unit test not catch this before it shipped?
    Almost always because the test allocated a fresh buffer per case with `make`, so nothing aliased and the panicking path was never taken. The test has to exercise the same reused buffer the job uses, across several chunk sizes — one that fills it exactly, one well short — so the plaintext sits at different offsets. Then a plain seal-and-open round trip fails on the first case.
  • How would you decide whether the in-place form is worth the hazard?
    Benchmark it with `-benchmem` and compare allocations and bytes per operation against the `Seal(nil, ...)` version at realistic chunk sizes. AES-GCM on a large chunk is a lot of work per allocation, so the saving is often in the noise; if it is, take the boring form. If it is real, keep the exact `plain[:0]` idiom and pin it down with a comment and a round-trip test.

It is the difference between rewriting a sentence starting at its first word, and starting twelve words earlier: the second one erases the words you have not read yet.

saying these in an interview costs you the question

  • Assumes any shared backing array is fine for Seal
  • Thinks Seal only writes forward past dst's length
  • Treats the panic as a bug in crypto/cipher
  • Recovers from the panic instead of fixing the overlap
  • Assumes dst is untouched when Open returns an error
  • Optimises the allocation away without a benchmark