skip to content

Why do multipart temp files pile up on disk in a Go upload service after every request?

level: seniorimportance: nice to knowfreq 28%

answer

  1. the collector frees memory, not files
  2. one method deletes them all
  3. whoever built the form owns it
  4. the server only sweeps its own
  5. no spill means nothing to clean

basics

~20 s

net/http removes the temp files behind r.MultipartForm when the handler returns. They pile up when your own code called mime/multipart's ReadForm, because the server never sees that form — defer its RemoveAll, or stream the parts so nothing spills.

solid answer

~40 s

For the ordinary path there is an owner: parsing the form populates `r.MultipartForm`, and the server calls `r.MultipartForm.RemoveAll()` as it finishes the response, deleting every temp file the parse created. Files survive when nothing is holding that contract — most often because the code took `r.MultipartReader()` and called `ReadForm` on it itself, producing a `*multipart.Form` the server knows nothing about, or because multipart parsing happens outside an HTTP handler entirely. The fix is `defer form.RemoveAll()` on any form you created yourself. Two related traps: the process being killed mid-request leaves its temp files behind, and keeping a `*multipart.FileHeader` past the handler is useless anyway because the file it points at has already been deleted. The diagnostic is the temp directory itself — count `multipart-` files across a load run.

code

go · 17 lines
go
mr, err := r.MultipartReader()
if err != nil {
	http.Error(w, "expected a multipart/form-data body", http.StatusBadRequest)
	return
}

form, err := mr.ReadForm(4 << 20) // spills above 4 MiB, same as the buffered path
if err != nil {
	http.Error(w, "malformed multipart body", http.StatusBadRequest)
	return
}
defer form.RemoveAll() // the server cleans r.MultipartForm, not this one

for _, fh := range form.File["docs"] {
	// copy each part to its real destination before returning
	_ = fh
}

go deeper

for a junior

Know that oversized parts become real files on disk and that something has to delete them; the parsed form exposes a method that removes them all.

for a middle

Explain the ownership rule: the server removes the form it parsed onto the request when the handler returns, and any form your own code built is yours to remove with a deferred call.

for a senior

Diagnose it from the filesystem rather than from a profile, name the paths where cleanup never runs, and know that a header kept past the request points at a file that is already gone.

for a principal

Treat the memory budget as a spending decision on somebody else's disk: set the default across services, decide whether large uploads stream instead of spilling, and make the temp volume's high-water mark an explicit capacity number.

## Who deletes what A spilled multipart part is a real file, created in the directory `os.TempDir()` names — `$TMPDIR` on Unix, otherwise the platform default — with a name beginning `multipart-`. Nothing in the Go runtime reclaims it: the garbage collector frees memory, not files, and an unreferenced `*multipart.FileHeader` leaves its temp file exactly where it was. The cleanup is explicit and lives on the form: ```go func (f *multipart.Form) RemoveAll() error ``` It deletes every temp file backing that form's file parts. In the normal server path you get this for free: `r.ParseMultipartForm` (and therefore `r.FormFile` and the form accessors) stores the parsed form on `r.MultipartForm`, and when the response is finished the server calls `RemoveAll` on it. Handlers that use the buffered API and let the request end normally do not leak. ## The three ways files survive **1. A form the server does not know about.** If the handler takes `r.MultipartReader()` and then calls `ReadForm(maxMemory)` on that reader — a reasonable thing to do when you want streaming's control over the request but the buffered API's convenient maps — the resulting `*multipart.Form` is a local variable. `r.MultipartForm` holds only the server's internal marker, so the server's cleanup deletes nothing. The same is true anywhere multipart parsing happens away from a handler: a queue worker parsing a stored body, a test harness, a client parsing a multipart response. Whoever created the form owns it, and the fix is one deferred line. **2. The handler never finishes.** Cleanup runs as part of finishing the response, so a process killed or crashing mid-upload leaves its spill behind. On a container that restarts frequently and keeps its writable layer, or on a host whose temp directory is not a tmpfs and is never swept, those files accumulate restart after restart. Systems whose temp directory is cleaned on boot hide this; ones that keep a persistent volume for it do not. **3. Somebody kept the header.** A handler that pushes `*multipart.FileHeader` values onto a queue for processing after the response is written is making a subtler version of the same mistake. For a small part the bytes are held in the header and it works, which is why the bug ships. For a spilled part the temp file has been deleted by then and `fh.Open()` fails — an error that shows up only for uploads above the memory budget, that is, only in production and only for the biggest customers. ## Diagnosing it The spill is invisible to the tools people reach for first. It is not heap, so a heap profile shows nothing; it is not goroutines, so the goroutine profile shows nothing. The signal is the filesystem: - Watch the temp directory's file count and total size during a load run and, crucially, **after** it settles. A count that returns to zero when traffic stops is healthy; a monotonically rising floor is the leak. - Files named `multipart-*` older than your longest request are proof — nothing legitimate keeps one alive that long. - Correlate with `TMPDIR`: in many deployments the temp directory sits on the smallest, most-shared filesystem in the box, so the first symptom is an unrelated component failing with "no space left on device". ## Designing the leak out Three moves, in increasing order of how much they change: 1. **Defer `RemoveAll` on any form you created.** Cheap, explicit, and harmless if the server also sweeps — the second pass simply finds nothing to delete. 2. **Copy inside the request.** Read every part to its real destination before the handler returns, and never carry a `*multipart.FileHeader` past that point. Background processing should start from your storage, not from the parser's scratch space. 3. **Do not spill at all.** Streaming the parts and copying each to its destination means no temp file is ever created, so there is nothing to clean up and no accounting to get wrong. For a service whose uploads are large this is the honest fix rather than better housekeeping. The last point is worth making to whoever owns the disk: the in-memory budget you choose is also a decision about how much of every upload lands on their filesystem, how long it stays there, and what the high-water mark is under concurrency. Lowering the memory budget without saying so moves cost from your service's RAM to their volume.

  • Is it harmful to call r.MultipartForm.RemoveAll() yourself if the server also does it?
    No. The second sweep simply finds the files already gone. Deferring it in the handler buys determinism — the space is reclaimed as soon as you are done rather than when the response finishes — and it documents the ownership for the next reader. The only care needed is guarding against a nil form if parsing failed.
  • A handler queues *multipart.FileHeader values for a worker to process later. What breaks?
    For parts small enough to stay in memory, nothing — the header carries the bytes. For spilled parts the temp file has been deleted by the time the worker runs, so `Open()` fails. The bug therefore only appears above the memory budget. Copy the bytes to your own storage inside the handler and queue that location instead.
  • How would you confirm the leak in a load test rather than guessing?
    Sample the temp directory's file count and byte total before, during and well after a run. Healthy traffic returns the count to its baseline within one request timeout; a leak leaves a floor that grows run over run. Heap and goroutine profiles will look clean throughout, which is itself a useful signal.
  • Where do the temp files actually get created?
    In whatever directory `os.TempDir()` reports — `$TMPDIR` if set on Unix, otherwise the platform default — with names starting `multipart-`. In containers that is usually the writable layer or a small ephemeral volume, which is why an upload spill often surfaces first as an unrelated component failing to write.

saying these in an interview costs you the question

  • Says the garbage collector deletes the temp files
  • Expects the OS to sweep the temp directory automatically
  • Thinks closing a multipart.File removes its temp file
  • Keeps a *multipart.FileHeader for use after the handler returns
  • Assumes lowering the memory budget is free because RAM is saved