skip to content

In Go's r.ParseMultipartForm(maxMemory), what does the maxMemory argument actually bound?

level: middleimportance: should knowfreq 44%

answer

  1. a budget, not a limit
  2. the excess goes somewhere, not away
  3. text parts get their own reservation
  4. ten extra megabytes for non-file parts
  5. RAM traded for temp-directory space

basics

~20 s

maxMemory caps the bytes of file parts kept in RAM; the rest spills to temporary files on disk. It is not an upload size limit — a huge body is still received in full, just written to disk.

solid answer

~40 s

It is a memory budget, not a request limit. `ParseMultipartForm` reads the whole body and keeps up to `maxMemory` bytes of the *file* parts in memory; the remainder of those parts is written to temporary files in the OS temp directory, with `*multipart.FileHeader` values hiding which is which. On top of that budget the parser reserves a further 10 MB for the non-file (text) parts, which always stay in memory. So lowering `maxMemory` does not make a 5 GB upload cheaper — it converts RAM pressure into disk pressure and IO. If you actually need a ceiling on what a client can send, that is a separate control, and if you need to handle genuinely large uploads without buffering at all, process the body as a stream of parts instead of calling `ParseMultipartForm`.

code

go · 18 lines
go
// Keep at most 4 MiB of file parts in RAM; the rest is written to temp files.
if err := r.ParseMultipartForm(4 << 20); err != nil {
	http.Error(w, "malformed multipart body", http.StatusBadRequest)
	return
}

for _, fh := range r.MultipartForm.File["docs"] {
	f, err := fh.Open() // in-memory or temp file — same interface either way
	if err != nil {
		http.Error(w, "cannot read part", http.StatusInternalServerError)
		return
	}
	n, err := io.Copy(io.Discard, f)
	f.Close()
	if err == nil {
		log.Printf("%s: %d bytes", fh.Filename, n)
	}
}

go deeper

for a junior

Know that the number is a memory budget and that bigger uploads end up in temporary files on disk. Recall that the accessors parse the body for you with a 32 MB default if you never call it yourself.

for a middle

Explain the split precisely: file parts up to the budget in RAM, the remainder on disk, plus a separate reservation for the text parts. Be able to say why the call cannot reject an oversized body.

for a senior

Reason about it as capacity: multiply by concurrency, know which filesystem the temp directory sits on in your deployment, and say when you would abandon this API for streaming rather than tune the number.

for a principal

Own the tradeoff between the RAM budget your service is sized for and the disk budget somebody else pays for, and set a default across services so one team's tuning does not fill a shared ephemeral volume.

## The signature and the promise ```go func (r *http.Request) ParseMultipartForm(maxMemory int64) error ``` It parses a `multipart/form-data` body in full and populates `r.MultipartForm`, a `*multipart.Form`: ```go type Form struct { Value map[string][]string File map[string][]*FileHeader } ``` Text parts land in `Value`, file parts as descriptors in `File`. The documented rule for the argument is precise: up to `maxMemory` bytes of the file parts are stored **in memory**, with the remainder stored **on disk in temporary files**, and a further **10 MB is reserved for the non-file parts**. So the real in-memory high-water mark for one request is roughly `maxMemory + 10 MB`, not `maxMemory`. ## What it is not It is not a rejection threshold. Nothing about `ParseMultipartForm` returns an error because the body was too big — a 5 GB upload with `maxMemory` of 1 MB parses successfully, having written about 5 GB into your temp directory. This is the single most common misreading, and it matters because it changes which resource runs out: with a large `maxMemory` you fall over on RAM under concurrency, with a small one you fall over on disk. Bounding what a client may send at all is a separate concern from this argument, and streaming the body part by part is the way to avoid buffering entirely. It is also not per-part. The budget is a total across all the file parts in the request. Ten 1 MB files against a 4 MB budget means roughly four in memory and the rest on disk — but the split is by bytes as they are read, not by whole files, so a single large part can be partly in memory and partly on disk from the parser's point of view. ## Where the spill goes Spilled parts become temporary files created in the directory `os.TempDir()` reports — `$TMPDIR` on Unix if set, otherwise `/tmp`; the per-OS default elsewhere. Their names begin with a `multipart-` prefix, which is exactly what makes the spill visible: list the temp directory during a load run and you can watch the upload traffic. In a container the temp directory is usually the writable layer or an ephemeral volume, and it is frequently much smaller than people assume, so the spill lands on the tightest disk in the deployment. `*multipart.FileHeader` deliberately hides the difference. `fh.Open()` returns a `multipart.File` either way — a section reader over an in-memory byte slice, or a reader over the temp file — and both support `Read`, `ReadAt` and `Seek`. Your handler code does not change; only the resource cost does. ## Choosing a value Think of it as `concurrency × maxMemory` of RAM against `concurrency × (average upload − maxMemory)` of disk, and pick the side you have headroom on: - **Small uploads (avatars, CSVs of a few hundred KB).** Set `maxMemory` comfortably above the expected size and no spill happens at all; the temp directory stays empty and you avoid the IO entirely. - **Mixed or unpredictable sizes.** A modest budget — a few MB — is the usual compromise; small requests stay in RAM, occasional big ones pay disk. - **Genuinely large uploads.** Do not use `ParseMultipartForm` at all. Take the body as a stream of parts and copy each one to its destination as it arrives, so neither RAM nor the temp directory ever holds the whole thing. There is one implicit value you should know about: `r.FormFile` and the form accessors call `ParseMultipartForm` for you with a default budget of 32 MB when `r.MultipartForm` is still nil. A service that never calls `ParseMultipartForm` explicitly is running with that 32 MB budget per concurrent upload, which is a lot of RAM at a few hundred in-flight requests — calling it explicitly with a value you chose is the point of calling it at all. ## The failure mode to picture A file-upload endpoint sized for 2 MB avatars runs fine for months. A batch client starts posting 400 MB archives. RAM looks flat — the budget is doing its job — but the temp filesystem fills, and every component on the box that needs to write a temp file starts failing with "no space left on device", including ones that have nothing to do with uploads. The spill is invisible in a heap profile because it is not heap; the only signal is the size of the temp directory over time. That is the diagnostic to reach for, and the cost lands on whoever owns that disk rather than on the upload service's memory budget.

  • So how large a body can a client send if maxMemory is 1 MB?
    As large as it likes, as far as this call is concerned. The parser reads the whole body and writes everything past the budget to temp files, so the practical ceiling is the free space on the temp filesystem. Capping what a client may send is a separate control applied to the body, not this argument.
  • What budget applies if the handler never calls ParseMultipartForm explicitly?
    The accessors call it for you the first time they need a parsed form, with a default of 32 MB. That is per concurrent request, so a service that relies on the implicit call is quietly authorising 32 MB of in-flight buffering per upload. Calling it explicitly with a chosen number is the reason to call it at all.
  • Does maxMemory apply to the text fields as well?
    No. The budget covers the file parts; the non-file parts get a separate reservation of about 10 MB and are always held in memory. That is why the real per-request memory ceiling is roughly maxMemory plus 10 MB rather than the number you passed.

saying these in an interview costs you the question

  • Thinks maxMemory rejects uploads larger than it
  • Believes a small budget makes large uploads cheap
  • Says the excess is streamed to the handler, not stored
  • Assumes the spill goes to the working directory
  • Ignores that concurrency multiplies both the RAM and the disk cost