skip to content

In Django, how do MemoryFileUploadHandler and TemporaryFileUploadHandler decide where an upload is kept, and what does FILE_UPLOAD_MAX_MEMORY_SIZE control?

level: middleimportance: should knowfreq 45%

answer

  1. two handlers, tried in order
  2. 2.5 MB threshold
  3. judged on the whole request
  4. InMemoryUploadedFile versus TemporaryUploadedFile
  5. a threshold, not a limit

basics

~10 s

FILE_UPLOAD_HANDLERS runs MemoryFileUploadHandler first: if the whole request is at most FILE_UPLOAD_MAX_MEMORY_SIZE (2.5 MB), files become InMemoryUploadedFile objects; otherwise TemporaryFileUploadHandler writes TemporaryUploadedFile objects to disk. The setting is a threshold, not a size limit.

solid answer

~40 s

The default `FILE_UPLOAD_HANDLERS` is `["...MemoryFileUploadHandler", "...TemporaryFileUploadHandler"]`, and the multipart parser offers each file to them in order. `MemoryFileUploadHandler` activates in `handle_raw_input()` only when the request's content length is at most `FILE_UPLOAD_MAX_MEMORY_SIZE` (2,621,440 bytes, 2.5 MB). Note that this is the **whole request body**, not each file. When active it keeps the file in a `BytesIO` and raises `StopFutureHandlers`, producing an `InMemoryUploadedFile`. Otherwise `TemporaryFileUploadHandler` streams it into a `NamedTemporaryFile` in `FILE_UPLOAD_TEMP_DIR` (default `None`, the system temp directory), producing a `TemporaryUploadedFile` with `temporary_file_path()`. Nothing in this rejects a big file: a 20 MB scan simply goes to disk. The temporary file is deleted when it is closed, which Django does when the request finishes.

code

python · 13 lines
python
from django.http import JsonResponse
from django.views.decorators.http import require_POST


@require_POST
def inspect_upload(request):
    f = request.FILES["invoice"]
    return JsonResponse({
        "class": type(f).__name__,  # InMemoryUploadedFile or TemporaryUploadedFile
        "size": f.size,
        "on_disk": hasattr(f, "temporary_file_path"),
        "path": f.temporary_file_path() if hasattr(f, "temporary_file_path") else None,
    })

go deeper

for a junior

Know that small uploads stay in memory and larger ones go to a temporary file, with a default switch at 2.5 MB.

for a middle

Explain the handler chain, the whole-request comparison in handle_raw_input, StopFutureHandlers, and the two UploadedFile classes.

for a senior

Size FILE_UPLOAD_MAX_MEMORY_SIZE against worker memory and concurrency, and put FILE_UPLOAD_TEMP_DIR on monitored disk with room for peak uploads.

for a principal

Decide whether large files should pass through Django workers at all or go directly to object storage, trading simplicity against capacity.

## Upload handlers in one paragraph When Django parses a `multipart/form-data` POST, it does not decide on its own where file bytes go. It hands each file, chunk by chunk, to a list of **upload handlers**, objects that subclass `django.core.files.uploadhandler.FileUploadHandler`. The list comes from the `FILE_UPLOAD_HANDLERS` setting and can be changed per request through `request.upload_handlers`. The defaults are: ```python FILE_UPLOAD_HANDLERS = [ "django.core.files.uploadhandler.MemoryFileUploadHandler", "django.core.files.uploadhandler.TemporaryFileUploadHandler", ] ``` Order matters: the parser consults handlers from first to last, and a handler can stop the ones after it. ## How MemoryFileUploadHandler decides Before any file is read, the parser calls `handle_raw_input(input_data, META, content_length, boundary, encoding)` on every handler. `MemoryFileUploadHandler` uses it to set a flag: - it takes the request's content length (in current releases, the actual size of a seekable buffered stream when there is one, otherwise the `Content-Length` value); - it activates only if that length is **at most** `FILE_UPLOAD_MAX_MEMORY_SIZE`, whose default is `2621440` bytes (2.5 MB). If active, its `new_file()` creates a `BytesIO` and raises `StopFutureHandlers`, so the temporary-file handler never sees that file. When the file completes, it returns an `InMemoryUploadedFile`. The subtle point: the comparison uses the **size of the whole request body**, not the size of each file. Two 2 MB files in one request exceed the threshold together, so both go to disk. ## How TemporaryFileUploadHandler stores data If the memory handler is inactive, `TemporaryFileUploadHandler.new_file()` creates a `TemporaryUploadedFile`, which wraps `tempfile.NamedTemporaryFile` with a `.upload` suffix plus the original extension, in `FILE_UPLOAD_TEMP_DIR`. That setting defaults to `None`, meaning Python's default temporary directory. Each chunk is written as it arrives, so memory use stays flat regardless of file size. | | `InMemoryUploadedFile` | `TemporaryUploadedFile` | |---|---|---| | Created by | `MemoryFileUploadHandler` | `TemporaryFileUploadHandler` | | When | request body <= 2.5 MB | anything larger | | Storage | `BytesIO` in the worker's memory | named temp file on disk | | Extra API | none | `temporary_file_path()` | | `multiple_chunks()` | always `False` | `True` when larger than the chunk size | Both are `UploadedFile` subclasses with `name`, `size`, `content_type`, `charset`, `read()` and `chunks()`, so view code rarely needs to care which one it got. `temporary_file_path()` is the exception: tools that need a real path (an OCR command on a scanned invoice, for instance) should check `hasattr(f, "temporary_file_path")`. ## Cleanup The named temporary file is deleted when it is closed. Django registers `request.close()` as a resource closer on the response, and `request.close()` closes every file in `request.FILES`, so temp files disappear when the response is finished. A process that dies mid-request can leave files behind, which is one reason to point `FILE_UPLOAD_TEMP_DIR` at a directory you monitor. ## What the setting does not do - It does **not** limit upload size: a 200 MB file is accepted and written to disk. - It is **not** the same as `DATA_UPLOAD_MAX_MEMORY_SIZE`, which limits non-file request data and has the same 2.5 MB default. - Raising it trades memory for disk I/O: every concurrent upload below the threshold occupies worker memory. ## Tuning checklist - Keep the default 2.5 MB unless you have measured a reason; most scanned invoices will go to disk either way. - Multiply the threshold by the number of concurrent uploads a worker can hold to estimate the memory it can cost. - Point `FILE_UPLOAD_TEMP_DIR` at a volume with space for peak concurrent uploads, and monitor it. - Put a custom handler **before** the defaults if it must see every chunk, because an active memory handler stops later handlers from receiving a file. - Remember that the threshold never rejects anything: size limits need a separate control. ## Under ASGI Django's ASGI handler first spools the whole request body into a `SpooledTemporaryFile` with `max_size=FILE_UPLOAD_MAX_MEMORY_SIZE` before the view runs. The same setting therefore also decides when that buffered body moves from memory to disk.

  • Two 2 MB invoice scans are uploaded in one request. Which UploadedFile class does each become with default settings?
    Both become `TemporaryUploadedFile`. `MemoryFileUploadHandler` compares the whole request's content length, about 4 MB here, with `FILE_UPLOAD_MAX_MEMORY_SIZE` (2.5 MB). It stays inactive, so `TemporaryFileUploadHandler` writes each file to a temporary file even though each alone would fit under the threshold.
  • When are temporary upload files deleted?
    `TemporaryUploadedFile` wraps a `NamedTemporaryFile`, which is removed when closed. Django registers `request.close()` on the response, and it closes every file in `request.FILES` once the response is finished. If you need the data afterwards, save or move it in the view; a crashed worker can leave stray files in `FILE_UPLOAD_TEMP_DIR`.

The mailroom clerk looks at the size of the whole delivery van, not each parcel: a small van is unloaded onto the desk, and a big one goes straight to the storeroom, even if every parcel inside is small.

saying these in an interview costs you the question

  • FILE_UPLOAD_MAX_MEMORY_SIZE is the maximum allowed file size
  • The memory decision is made separately for each file
  • Files over the threshold are rejected with RequestDataTooBig
  • Temporary upload files stay on disk until a cron job removes them
  • DATA_UPLOAD_MAX_MEMORY_SIZE and FILE_UPLOAD_MAX_MEMORY_SIZE control the same thing