skip to content

In Django, how do you store user avatars in remote object storage instead of MEDIA_ROOT, and what must the storage backend provide?

level: middleimportance: should knowfreq 40%

answer

  1. swap the backend, keep the field
  2. a global default or per field
  3. a callable keeps migrations stable
  4. path() does not exist remotely

basics

~20 s

Point STORAGES['default'], or a named alias passed to the avatar field through a storage callable, at an object-storage backend. The backend is a deconstructible Storage subclass implementing _open and _save plus exists, url, delete and size; only path() stays unavailable.

solid answer

~50 s

Django writes uploads through a storage object, so moving avatars off local disk is configuration, not model rewriting. Either change the `BACKEND` of `STORAGES['default']`, which every `FileField` without its own `storage` uses, or define an alias such as `STORAGES['avatars']` and give the field `storage=select_avatar_storage`, a module-level function returning `storages['avatars']`. A callable is evaluated when models load, and migrations record the function rather than a storage instance, so per-environment configuration does not cause migration churn. The backend is a deconstructible `Storage` subclass that must implement `_open` and `_save` and normally overrides `exists`, `url`, `delete` and `size`, which otherwise raise `NotImplementedError`; `url()` usually returns an object-store link. Remote backends do not implement `path()`, so code calling `avatar.path` or passing the file to APIs that need a local path breaks; use `avatar.open()` and `avatar.url` instead. `DEFAULT_FILE_STORAGE` was removed in Django 5.1.

code

python · 9 lines
python
# settings.py
STORAGES = {
    'default': {'BACKEND': 'django.core.files.storage.FileSystemStorage'},
    'staticfiles': {'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage'},
    'avatars': {
        'BACKEND': 'config.storage.AvatarObjectStorage',  # a Storage subclass
        'OPTIONS': {'location': 'avatars'},
    },
}

go deeper

for a junior

Recall that FileField writes through a storage object and that STORAGES['default'] chooses it unless a field sets its own storage.

for a middle

Explain the Storage methods a backend implements, why path() is local-only, and why a callable storage keeps migrations stable.

for a senior

Find and fix code that assumes local disk, such as path(), MEDIA_ROOT walks and hand-built URLs, and plan signed URLs for private files.

for a principal

Weigh one global upload backend against per-field aliases for different privacy and retention needs, and own the migration of existing files.

## The storage abstraction Every `FileField` talks to a **storage object**, an instance of a subclass of `django.core.files.storage.Storage`. The field never touches the file system itself; it calls `storage.save()`, `storage.open()`, `storage.url()` and `storage.delete()`. Replacing local disk with an object store is therefore a matter of choosing a different storage class. ## Choosing where avatars go | Option | How | Effect | |---|---|---| | Change the global default | `STORAGES['default']['BACKEND']` | every `FileField` without its own `storage`, plus `default_storage` | | Named alias for one field | `STORAGES['avatars']` plus `storage=` on the field | only that field | | Storage instance on the field | `storage=SomeStorage(...)` | the instance is serialised into migrations | | Storage callable on the field | `storage=select_avatar_storage` | the function reference is serialised, the result can vary by environment | The callable form is what the Django docs show for selecting storage at run time: ```python from django.core.files.storage import storages from django.db import models def select_avatar_storage(): return storages['avatars'] class Profile(models.Model): avatar = models.ImageField(upload_to='avatars/', storage=select_avatar_storage) ``` Two details matter: - The callable is **evaluated when model classes load**, not per request. If tests override `STORAGES`, the docs recommend a `LazyObject` subclass whose `_setup()` resolves `storages['avatars']`, so the lookup happens on first use. - `STORAGES` replaces Django's default dictionary rather than merging with it, so define both `default` and `staticfiles` when you add an alias. ## What a backend must implement `Storage` provides the public methods and the name handling; the Django docs on custom storage split the rest into required and usual: 1. **Required**: `_open(name, mode)` returning a `File`, and `_save(name, content)` writing the bytes and returning the name actually used. 2. **Normally overridden** (they raise `NotImplementedError` otherwise): `exists(name)`, which `get_available_name()` relies on to avoid collisions; `url(name)`, an absolute URL a browser can fetch; `delete(name)`; `size(name)`; and `listdir(path)`, which a backend may deliberately omit. 3. **Deconstructible**: the class must be serialisable so migrations can record it when a field uses it; Django's `deconstructible` decorator handles that. `path(name)` is only for storages reachable through Python's `open()`. The base implementation raises `NotImplementedError`, and a local storage must override it while a remote one should not. ## Code that breaks when you switch - `profile.avatar.path` and anything built on it, such as a thumbnail step opening the local path. - Code that assumes `MEDIA_ROOT` layout, for example scripts walking the directory. - Hand-built URLs like `MEDIA_URL + name`; use `profile.avatar.url` so the backend decides. - `exists()` calls in hot paths: on a remote store each one is a network request, and `get_available_name()` calls it at least once per save. A random `upload_to` name keeps that to one call. ## Serving and privacy For public avatars, `url()` typically returns a stable link on the object store or a CDN in front of it, and the Django process never streams the bytes. For private files, backends commonly return **time-limited signed URLs** from `url()`, so access is granted by the application but served by the store. Either way, `MEDIA_URL` and `MEDIA_ROOT` become irrelevant for that field, because the remote storage defines its own location and base URL.

  • Why does passing a storage instance directly to FileField(storage=...) cause migration noise across environments?
    `FileField.deconstruct()` records the storage in migrations unless it is `default_storage`. An instance is serialised with its constructor arguments, so different options per environment make `makemigrations` detect a change. A callable is recorded as a function reference, which stays the same while the storage it returns varies.
  • A thumbnail task that worked locally fails after moving avatars to object storage; what is the likely cause?
    The task probably used `profile.avatar.path` to open the image from disk. Remote storages do not implement `path()`, so it raises `NotImplementedError`. Read the file through `profile.avatar.open()`, or download to a temporary file, and write the result back through the storage with `save()`.

The storage is like a cloakroom ticket system: the model keeps only the ticket number, and whether the coat hangs in the back room or in a warehouse across town is the attendant's business, as long as the attendant can fetch it and tell you where to collect it.

saying these in an interview costs you the question

  • Moving uploads to object storage requires changing every FileField's column type.
  • DEFAULT_FILE_STORAGE is still how Django 6.1 selects the upload backend.
  • field.path works the same on every storage backend.
  • A storage callable is evaluated on every request.
  • Adding a STORAGES alias keeps Django's built-in default entries automatically.