skip to content

In Django, how do you prefetch related objects for a list of model instances that did not come from a QuerySet?

level: middleimportance: nice to knowfreq 22%

answer

  1. no QuerySet left to chain on
  2. a function, not a method
  3. fills caches in place
  4. async variant with an a-prefix

basics

~10 s

Call django.db.models.prefetch_related_objects(instances, 'items__product'), or await aprefetch_related_objects() in async code. It runs the same batched queries prefetch_related() would and fills each instance's cache in place, skipping relations already prefetched.

solid answer

~40 s

`prefetch_related()` only works on an unevaluated `QuerySet`. When code already holds instances, such as orders read back from a cache, merged from two QuerySets or fetched earlier with `get()`, call `prefetch_related_objects(orders, 'items__product')` from `django.db.models`. It accepts the same string lookups and `Prefetch` objects, runs one batched query per lookup hop immediately, and fills each instance's prefetch cache (or `to_attr` attribute) in place, returning `None`. The instances must all be the same model class and the collection must be re-iterable, so pass a list rather than a generator. Instances whose relation is already fetched are skipped. Async code uses `await aprefetch_related_objects(...)`, available since Django 5.0.

code

python · 12 lines
python
from django.db.models import Prefetch, aprefetch_related_objects

from .models import OrderItem


async def hydrate_orders(orders):
    # orders: a list of Order instances, e.g. read back from a cache
    await aprefetch_related_objects(
        orders,
        Prefetch('items', queryset=OrderItem.objects.select_related('product')),
    )
    return orders

go deeper

for a junior

Remember that prefetch_related() needs a QuerySet, and that a separate function exists for a list of instances you already hold.

for a middle

Explain that prefetch_related_objects() fills caches in place, accepts Prefetch objects, needs same-class re-iterable instances and skips relations already loaded.

for a senior

Apply it where loading is decided late: cached instance lists, merged results, a single object from get(), and the async variant in async views.

for a principal

Use it to keep data-loading decisions at the rendering edge while caches and helpers pass plain instances around without hidden per-row queries.

## The gap prefetch_related() leaves `prefetch_related()` is a `QuerySet` method: it records lookups and applies them when that QuerySet is evaluated. Code often ends up holding model instances that no QuerySet will evaluate again: - a list of `Order` objects read back from a cache; - instances assembled from two QuerySets and merged or sorted in Python; - an object fetched earlier with `get()`, before anyone knew its items would be needed; - a helper function that receives `list[Order]` from its caller. Looping over `order.items.all()` on such a list runs one query per order, and there is no QuerySet left to call `prefetch_related()` on. ## prefetch_related_objects() `django.db.models.prefetch_related_objects(model_instances, *related_lookups)` runs the prefetch machinery against an iterable you already have. It is the same routine `prefetch_related()` uses internally after the main query, exposed as a function. ```python from django.db.models import Prefetch, prefetch_related_objects orders = load_cached_orders() # a list of Order instances prefetch_related_objects( orders, Prefetch('items', queryset=OrderItem.objects.select_related('product')), ) ``` What happens: 1. Django reads the lookups, which may be strings like `'items__product'` or `Prefetch` objects. 2. For each lookup level it skips instances whose relation is already fetched and runs one batched query for the rest. 3. It stores the results in each instance's prefetch cache, or on the `to_attr` attribute, exactly as `prefetch_related()` would. 4. It returns `None`: the instances are modified **in place**. ## Compared with prefetch_related() | | `QuerySet.prefetch_related()` | `prefetch_related_objects()` | |---|---|---| | Called on | an unevaluated QuerySet | a re-iterable collection of instances | | When queries run | when the QuerySet is evaluated | immediately, during the call | | Accepts `Prefetch` objects | yes | yes | | Returns | a new QuerySet | `None`; caches are filled in place | | Async form | evaluate the QuerySet with `async for` | `aprefetch_related_objects()` | ## Rules and limits - **Same model class.** Django documents that the instances must all be of the same class; the prefetch inspects the first instance and assumes the rest match. - **Iterable more than once.** Pass a list or tuple, not a generator, because the function walks the collection several times. - **Already-fetched relations are skipped.** An instance whose `items` were prefetched earlier is not queried again for that lookup. - **No joins.** There is no `select_related()` counterpart for plain lists; a `ForeignKey` named in the lookups is loaded with its own batched query. Put `select_related()` inside a `Prefetch` queryset to load the next hop within the prefetch query. - **Prefetch objects keep their power.** A custom queryset, a slice and `to_attr` behave exactly as they do with `prefetch_related()`. - **Database routing.** The prefetch query uses the database associated with the instances unless the `Prefetch` queryset names another with `using()`. ## Async code Since Django 5.0, `aprefetch_related_objects()` offers the same operation to async views: `await aprefetch_related_objects(orders, 'items')`. It wraps the synchronous function, so the queries are identical; it only moves the blocking ORM work off the event loop. ## A pitfall: calling it per instance The function batches across the collection it receives. Calling it inside a loop, one instance at a time, runs one query per call and recreates the per-row pattern it was meant to remove: ```python # Wrong: one prefetch query per order for order in orders: prefetch_related_objects([order], 'items') # Right: one prefetch query for the whole list prefetch_related_objects(orders, 'items') ``` Collect the instances first, then prefetch once. ## When it is the right answer The function earns its place wherever the loading decision is made after the instances exist. A typical example is an order list cached as instances for a few minutes: the cache stores orders, and the view calls `prefetch_related_objects()` on the cached list before rendering, so a cache hit costs one batched query per lookup hop instead of one query per order. It is also useful for a single instance: `prefetch_related_objects([order], 'items__product')` loads a detail page's graph without rewriting the earlier `get()`. In an interview, name the function, say it works in place on a re-iterable collection of same-class instances, and mention that it accepts `Prefetch` objects. That shows you understand prefetching as a separate step Django runs after the main query rather than something baked into the SQL.

  • Can you pass a generator of Order instances to prefetch_related_objects()?
    No. Django requires an iterable that can be traversed more than once, because it walks the instances at each lookup level and again when attaching results. Materialise the generator into a list first.
  • What does it do for instances whose items were already prefetched?
    It skips them for that lookup and queries only for the instances that still lack the relation, so calling it on a partly prepared list does not repeat work. Their existing cache is left as it is.

saying these in an interview costs you the question

  • prefetch_related() can be called directly on a Python list of instances
  • prefetch_related_objects() returns a new list and leaves the originals untouched
  • prefetch_related_objects() accepts only string lookups, not Prefetch objects
  • Instances of different models can be mixed in one prefetch_related_objects() call
  • aprefetch_related_objects() arrived with the 6.1 fetch modes