skip to content

In Django REST Framework's SearchFilter, what do the ^, =, @ and $ prefixes on search_fields do, and how are several search terms combined?

level: middleimportance: should knowfreq 42%

answer

  1. prefixes pick an ORM lookup
  2. no prefix means contains
  3. @ needs PostgreSQL
  4. terms AND-ed, fields OR-ed

basics

~10 s

SearchFilter prefixes choose the lookup: ^ istartswith, = iexact, @ full-text search (PostgreSQL only), $ iregex, none icontains. Each search term must match at least one of the search_fields, and all terms must match.

solid answer

~40 s

`SearchFilter` reads `?search=` and applies it to the view's `search_fields`. A prefix on a field name picks the lookup: `^` is `istartswith`, `=` is `iexact`, `@` is Django's `search` full-text lookup (PostgreSQL with `django.contrib.postgres` installed), `$` is `iregex`, and no prefix is `icontains`. The parameter is split into terms on whitespace and commas, with quoted phrases kept whole. For each term DRF ORs the lookups across all fields, then ANDs the terms, so `?search=python remote` returns rows where 'python' matches some field and 'remote' matches some field. Searching through a many-to-many path is de-duplicated with an `Exists()` subquery rather than `distinct()`, and the docs warn that `$` lets clients send costly regexes.

code

python · 15 lines
python
from rest_framework import filters, generics

from jobs.models import Job
from jobs.serializers import JobSerializer


class JobSearchView(generics.ListAPIView):
    queryset = Job.objects.all()
    serializer_class = JobSerializer
    filter_backends = [filters.SearchFilter]
    search_fields = ["title", "^company__name", "=reference_code", "skills__name"]

# GET /jobs/?search=python%20"berlin%20office"
# terms: ['python', 'berlin office']; each must match title, company start,
# exact reference code or a skill name

go deeper

for a junior

Recall that no prefix means a case-insensitive contains match and that the query parameter is search by default.

for a middle

Give the full prefix table, and explain that terms are split on spaces and commas, OR-ed across fields and AND-ed across terms.

for a senior

Flag the regex prefix as a denial-of-service vector, explain the Exists-based de-duplication for to-many paths, and know when to move to full-text search.

for a principal

Decide when SearchFilter's simple matching stops being enough and a dedicated search engine or database full-text index should own relevance and ranking.

## How SearchFilter reads the request `rest_framework.filters.SearchFilter` is DRF's built-in free-text search backend, modelled on the Django admin's `search_fields`. It only acts when two things are true: the view declares a `search_fields` attribute, and the request carries a non-empty value in the search parameter. The parameter is `search` by default and can be renamed with the `SEARCH_PARAM` key in the `REST_FRAMEWORK` settings. The raw value is split into **terms**: - whitespace separates terms, and so do commas; - a phrase in quotes, such as `"senior engineer"`, stays a single term; - empty fragments are dropped. ## What each prefix means A prefix on a name in `search_fields` selects the ORM lookup used for that field. The table is the class attribute `lookup_prefixes`: | Prefix | Lookup | Meaning | Note | |---|---|---|---| | none | `icontains` | case-insensitive substring | the default | | `^` | `istartswith` | case-insensitive prefix | good for names and codes | | `=` | `iexact` | case-insensitive exact match | not case-sensitive | | `@` | `search` | full-text search | PostgreSQL with `django.contrib.postgres` in `INSTALLED_APPS` | | `$` | `iregex` | case-insensitive regular expression | the pattern comes from the client | Without a prefix, a field name may also carry an explicit lookup after `__` (for example `title__exact`); DRF detects a valid lookup at the end of the path and uses it instead of `icontains`. For a job board, `search_fields = ['title', '^company__name', '=reference_code']` matches titles anywhere, company names from the start, and reference codes exactly (ignoring case). ## How terms and fields combine DRF builds one `Q` object per term and field, then combines them in two steps: 1. for each term, the lookups over all `search_fields` are **OR-ed**: the term may match any field; 2. the per-term conditions are **AND-ed**: every term must match somewhere. So `?search=python remote` returns jobs where 'python' appears in some searched field **and** 'remote' appears in some searched field (possibly a different one). It does not look for the phrase 'python remote', and one matching term is not enough. ## Related fields, JSON keys and duplicates - Related paths use the double-underscore notation: `company__name`, `skills__name`. - Nested keys in a `JSONField` or `HStoreField` use the same notation, such as `data__stack`. - A path through a **many-to-many** relation (or another to-many join) would multiply rows. DRF detects that and, instead of calling `distinct()`, re-filters the original queryset with an `Exists()` subquery on the primary key, the same approach the Django admin uses. Fields that are queryset annotations skip this check. ## Safety, customisation and the 3.18 variant - **Regex search is a denial-of-service risk.** With `$`, the client writes the regular expression that the database evaluates on every candidate row. The DRF docs warn about catastrophic-backtracking patterns; prefer `^`, `=` or full-text search for untrusted clients. - **Dynamic fields.** Override `get_search_fields(self, view, request)` in a subclass to change the searched fields per request, for example searching only titles when a `title_only` parameter is present. - **Term handling.** Override `get_search_terms(self, request)` to cap or normalise the terms. - **Accent-insensitive search.** DRF 3.18.0 added `UnaccentedSearchFilter`, which wraps the lookups in Django's `unaccent` transform (`unaccent__icontains` and so on) so 'Jeremy' matches 'Jérémy'. It needs PostgreSQL, the `unaccent` extension and `django.contrib.postgres`; the `@` prefix stays accent-sensitive because `unaccent` cannot wrap the full-text lookup.

  • How would you search only the title when the client passes ?title_only=1?
    Subclass `SearchFilter` and override `get_search_fields(self, view, request)`: return `['title']` when `request.query_params.get('title_only')` is set, otherwise `super().get_search_fields(view, request)`. List the subclass in the view's `filter_backends` instead of `SearchFilter`.
  • Why might ?search=python on a many-to-many skills field return each job once rather than once per matching skill?
    `SearchFilter.must_call_distinct()` notices a to-many path in `search_fields`. DRF then re-filters the original queryset with `Exists()` over a subquery that matches the primary key, so each job appears once. It avoids `distinct()`, which can clash with ordering on other columns.

saying these in an interview costs you the question

  • The = prefix does a case-sensitive exact match
  • Several search terms are OR-ed, so one matching term is enough
  • The @ prefix works on any database backend
  • SearchFilter searches the phrase as typed rather than splitting it into terms
  • Searching across a many-to-many field always returns duplicate rows