skip to content

Query Engine

FT.CREATE defines secondary indexes over hashes and JSON, and FT.SEARCH and FT.AGGREGATE query them, vectors included. Interviewers ask what this costs and when a search engine is the better fit.

part ofRedisread it →
on this pageshow

explore

questions

6

With the Redis FT.CREATE command, how do you index hashes under a product: prefix, and when should a field be TAG rather than TEXT?

level: middleimportance: must knowfreq 44%

answer

  1. type, prefix, schema
  2. the server follows your writes
  3. whole value or words inside it
  4. curly braces against a field term

basics

~20 s

FT.CREATE declares which keys an index follows, by data type and prefix, and which fields it indexes; ordinary HSET writes then keep it current. Use TAG for exact-match labels such as categories, and TEXT for words searched inside longer strings.

solid answer

~50 s

`FT.CREATE idx:products ON HASH PREFIX 1 product: SCHEMA ...` tells Redis to follow every hash whose key starts with `product:` and to index the declared fields. The documents stay ordinary hashes. Existing keys are picked up by a background scan, while new and changed keys are indexed synchronously as part of the `HSET`, so the application makes no second write. I pick the type by how a value is matched. `TAG` keeps the whole value, splits only on a separator and is queried as `@category:{laptops}`, at about one or two bytes per entry. `TEXT` tokenizes and stems, stores frequencies and positions, and suits names and descriptions queried as `@title:wireless`. `NUMERIC` takes ranges and `GEO` takes a radius. `SORTABLE` adds memory in exchange for fast sorting, so I declare it only on fields I sort or aggregate by.

code

plaintext · 11 lines
plaintext
127.0.0.1:6379> FT.CREATE idx:products ON HASH PREFIX 1 product: SCHEMA title TEXT WEIGHT 5 brand TAG SORTABLE category TAG price NUMERIC SORTABLE location GEO
OK
127.0.0.1:6379> HSET product:1 title "Aero 14 ultrabook" brand "Aero" category "laptops,ultrabooks" price 999 location "-122.41,37.77"
(integer) 5
127.0.0.1:6379> FT.SEARCH idx:products "@category:{laptops} @price:[500 1500]" RETURN 2 title price
1) (integer) 1
2) "product:1"
3) 1) "title"
   2) "Aero 14 ultrabook"
   3) "price"
   4) "999"

go deeper

for a junior

Recall the shape of the command: ON HASH, PREFIX with its count, then SCHEMA with a type per field, and that writes use plain HSET.

for a middle

Explain background scanning of existing keys against synchronous indexing of new ones, then justify TAG or TEXT by how the value gets matched and what each stores.

for a senior

Show you weigh each indexed field and SORTABLE as a memory and write cost, and that you plan schema changes around a rebuilt index and an alias from the first release.

for a principal

Treat the schema as a contract between teams: who may add indexed fields to a shared instance, and how a change is rolled out and reviewed against its memory cost.

## What FT.CREATE declares A Redis **query-engine index** is a secondary index the server maintains for you. `FT.CREATE` takes a name and three decisions: - **Which data type it follows:** `ON HASH`, the default, or `ON JSON`. - **Which keys it follows:** `PREFIX 1 product:` restricts it to keys that start with `product:`. Without `PREFIX`, it follows every key of that type. - **Which fields it indexes:** everything after `SCHEMA`, each with a type and options. The index does not hold your documents. The hashes stay ordinary hashes, readable with `HGETALL`, and only declared fields feed the index structures. A hash field left out of `SCHEMA` cannot be queried, although `FT.SEARCH` can still return it from the hash. `FT.CREATE` replies `OK`, and it works only on database 0: anywhere else the reply is the error `Cannot create index on db != 0`. ## How keys reach the index 1. **Existing keys** are scanned in the background after `FT.CREATE` returns, so queries made meanwhile see only part of the data. 2. **New and modified keys** are indexed synchronously, so the change is searchable by the time the write replies. 3. **Deleted, expired and evicted keys** leave the index without any call from the application. That is the practical difference from a lookup built by hand: the application keeps using `HSET`, `DEL` and `EXPIRE`, and there is no second write to get wrong. ## Choosing a field type | Type | What is indexed | Query form | Typical catalogue field | |---|---|---|---| | `TEXT` | Tokenized, lower-cased, stemmed terms | `@title:wireless` | Title, description | | `TAG` | Whole values, split only on a separator | `@category:{laptops}` | Category, brand, status, SKU | | `NUMERIC` | Integers and floats | `@price:[100 500]` | Price, rating, timestamp | | `GEO` | A `lon,lat` string | `@location:[-122.41 37.77 5 km]` | Store position | | `VECTOR` | An embedding | A KNN clause | Similarity search | ## TAG against TEXT Ask how the value will be matched. A shopper filtering by category wants the exact label; a shopper typing into a search box wants a word inside a longer string. - **`TAG` keeps the value whole.** It splits only on `SEPARATOR`, a comma by default, and lower-cases unless you add `CASESENSITIVE`. There is no stemming. - **`TAG` is cheap.** The docs put it at about one or two bytes per entry, because it stores document IDs without frequencies or positions. - **`TEXT` breaks the value into terms and stems them.** A query for `laptops` also matches `laptop`, which is what you want in a description and not in a category. - **`TAG` is invisible to a bare query.** `FT.SEARCH idx:products laptops` searches `TEXT` fields. A tag needs `@category:{laptops}`. - **Punctuation in a tag query needs escaping.** A hyphen in `in-stock` takes a backslash inside the braces. One hash field can be indexed both ways with aliases: `sku AS sku_text TEXT sku AS sku_tag TAG SORTABLE`. `NUMERIC` and `GEO` are simpler choices. Numeric ranges are inclusive, and a `(` makes a bound exclusive, as in `@price:[(100 500]`. A `GEO` value is a string with longitude first, separated from latitude by a comma. It belongs to this index and has nothing to do with keys written by `GEOADD`. On JSON documents the same types apply, and the identifier is a JSONPath with an alias, such as `$.title AS title TEXT`. ## SORTABLE and other options `SORTABLE` keeps the value available to `SORTBY` and to `FT.AGGREGATE` at very low latency, and adds memory overhead, so the docs advise against it on large text. You can sort a field without it, more slowly. On hashes `SORTABLE` normalizes the value (lower case, no diacritics) unless you add `UNF`. `WEIGHT 5` on a `TEXT` field multiplies that field's importance when scoring. `NOINDEX` together with `SORTABLE` makes a field sortable but not searchable. ## Changing the schema `FT.ALTER` can only add fields. To remove a field or change its type, create a new index, wait for it to fill, and move an alias with `FT.ALIASUPDATE` so queries switch without an application change. `FT.DROPINDEX` removes the index and leaves the hashes in place unless you pass `DD`. ## Which release has it Since Redis 8.0 the query engine is part of Redis Open Source and ships in all binary distributions. From a source build, the modules load from `redis-full.conf` or a release tarball's `redis.conf`, so a plain build with no module config has no `FT.*` commands. On 7.x it was the separate RediSearch module.

  • A field's type is wrong in production. Can you change it in place on a Redis query-engine index?
    No. `FT.ALTER` only adds fields, and it cannot remove or modify one. Create a new index with the corrected schema, let its background scan finish, then point the alias at it with `FT.ALIASUPDATE`. Applications that query through the alias switch with no code change, and the old index can then go with `FT.DROPINDEX`, which keeps the hashes unless you pass `DD`.
  • Why would you index one Redis hash field as both TAG and TEXT?
    Because the same value is sometimes matched exactly and sometimes searched inside. A SKU is the usual case: `sku AS sku_tag TAG` serves exact lookups, and `sku AS sku_text TEXT` lets a bare search find it. You pay for both structures, so do it only when both query shapes exist.
  • What is the PREFIX trap when a Redis index follows product: keys?
    Prefix matching is literal, and the key's type is checked as well. `PREFIX 1 product:` does not match `products:1`, and a string key at `product:7` is skipped by an `ON HASH` index. Neither case raises an error or increments a failure counter, so the only sign is a `num_docs` lower than you expected.

An index definition is a standing instruction to a librarian: catalogue every book on shelves labelled product, by title and category. Books already shelved get catalogued gradually in the background; a new book is catalogued as it is shelved.

  • FT.CREATE copies each hash into a separate search store.
  • The application has to write to the index after every HSET.
  • TEXT and TAG differ only in how much memory they use.
  • A bare FT.SEARCH term also finds values held in TAG fields.
  • Leaving out PREFIX indexes nothing until a prefix is added.
  • FT.DROPINDEX deletes the indexed hashes by default.

Assumes Redis 8.10; the query engine has been part of Redis Open Source since 8.0, and was the separate RediSearch module on 7.x.

open as a page

In Redis, when do you use FT.AGGREGATE rather than FT.SEARCH, and what makes an aggregation pipeline expensive?

level: middleimportance: should knowfreq 36%

basics

~20 s

FT.SEARCH returns matching documents, ranked or sorted. FT.AGGREGATE takes the same query and runs the matches through GROUPBY, REDUCE, APPLY and SORTBY steps. Pipelines get expensive when they read hashes with LOAD or sort every record.

open as a page

A Redis query-engine index is outgrowing its node: how do you read its cost in FT.INFO, and what changes when you move to cluster mode?

level: seniorimportance: should knowfreq 28%

basics

~20 s

FT.INFO splits index memory into fields such as inverted_sz_mb, sortable_values_size_mb and vector_index_sz_mb. Trim unused fields and options first. In cluster mode each shard indexes its own keys, and every query fans out to all shards and merges.

open as a page

After FT.CREATE over millions of existing Redis hashes, FT.SEARCH still misses products: how do you work out why?

level: seniorimportance: should knowfreq 31%

basics

~20 s

Start with FT.INFO. Check indexing and percent_indexed for an unfinished background scan, hash_indexing_failures for rejected documents, and an OOM status for a scan that stopped. Then check the prefix, key type and SKIPINITIALSCAN, which fail silently, and finally the query.

open as a page

As the lead, would you keep product search in Redis's query engine or run a dedicated search engine, and what evidence decides it?

level: principalimportance: should knowfreq 33%

basics

~20 s

Keep search in Redis when the documents live there durably, the corpus fits in RAM affordably, and its scorers and weights satisfy relevance. Move out when RAM cost, relevance needs or restart behaviour stop fitting, judged on your own measurements.

open as a page

In a Redis query-engine index, how do you add similarity search over hashes, and when do you use FT.HYBRID instead of an FT.SEARCH KNN query?

level: middleimportance: nice to knowfreq 27%

basics

~20 s

Declare a VECTOR field in FT.CREATE and store each embedding as raw bytes in the hash. FT.SEARCH KNN ranks by vector distance, with any filter only restricting candidates. FT.HYBRID, since Redis 8.4, also scores a text search and fuses both rankings.

open as a page