skip to content

In a Redis query-engine index, how do you add similarity search over hashes, and when do you use FT.HYBRID instead of an FT.SEARCH KNN query?

level: middleimportance: nice to knowfreq 27%

answer

  1. a schema field with an algorithm
  2. raw bytes in a hash field
  3. a filter restricts, it does not rank
  4. two legs, then COMBINE

basics

~20 s

Declare a VECTOR field in FT.CREATE and store each embedding as raw bytes in the hash. FT.SEARCH KNN ranks by vector distance, with any filter only restricting candidates. FT.HYBRID, since Redis 8.4, also scores a text search and fuses both rankings.

solid answer

~40 s

I add a field such as `embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 768 DISTANCE_METRIC COSINE` to the schema, where 6 counts the attribute tokens that follow, and write each vector as raw bytes with `HSET`. A blob of the wrong size is an indexing failure. A KNN query in `FT.SEARCH` needs `DIALECT 2`, because the default is still dialect 1. Its filter narrows the candidates and the ranking comes from distance alone, which fits pure similarity inside a category. When words matter too, such as a model number in the search box, I use `FT.HYBRID`, available since Redis 8.4. It runs a `SEARCH` leg and a `VSIM` leg and fuses them with `COMBINE RRF` by default, or `LINEAR` with weights. It returns only keys and scores unless I add `LOAD`.

code

plaintext · 3 lines
plaintext
FT.CREATE idx:products ON HASH PREFIX 1 product: SCHEMA title TEXT category TAG price NUMERIC embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 768 DISTANCE_METRIC COSINE

FT.HYBRID idx:products SEARCH "wireless headphones" VSIM @embedding $vec KNN 2 K 20 FILTER "@category:{headphones}" COMBINE RRF 2 WINDOW 40 LOAD 2 @title @price LIMIT 0 10 PARAMS 2 vec <768 FLOAT32 values as bytes>

go deeper

for a junior

Know that a VECTOR field sits in the FT.CREATE schema with an algorithm, TYPE, DIM and DISTANCE_METRIC, and that a KNN query passes the vector through PARAMS.

for a middle

Explain the token count, the raw bytes in the hash and why DIALECT 2 is needed, then say what a filter does to a KNN query and what FT.HYBRID adds.

for a senior

Trace failure modes: a dimension change that drops whole documents, filters that apply to one leg only, and the memory a vector field adds as seen in vector_index_sz_mb.

for a principal

Weigh whether vector search shares an instance with the rest of the catalogue, since embeddings can outweigh everything else, and how a model change gets rolled out as a new index.

## Declaring a vector field A `VECTOR` field is one more entry in the `FT.CREATE` schema, so a catalogue can filter, search text and compare embeddings through a single index: `embedding VECTOR HNSW 6 TYPE FLOAT32 DIM 768 DISTANCE_METRIC COSINE` - **The number after the algorithm counts tokens**, so three name and value pairs make 6. Writing 3 is a common syntax mistake. - **`TYPE`** is the element type, such as `FLOAT32` or `FLOAT16`. - **`DIM`** is the vector width. The query vector must have exactly the same dimensions. - **`DISTANCE_METRIC`** is `L2`, `IP` or `COSINE`. ## Storing vectors in a hash A hash value is a binary-safe string, so you `HSET` the vector's underlying bytes. The size has to equal `DIM` times the element width. Anything else fails with `Could not add vector with blob size ...`, and that failure costs the whole document its place in the index, along with its text and tag fields. It shows up in `hash_indexing_failures` in `FT.INFO`. ## Choosing the algorithm The choice belongs to the schema. The theory behind each is a separate subject; these are the Redis-level differences. | Algorithm | Redis docs' guidance | Worth knowing | |---|---|---| | `FLAT` | Small datasets, under about 1M vectors, or when perfect accuracy matters | Brute force | | `HNSW` | Larger datasets where latency and scale matter more than perfect accuracy | Optional `M`, `EF_CONSTRUCTION`, `EF_RUNTIME` | | `SVS-VAMANA` | Graph-based, built to pair with compression | `COMPRESSION` options; a stock Redis Open Source build falls back to basic 8-bit scalar quantization | ## KNN through FT.SEARCH A KNN query lives inside the query string and passes the vector through `PARAMS`: `FT.SEARCH idx:products "(@category:{headphones})=>[KNN 10 @embedding $vec AS dist]" PARAMS 2 vec <blob> SORTBY dist DIALECT 2` Three things to get right: 1. **`DIALECT 2` is required.** `search-default-dialect` is still 1, so vector queries must ask for it unless the server default has been changed. 2. **The part before the arrow only restricts.** Words placed there narrow the candidates; they do not contribute to the ranking. 3. **Results rank by distance.** This fits queries that are purely about similarity, such as products like this one within a category. ## Choosing between the two - **Similarity inside a filter:** products like this one, in this category, under this price. `FT.SEARCH` KNN answers it, and the distance is the only ranking you want. - **Words and meaning together:** a search box where shoppers type both model numbers and loose descriptions. `FT.HYBRID` lets either signal lift a product. - **Before Redis 8.4:** only the `FT.SEARCH` form exists, so an upgrade decision comes first. ## FT.HYBRID `FT.HYBRID`, available since Redis 8.4, runs two legs and fuses their results: - **`SEARCH`** takes the usual query syntax and scores with `BM25STD` by default. - **`VSIM`** names the vector field and a parameter, with `KNN` or `RANGE`. `K` defaults to 10. - **`COMBINE`** defaults to `RRF` with `WINDOW 20` and `CONSTANT 60`. `LINEAR` takes `ALPHA` and `BETA` weights. - **After fusion** you can `GROUPBY`, `APPLY` and `SORTBY`. `LIMIT` defaults to 10. Reach for it when a shopper's words and their meaning both carry signal, so that an exact model number and a loose description should each be able to lift a product. The traps are specific to the command: - **No `LOAD`, no fields.** It returns keys and scores unless you add `LOAD`. - **No `DIALECT` argument.** The parser rejects one. - **Filters are per leg.** A `FILTER` inside `VSIM` pre-filters the vector candidates without affecting scores, and the `SEARCH` expression governs only the text leg. A category limit written only in `SEARCH` lets the vector leg bring in other categories. - **`EXPLAINSCORE`**, in Redis 8.10 and later, shows how each combined score arose, and it cannot be combined with `GROUPBY`. ## Costs Vectors are the heavy part of a catalogue index. `FT.INFO` reports them in `vector_index_sz_mb`. At `DIM 768` and `FLOAT32`, one raw vector is about 3 KB, and the hash holds its own copy of the bytes beside what the vector index keeps. In cluster mode, `$SHARD_K_RATIO` in `FT.SEARCH` and `SHARD_K_RATIO` in `FT.HYBRID` ask each shard for fewer candidates (0.1 to 1.0, default 1.0), trading accuracy for speed, and unbalanced shards can then return fewer than `K` results.

  • A Redis FT.SEARCH KNN query is rejected on one server, yet the same text runs on another. What do you check first?
    The dialect. Vector search requires `DIALECT 2`, and `search-default-dialect` is 1 on a stock server, so the other server may have had its default changed. Compare the setting with `CONFIG GET search-default-dialect`. Then append `DIALECT 2` to the command, which makes the query independent of either server's default.
  • A batch of Redis product hashes disappears from text searches right after a new embedding model ships. Why?
    The new model probably changed the vector width. A blob that does not match the field's `DIM` fails with a blob-size error, and the document as a whole is not indexed, text fields included. `hash_indexing_failures` and `last indexing error` in `FT.INFO` confirm it. A new width needs a new index, moved into place with an alias.

  • Words in front of the KNN arrow raise a document's rank.
  • A vector goes into a Redis hash as a comma-separated string.
  • The count after HNSW in FT.CREATE is the number of attribute pairs.
  • FT.HYBRID returns all hash fields by default.
  • A category in the SEARCH leg of FT.HYBRID also restricts the vector leg.
  • A wrong-sized vector only loses that one field from the index.

Assumes Redis 8.10: FT.HYBRID exists since 8.4 and EXPLAINSCORE since 8.10; on earlier releases only the FT.SEARCH KNN form is available.