Would you adopt Redis's beta vector set type for a product feature, or use a query-engine index or a dedicated vector store, and how would you decide?
answer
- what beta actually promises
- can the data be rebuilt
- one key, one slot
- memory per element, measured
- filters decide more than speed
basics
~20 sAdopt it for bounded, rebuildable similarity jobs on Redis you already run, keeping source embeddings elsewhere. Choose the query engine when filters, text or documents matter, and a dedicated store when the data outgrows what you will hold in RAM.
solid answer
~50 sI would ask what losing or rebuilding this data costs, then look at scale and query shape. The label is a real term: the 8.0 notes said the API may change or break, and the 8.10 README still says beta. So I would use it where the set is rebuildable, such as a semantic cache or routing table, on Redis the team already operates, with embeddings kept in a system of record. If queries need selective filters, text or per-document expiry, the query engine fits better. If the data is the product's primary retrieval corpus and exceeds what we will hold in memory, at roughly a kilobyte per element at 300 dimensions and more at higher ones, I would price a dedicated vector store. Whichever I pick, I would pin the version, test upgrades and keep an exit path.
go deeper
Know that the type is labelled beta and that Redis also has a query engine that does vector search, so a choice exists at all.
Explain the practical limits behind the choice: the single key in a single slot, data held in RAM, and filters tested during the search with no index.
Bring numbers: memory per element from MEMORY USAGE, the gap between reads and writes in throughput, and what the AOF and replication setups do with this type.
Frame it as reversibility. A beta type is acceptable where the data can be rebuilt and an exit path is cheap, so make those two things true before you commit.
## What the beta label actually says A **beta** label is a statement about the contract. Redis's own record: - The 8.0-RC1 release notes say of the type: "We may change, or even break, the features and the API in future versions." - The 8.10.3 README still lists "Vector set (beta)", although the documentation pages for the type carry no such label, so a reader of the docs alone would miss it. - The surface has kept moving: `WITHATTRIBS` arrived in 8.2 and `VRANGE` in 8.4. - The 8.10.1 and 8.10.2 notes list security fixes for the type, covering a use-after-free between `VREM` and background `VSIM` threads and a crash on deeply nested JSON. Fixes are a good sign of attention, and also evidence that the code is still maturing. None of this rules the type out. It tells you what you are signing up for: pinned versions, an upgrade test and a way out. ## The limits that drive the decision | Concern | Vector set | Query-engine index | Dedicated vector store | |---|---|---|---| | Contract | Beta per README | No beta label | Depends on the product | | New system to run | None if on Redis 8 | None, with modules loaded | Yes | | Selective filters | Weak, capped walk | Real indexes on fields | Evaluate for your filters | | Data shape | Name, vector, small JSON | Hash or JSON documents | Evaluate for your data | | Beyond one node | Split keys yourself | Engine fans out | Evaluate at your scale | Three of these deserve numbers before the meeting. - **Memory.** Everything is in RAM. The module README measured 3 million 300-dimension `Q8` elements with short JSON attributes at about 3.1 GB, near 1 KB each, of which only 300 bytes are the vector. As a rough projection, 10 million 1,536-dimension elements would take about 23 GB. Confirm with `MEMORY USAGE` on a sample. - **Cluster.** Every vector set command names one key, and a key lives in one slot on one node, so one set cannot span shards. The documented pattern is to hash elements across several keys and query all of them. Per the docs, writes scale nearly linearly and reads do not, since each query must visit every key, and the client merges results with `WITHSCORES`. - **Throughput.** The docs expect around 50,000 `VSIM` per second on a 3-million-element set with 300 dimensions on one host, scaling with cores up to 32 threads, but only a few thousand `VADD` per second. Check whether your workload is write-heavy before believing the headline. Operations matter too. An RDB file holds the graph, so a 3-million-element set loads in about 15 seconds with no rebuild. On the other side, with `aof-use-rdb-preamble no` the command `BGREWRITEAOF` replies that the rewrite started, and then the background rewrite fails, as the type has no AOF rewrite callback. The default of `yes` avoids that. ## How I would decide 1. **Can the data be rebuilt from elsewhere?** If not, I would not make a beta type its only home. 2. **What do queries look like?** Selective filters, text or aggregation point to the query engine, whatever the rest of the answers are. 3. **What is the size in a year?** If it fits one node, or a few keys you are willing to fan out across, the type stays viable. If not, compare with a dedicated store. 4. **What does the team already run?** A team running Redis 8 adds no new system with a vector set or a query-engine index. A dedicated store is one more thing to operate, secure and upgrade, which it must earn. 5. **Who owns the upgrade?** Someone must read the release notes for each Redis upgrade while the label stands. A good first use is therefore a semantic cache, semantic routing or 'related items' on a bounded collection. Losing it costs money and latency, not truth. ## Lowering the risk if you adopt - **Keep source embeddings outside Redis.** `VEMB` returns approximations under quantization, so the set cannot rebuild itself. - **Wrap the commands.** A small internal interface around `VADD` and `VSIM` keeps a later move to an index or another store cheap. - **Measure recall.** Schedule `VSIM ... TRUTH` comparisons on a sample, since a silent recall drop will not appear in any error rate. - **Rehearse an upgrade.** Replay the suite on the next Redis version, including replication, because replies and options can change. - **Split early if you will split at all.** Moving from one key to several later is a rebuild, so choose the partition scheme before the data grows.
- What would you test on replicas before relying on a vector set in production?Fail over and compare results with `VCARD` and `VSIM` on both sides, including updates. In 8.10.3's `vset.c`, `VADD` calls `RedisModule_ReplicateVerbatim` only when a new element was added, so by the source a changed vector on an existing element does not reach replicas or the AOF. Verify that, and if it holds, write updates as `VREM` then `VADD`.
- Why not start on the query engine and skip this decision?Often that is the right default, especially if the data is already in hashes or JSON. The vector set earns its place when there is no document to index, and a schema, prefix and per-document keys would be overhead for a small similarity job. If you expect filters or text later, starting on the index avoids a migration.
- How would you grow a vector set past one node on Redis Cluster?Hash each element to one of N keys, such as `crc32(item) % N`, so keys land on different slots. Send `VSIM ... WITHSCORES` to all of them in parallel and merge by score in the client. Latency becomes that of the slowest key. Choose N generously up front, because changing it later means rebuilding.
- Beta only means new; the API is guaranteed stable since 8.0.
- Adding more keys and shards multiplies a vector set's query throughput.
- Redis Cluster spreads one vector set across slots by itself.
- Shipping in the Redis binary means adopting the type has no operational cost.
- Vector sets and query-engine indexes share storage, so switching is a setting change.
Assumes Redis 8.10.3; the beta label, release-note fixes, AOF behaviour and replication path were read from its README, 00-RELEASENOTES, aof.c and vset.c.