When would you choose FAISS IndexHNSWFlat over IndexIVFFlat?

## Two different bets Both indexes are approximate and both keep vectors uncompressed, so this is not a memory-versus-accuracy question — it is a question about the shape of the workload. HNSW builds a navigable multi-layer graph over the vectors and answers a query by walking it. IVF partitions the space into `nlist` cells with a coarse quantizer and answers a query by scanning the cells nearest to it. Everything that differs downstream follows from those two structures. ## Training `IndexIVFFlat` needs `train()` before `add()`, because its centroids are learned by clustering a sample. That imposes a pipeline: collect a representative sample, train, then ingest. If the sample is skewed — one tenant, one language, one week — the centroids fit that slice and everything else clusters badly, and the resulting recall loss is invisible until someone measures it. `IndexHNSWFlat` has nothing to learn. `is_trained` is True on construction and you can add vectors from an empty state. For a system that starts with no data and grows, or one where you cannot easily obtain a representative sample up front, that is a genuine operational simplification. ## Memory HNSW is the more expensive of the two per vector. It stores the raw float32 vectors, exactly as flat does, and additionally the graph adjacency: on the order of `M * 2 * 4` bytes per vector for the base layer at connectivity `M`, plus the sparse upper layers. At d=768 and M=32 that is roughly 3072 bytes of vector plus a few hundred bytes of links. IVF stores the raw vectors, an int64 id per entry, and a centroid table of `nlist * d * 4` bytes. Per vector that is essentially the flat cost plus 8 bytes. So on the same corpus HNSW is meaningfully larger, and it does not compress — which is why very large corpora end up on IVF with a compressed encoding rather than on HNSW at all. ## Build time and ingestion Inserting into HNSW means searching the existing graph to find neighbours and then wiring links, for every vector. That is far more work than IVF's "find the nearest centroid, append to that list", and it does not batch as cleanly. Building an HNSW over tens of millions of vectors is measured in hours; the same corpus into a trained IVF index is much faster. The construction quality knob, `index.hnsw.efConstruction`, trades build time for graph quality — raising it makes both the build slower and the resulting recall better. ## Deletions and churn This is the axis people discover too late. `IndexIVFFlat` implements `remove_ids`, so an entry can be dropped from its inverted list. FAISS's HNSW implementation does not support removal — pulling a node out of the graph would strand its neighbours and break navigability. If your corpus has real deletes, HNSW means either periodic full rebuilds with an atomic swap, or maintaining a tombstone set in the application and over-fetching so that filtered results still fill the top-k. Both are workable, both are work, and neither should be discovered in production. ## Latency profile HNSW's strength is single-query latency at high recall. Graph traversal touches a small, adaptive number of vectors and reaches high recall with less work than IVF typically needs, especially for one query at a time — which is the shape of an interactive search or RAG request. IVF's scan is embarrassingly parallel and batches beautifully, so for large batch workloads on many cores the gap narrows or reverses. Both expose a knob that trades accuracy for speed — `index.hnsw.efSearch` for the graph, `index.nprobe` for the lists — and the honest answer is that you set it by measuring recall against a flat ground truth for your data, not by copying a number from a benchmark on someone else's. ## They compose The interesting production answer is that this is not always an either/or. Once `nlist` is large, IVF's coarse quantizer — which by default brute-forces over all centroids — becomes a bottleneck, and the standard fix is to make the quantizer itself an HNSW index: `faiss.index_factory(d, "IVF65536_HNSW32,Flat")`. Here HNSW is not the storage structure; it is the routing structure over 65536 centroids. Recognising that composition is a strong signal, because it shows you understand what each structure is actually for. ## A decision sketch - Corpus under a few million, fits in RAM, near-static, strict p99 on single queries: **HNSW**. - Corpus large, or memory-tight, or you will need compression later: **IVF**, with a `Flat` encoding while it fits and a compressed encoding when it stops fitting. - Frequent deletions: **IVF**, or accept a rebuild pipeline. - No representative training sample available at day zero: **HNSW**, or start flat and migrate. - Very large `nlist`: **both**, with HNSW as the coarse quantizer.