skip to content

FAISS

5 roadmaps16 questions

Meta's similarity-search library — not a database, but the index implementations underneath many of them: flat, IVF, HNSW, and product quantization, with GPU support. Interviewers use it to test whether you can reason about index choice, recall, and memory instead of calling someone else's API.

on this pageshow

explore

questions

16

In FAISS, what does IndexFlatL2 do and when is it the right index?

level: juniorimportance: must knowfreq 78%

answer

  1. No training, no compression
  2. Every vector compared, every query
  3. Four bytes per dimension per vector
  4. Squared L2, not L2
  5. Ground truth for measuring recall

basics

~20 s

IndexFlatL2 stores every vector uncompressed and compares a query against all of them, returning exact L2 nearest neighbours with no training step. It is the right choice for small collections and as a recall baseline, but its cost grows linearly with the dataset.

solid answer

~50 s

`faiss.IndexFlatL2(d)` is brute force: it keeps the raw float32 vectors and scans all of them per query, so results are the true nearest neighbours — recall is 100% by construction. `is_trained` is already True, you just call `add(xb)` then `search(xq, k)`, which returns a distance matrix and an id matrix. Two consequences matter in practice. Memory is exactly `4 * d` bytes per vector, so 1M 768-dim vectors need about 3 GB. Query time is O(N*d), which is fine for tens or hundreds of thousands of vectors but not for tens of millions. `IndexFlatIP` is the inner-product twin; combined with `faiss.normalize_L2` on both the stored vectors and the query it gives cosine similarity. Flat indexes also serve as the coarse quantizer inside IVF indexes and as the ground truth you measure an approximate index's recall against.

code

python · 14 lines
python
import numpy as np
import faiss

d = 128
xb = np.random.random((10000, d)).astype('float32')
xq = np.random.random((5, d)).astype('float32')

index = faiss.IndexFlatL2(d)
print(index.is_trained)   # True - nothing to train
index.add(xb)
print(index.ntotal)       # 10000

distances, ids = index.search(xq, 4)
print(distances[0])       # squared L2 distances

go deeper

for a junior

Know that a flat index scans everything and returns exact neighbours, that it needs no training, and that you call add() then search() to get distances and ids back.

for a middle

Be ready to compute its memory as 4 bytes per dimension per vector, explain that returned L2 values are squared, and show the normalize_L2 plus inner-product recipe for cosine.

for a senior

Show judgment about the crossover point: quote the memory and latency arithmetic for your corpus, and explain that you keep a flat index as ground truth to measure any approximate index's recall.

for a principal

Own the framing that exactness is a product decision, not a default. Argue when a small corpus should stay flat forever to avoid a training pipeline and a recall regression nobody is measuring.

## What a flat index is FAISS calls an index "flat" when it stores the vectors verbatim and answers a query by comparing it against every stored vector. There is no graph, no partitioning, no compression — just a contiguous array of float32 values and a loop over it, heavily vectorised with SIMD and multithreaded with OpenMP. `IndexFlatL2` computes squared Euclidean distance; `IndexFlatIP` computes the inner (dot) product and returns the largest values first. ## The API shape ``` index = faiss.IndexFlatL2(d) index.is_trained # True index.add(xb) # xb is an (n, d) float32 C-contiguous numpy array index.ntotal # n D, I = index.search(xq, k) ``` Three details trip people up. First, FAISS is strict about dtype and layout: vectors must be `float32` and C-contiguous, or the call fails. Second, `is_trained` is True from the start — flat indexes have nothing to learn, unlike IVF or PQ variants which need a `train()` pass before any `add()`. Third, `D` from `IndexFlatL2` holds **squared** L2 distances, not distances; if you display them to users or threshold on them, take the square root yourself. `search` returns two arrays of shape `(nq, k)`: `I` holds the ids of the neighbours and `D` the corresponding scores. Ids are sequential integers assigned in insertion order, starting at 0 — the flat index has no notion of your application's identifiers. When fewer than `k` results exist, the missing slots come back as id `-1`. ## Metrics and cosine FAISS ships L2 and inner product as the two primary metrics. There is no dedicated cosine index, because cosine similarity is exactly the inner product of unit-length vectors. The standard recipe is to L2-normalise vectors in place with `faiss.normalize_L2(x)` before adding them, use `IndexFlatIP`, and normalise every query the same way. Forgetting to normalise the query is a common bug: results still come back, they are just ranked by raw dot product, which favours long vectors. ## What it costs Memory is trivial to compute and impossible to reduce: `4 * d` bytes per vector, plus a small object overhead. Some concrete points on that curve, for 768-dimensional embeddings: 100k vectors ≈ 0.3 GB, 1M ≈ 3 GB, 10M ≈ 30 GB, 100M ≈ 300 GB. That arithmetic is the single most useful thing to be able to do on the spot, because it tells you immediately whether a flat index is even on the table. Latency scales with `N * d` multiply-adds per query. On a modern multi-core CPU, a few hundred thousand 768-dim vectors is still single-digit milliseconds per query, and batching queries (passing many rows in one `search` call) amortises far better than looping one query at a time. Past a few million vectors, per-query latency stops being acceptable for interactive search and you move to a partitioned or graph index. ## When flat is genuinely the right answer - **Small corpora.** Below roughly 100k–1M vectors, an approximate index buys you latency you did not need while costing recall and a training step. Many production RAG corpora never leave this range. - **Recall measurement.** To know an approximate index's recall you need ground truth, and the only cheap source is a flat index over the same data. Build one on a sample, compare the top-k id sets, and you have a recall@k number instead of a guess. - **As a component.** IVF indexes need a coarse quantizer to assign vectors to lists, and that quantizer is normally an `IndexFlatL2`. Flat is also the refinement stage behind compressed indexes when exact re-ranking is wanted. - **Rapidly changing data.** Flat supports cheap appends and, when wrapped for ids, deletions, without any retraining or graph repair. ## Where it stops working The failure is not subtle — it is memory first and latency second. A team that starts flat and grows past its RAM budget must switch index type, and the switch is not free: partitioned indexes need training data, compressed indexes need a divisibility-compatible dimension, and both give up exactness. Knowing the flat baseline's numbers is what tells you when that moment arrives, and the honest interview answer starts with the arithmetic rather than with a product recommendation.

  • How do you get cosine similarity out of FAISS when there is no cosine index?
    Normalise both the stored vectors and the queries to unit length with `faiss.normalize_L2`, then use `IndexFlatIP` (or any inner-product index). Cosine is the dot product of unit vectors, so the ranking is identical. The usual bug is normalising the corpus but forgetting the query, which silently reverts the ranking to raw dot product and biases results toward long vectors.
  • Why would you keep a flat index around after deploying an approximate one?
    To measure recall. An approximate index gives you no signal about what it missed, so you build a flat index over the same data — or a representative sample — run the same queries through both, and compare the top-k id sets. That ratio is recall@k, and without it any claim about a tuning change is guesswork.
  • Are the distances returned by IndexFlatL2 actual Euclidean distances?
    No — they are squared L2 distances. FAISS skips the square root because it does not change the ranking and costs time. Ranking and top-k selection are unaffected, but any absolute threshold you apply, or any number you show a user, must be square-rooted first. `IndexFlatIP` instead returns inner products, where larger is better.

It is the linear scan of vector search: reading every row of a table with no index — always correct, and fine until the table gets big.

saying these in an interview costs you the question

  • Says IndexFlatL2 must be trained before adding vectors
  • Treats the returned distances as true L2 rather than squared
  • Claims a flat index compresses or approximates vectors
  • Proposes a flat index for tens of millions of vectors
  • Expects add_with_ids to work on a plain flat index

context

resolving links…

open as a page

How do you move a FAISS index onto a GPU, and what does StandardGpuResources do?

level: middleimportance: must knowfreq 62%

basics

~20 s

Create a faiss.StandardGpuResources() object, then call faiss.index_cpu_to_gpu(res, device, index). The resources object owns the GPU scratch memory and cuBLAS handles for that device, and you must keep a Python reference to it for as long as the index lives.

open as a page

What does the FAISS index_factory string "IVF4096,PQ64" build?

level: middleimportance: must knowfreq 70%

basics

~20 s

It builds an IVFPQ index: a coarse quantizer that splits the space into 4096 inverted lists, with each vector stored as a product-quantized code of 64 one-byte values instead of the raw floats. Both stages must be trained before adding data.

open as a page

Why must a FAISS IVF index be trained before add(), and on what data?

level: middleimportance: must knowfreq 76%

basics

~20 s

IVF and PQ indexes learn structure from data — k-means centroids for IVF, codebooks for PQ. Until train() runs, is_trained is False and add() raises. Train on a representative sample of the vectors you will actually search.

open as a page

When do you shard a FAISS index across GPUs instead of replicating it?

level: seniorimportance: must knowfreq 52%

basics

~20 s

Replicate when the index fits on one GPU and you need more queries per second — each device holds a full copy and queries are split between them. Shard when the index does not fit on one GPU: each device holds a slice, every query touches all of them, and results are merged.

open as a page

Which FAISS index fits 20M 768-dim vectors in a few GB of RAM?

level: seniorimportance: must knowfreq 62%

basics

~20 s

A flat index would need about 61 GB, so the answer is a compressed IVFPQ index such as index_factory(768, "IVF16384,PQ96"): 96 code bytes plus an 8-byte id per vector is roughly 2 GB. The saving is bought with lossy distances and a mandatory training pass.

open as a page

FAISS IVF recall is too low in production — how do you tune nprobe and prove it worked?

level: seniorimportance: must knowfreq 68%

basics

~20 s

Raise nprobe — the number of cells each query scans. It is a search-time knob needing no rebuild. Sweep it while measuring recall@k against exact brute-force results, then pick the smallest value that hits your recall target within the latency budget.

open as a page

How do you persist a FAISS index with write_index and read_index, and what is not saved?

level: juniorimportance: should knowfreq 55%

basics

~20 s

faiss.write_index(index, path) serialises the whole index — trained centroids, codebooks, codes and ids — and faiss.read_index(path) restores it ready to search, with no retraining. What it never stores is your documents or metadata; FAISS keeps only vectors and 64-bit ids.

open as a page

Which FAISS index types have GPU implementations, and what happens when one does not?

level: middleimportance: should knowfreq 40%

basics

~20 s

FAISS implements a subset of its indexes on GPU — flat brute force, IVF with flat, scalar-quantized or product-quantized codes, and binary flat. Cloning a CPU index type with no GPU counterpart raises an error instead of silently falling back to CPU.

open as a page

Why is a FAISS GPU index barely faster than CPU when you search one query at a time?

level: middleimportance: should knowfreq 38%

basics

~20 s

A single query leaves the GPU almost idle: fixed per-call costs — transferring the query, launching kernels, copying results back — dominate the tiny amount of work. FAISS GPU indexes are throughput engines; you get the speedup by searching thousands of queries in one call.

open as a page

In FAISS, why wrap an index in IndexIDMap, and what does that change?

level: middleimportance: should knowfreq 52%

basics

~20 s

FAISS numbers vectors sequentially from zero in insertion order, so a flat index cannot return your application's identifiers. IndexIDMap wraps an index so add_with_ids attaches your own 64-bit integer ids, which search then returns instead of positions, at the cost of an extra id table in memory.

open as a page

In FAISS IndexIVFPQ, what do the m and nbits parameters control and cost?

level: middleimportance: should knowfreq 52%

basics

~20 s

m is how many sub-vectors each vector is split into and nbits the bits per sub-code, so a stored vector shrinks to about m*nbits/8 bytes. Raising either improves accuracy but costs memory, distance-computation time and training data.

open as a page

A FAISS GPU index OOMs though the vectors fit in device RAM. What is consuming memory?

level: seniorimportance: should knowfreq 42%

basics

~20 s

Vectors are only part of the budget. StandardGpuResources pre-reserves a scratch pool, ids cost 8 bytes each by default, IVFPQ precomputed tables scale with nlist, and each search allocates workspace proportional to the query batch. Shrink the pool, narrow ids, use half precision, or batch smaller.

open as a page

When would you choose FAISS IndexHNSWFlat over IndexIVFFlat?

level: seniorimportance: should knowfreq 48%

basics

~20 s

Choose IndexHNSWFlat for low-latency single queries on a mostly static, moderately sized corpus that fits in RAM, since it needs no training and gives strong recall per unit of latency. Choose IndexIVFFlat when memory matters, the corpus is large, or entries must be deleted.

open as a page

How do you decide whether a FAISS deployment justifies GPUs at all?

level: principalimportance: should knowfreq 28%

basics

~20 s

Decide from workload shape, not hardware envy. GPUs win where there is bulk parallel work — index building, clustering, batch scoring, high-QPS batched retrieval — and where the index fits in device memory. Low-QPS single-query serving on a tuned CPU index is usually cheaper and simpler.

open as a page

What does OPQ preprocessing buy a FAISS PQ index, and what does it cost?

level: seniorimportance: nice to knowfreq 28%

basics

~20 s

OPQ learns a rotation applied before product quantization so that variance is spread evenly across sub-vectors instead of concentrating in a few. That typically lifts recall at the same code size, paying longer training and an extra matrix multiply on every vector and query.

open as a page