Training and Recall Tuning

## What "training" means in FAISS FAISS indexes fall into two groups. Some are pure containers: `IndexFlatL2` and `IndexFlatIP` store raw float32 vectors and compare a query against every one of them, and `IndexHNSWFlat` builds its graph incrementally as vectors arrive. These report `is_trained == True` from the moment they are constructed, and you can call `add()` immediately. The other group is *data-dependent*. An inverted-file index (`IndexIVFFlat`, `IndexIVFPQ`) does not search all vectors; it first partitions the vector space into `nlist` cells and stores each vector in the cell whose centroid is nearest. Those centroids are learned by running k-means over a training set. Product quantization goes further: it splits each vector into `m` sub-vectors and learns a codebook of 2^nbits centroids for each slice, so that a vector can later be stored as `m` small integer codes instead of floats. Neither the cell centroids nor the codebooks can be invented — they are fitted parameters, and `train()` is the fitting step. ## What happens if you skip it `index.is_trained` is False until `train()` returns. Calling `add()` on an untrained IVF index raises a RuntimeError from FAISS's internal assertion — it does not silently train itself, and it does not silently degrade. This is the single most common first-hour FAISS error, and the fix is always the same: train, then add. ``` index = faiss.IndexIVFFlat(faiss.IndexFlatL2(d), d, nlist) index.is_trained # False index.train(xt) index.add(xb) ``` ## How much training data FAISS's k-means implementation has two internal bounds that drive the practical answer. It expects at least about 39 points per centroid and prints a warning like *"WARNING clustering N points to K centroids: please provide at least ... training points"* when you give it fewer. Above about 256 points per centroid it randomly subsamples, so a training set larger than `256 * nlist` gives you no better centroids and only costs time. Product quantization has a parallel requirement: each sub-quantizer learns 2^nbits centroids, so with the default nbits=8 you want at least a few tens of thousands of training vectors before the codebooks mean anything. That gives a simple sizing rule. Pick `nlist` first (commonly on the order of sqrt(N) for N vectors), then take a random sample of roughly 40–256 times `nlist` vectors to train on. For a million vectors with nlist=1024, something like 100k–260k training vectors is plenty; training on the full million is wasted work. ## Representativeness beats volume The centroids define where every future vector and every future query will be routed. If the training sample is drawn from a different distribution than the corpus — one language when production is multilingual, one product category when the catalogue is broad, or synthetic random vectors during a prototype — the cells are unbalanced. Some cells hold a huge share of the corpus and others are nearly empty, so scanning `nprobe` cells scans far more or far less than the intended fraction, and queries frequently miss neighbours that sit just over a badly-placed boundary. The symptom is recall that is poor at every `nprobe` value, which is the tell that the problem is the training set rather than the search parameter. A random subsample of the real corpus is the right default. If your data has known strata, sampling proportionally across them protects against a skewed draw. ## Training is a build-time artifact Because training is expensive and deterministic given its input, the normal production pattern is: train once, add the corpus, `faiss.write_index(...)`, and ship the file. Serving processes only read it. You cannot "retrain in place" — changing the centroids invalidates every code and every cell assignment already stored, so a retrain means building a new index from scratch and swapping it in. Plan for that when the embedding model changes or when the corpus drifts substantially, and treat the trained index as an immutable build artifact tied to a specific model version. ## Failure modes to recognise - `add()` raising because `is_trained` is False — train first. - A clustering warning at build time — your training sample is too small for `nlist`; either sample more or lower `nlist`. - Uniformly poor recall across all `nprobe` values — suspect unrepresentative training data or an `nlist` far too large for the corpus size. - Someone "fixing" drift by training on new data and adding to the existing index — that is a rebuild, not an incremental update.