skip to content

What does OPQ preprocessing buy a FAISS PQ index, and what does it cost?

level: seniorimportance: nice to knowfreq 28%

answer

  1. a learned rotation before slicing
  2. balances variance across sub-vectors
  3. distance-preserving, so nothing is lost
  4. longer training, extra matmul per query
  5. the index becomes a pre-transform wrapper

basics

~20 s

OPQ learns a rotation applied before product quantization so that variance is spread evenly across sub-vectors instead of concentrating in a few. That typically lifts recall at the same code size, paying longer training and an extra matrix multiply on every vector and query.

solid answer

~40 s

Product quantization slices a vector into contiguous chunks and quantizes each independently, which assumes every chunk carries a similar amount of information. Real embeddings violate that: variance is often concentrated in a handful of dimensions, so some sub-quantizers waste their codebook while others cannot represent their slice. OPQ fits an orthonormal rotation matrix — `faiss.OPQMatrix`, usually built through an `index_factory` string such as `"OPQ16_64,IVF1024,PQ16"` — that rotates the space so variance is balanced before slicing. Typical gain is a few points of recall at identical code size. The costs are a noticeably longer `train()`, since the rotation and codebooks are fitted alternately, and a d x d matrix multiply applied to every added vector and every query. The index also becomes an `IndexPreTransform`, so `nprobe` must be set through `faiss.extract_index_ivf` or `ParameterSpace`.

code

python · 14 lines
python
import faiss
import numpy as np

d = 128
xt = np.random.random((50_000, d)).astype('float32')

# OPQ16_64: rotate and project to 64 dims, then IVF1024 with a 16-byte PQ code
index = faiss.index_factory(d, "OPQ16_64,IVF1024,PQ16")
index.train(xt)      # fits rotation, coarse centroids and codebooks
index.add(xt)

# the outer object is an IndexPreTransform: no .nprobe of its own
faiss.ParameterSpace().set_index_parameter(index, "nprobe", 16)
D, I = index.search(xt[:5], 10)

go deeper

for a junior

Know that OPQ is an optional rotation learned before product quantization, and that it aims to make the compressed codes more accurate for the same number of bytes.

for a middle

Explain why contiguous slicing hurts when variance is unevenly spread, that the rotation is orthonormal so distances are preserved, and that it is normally added as a token in an index_factory string.

for a senior

Weigh it as an engineering tradeoff: extra training time and a per-query matrix multiply against a few points of recall, measured on your own data. Know that the wrapper changes how nprobe must be set in serving code.

for a principal

Decide whether the complexity belongs in the stack at all — whether the recall it buys is cheaper than more bytes per vector or a re-ranking stage, and whether the extra build step is worth its place in a pipeline other teams must operate.

## The problem OPQ solves Product quantization cuts a vector into `m` contiguous slices and gives each slice its own independent codebook. That design is only efficient if the slices are comparably informative. Embedding vectors rarely oblige: dimensions are correlated, variance is unevenly distributed, and concatenated features (say, two encoders' outputs joined end to end) put wildly different scales in different regions of the vector. The consequence is that some sub-quantizers spend 256 centroids describing a nearly-constant slice while others try to summarise a high-variance slice with the same budget. Total quantization error, and therefore distance error, is worse than it needs to be for the bytes spent. ## What OPQ actually is OPQ — optimized product quantization — learns an orthonormal matrix R and stores `R * x` instead of `x`. Because R is orthonormal, it preserves L2 distances exactly, so nothing about the geometry is lost; it merely re-orients the axes so that the fixed contiguous slicing PQ performs lands on slices of comparable variance. Training alternates two steps: fit the PQ codebooks given the current rotation, then re-solve for the rotation given the codebooks, repeated for a number of iterations. In FAISS the transform is `faiss.OPQMatrix(d, m)` and can optionally output a different dimension, written in factory syntax as `OPQ16_64` — 16 sub-vectors, output dimension 64. Building it through `faiss.index_factory(d, "OPQ16_64,IVF1024,PQ16")` wraps everything in an `IndexPreTransform`, and `train()` on the composite fits the rotation, the coarse quantizer and the PQ codebooks in the right order. The rotation is applied automatically to added vectors and to queries — you never apply it yourself, and applying it manually as well is a classic self-inflicted bug. ## What it costs - **Training time.** The alternating optimisation is materially slower than plain PQ training; on large training sets this can turn a minutes-long build step into a much longer one. It is a one-off build cost, which is why it is usually acceptable. - **Query time.** Every query goes through a d x d matrix multiply before search. For a 768-dim embedding that is roughly 590k multiply-adds per query — small next to scanning thousands of candidate vectors, but not free, and it shows up in tail latency for very small `nprobe` settings where the scan itself is cheap. - **Ingest time.** The same multiply applies to every vector added. - **Operational surface.** The outer object is an `IndexPreTransform`, which has no `nprobe` attribute. Code that does `index.nprobe = 32` will either fail or silently set an attribute nobody reads. Use `faiss.extract_index_ivf(index).nprobe = 32` or `faiss.ParameterSpace().set_index_parameter(index, "nprobe", 32)`, and make sure the serving code does this rather than assuming a bare IVF index. ## When it is worth it OPQ earns its place when the code is aggressive — small `m`, few bytes per vector — because that is where quantization error dominates and where a few points of recall are hard to buy any other way. It matters more when the embeddings are known to be anisotropic: unnormalised outputs, concatenated feature blocks, or vectors from a model that was never trained with quantization in mind. It matters less when the code is already generous, when you are re-ranking a shortlist against full-precision vectors anyway, or when the index is flat or IVFFlat, where no quantization happens and there is nothing to optimise. OPQ is also a convenient dimension reducer when the source dimension is inconvenient for PQ. Because `m` must divide the dimension PQ sees, an `OPQ16_64` step turns an awkward input dimension into a clean 64, which sidesteps the divisibility constraint and shrinks the per-query multiply at the same time — at the cost of discarding some variance in the projection. ## How to decide Treat it as an A/B on the build, not an article of faith. Build the index both ways with the same `m` and `nbits`, measure recall@k against exact ground truth on the same held-out queries, and measure query latency at your serving `nprobe`. If OPQ buys recall you would otherwise have to spend memory on, keep it; if the curves overlap, drop it and save the build time and the matrix multiply. On well-behaved, normalised embeddings the gain is sometimes small enough not to be worth the extra moving part.

  • Do you need to apply the OPQ rotation to query vectors yourself?
    No, and doing so is a bug. The rotation lives inside an IndexPreTransform, which applies it to every vector on add and to every query on search. Passing an already-rotated query means the transform rotates it a second time and the results are garbage that still looks superficially plausible — plausible distances with wrong ids is the signature of this mistake.
  • Why does OPQ preserve distances if it changes every vector?
    The learned matrix is orthonormal, which means it is a pure rotation and reflection with no scaling or shearing, so L2 distances and inner products between rotated vectors are identical to those between the originals. The benefit comes entirely from where the fixed slice boundaries fall afterwards, not from changing the geometry.
  • When would you skip OPQ despite using PQ?
    When the code is already generous enough that quantization error is not the binding constraint, when a refine stage re-ranks candidates against full-precision vectors anyway, or when a measured A/B shows the recall curves overlapping. Skipping saves build time, an extra matrix multiply per query, and a wrapper layer that complicates parameter setting.

PQ chops a vector into fixed-width strips and describes each one; OPQ first turns the picture so the interesting detail is spread evenly across the strips instead of crammed into two of them.

saying these in an interview costs you the question

  • Thinking OPQ reduces recall loss by storing more bits
  • Applying the rotation manually to queries as well
  • Assuming OPQ helps a flat or IVFFlat index
  • Setting index.nprobe directly on the pre-transform wrapper
  • Treating OPQ as free because it is one factory-string token

context

resolving links…