skip to content

Vector Databases

4 roadmaps6 questions

Where embeddings live and how they are searched: in-memory or disk-resident quantized indexes, hybrid keyword-plus-vector retrieval, and reranking. Interviewers probe recall against latency and cost.

on this pageshow

explore

questions

6

In vector search, when do cosine similarity, dot product and L2 distance rank results identically?

level: juniorimportance: must knowfreq 58%

answer

  1. angle versus magnitude
  2. unit length collapses the three
  3. dot product is cheapest after normalizing
  4. squared L2 equals 2 minus 2 cosine
  5. build metric must match query metric

basics

~20 s

Once every vector is normalized to unit length, all three produce the same ordering: cosine equals the dot product, and squared L2 distance is 2 minus twice the cosine. Without normalization, the dot product favours long vectors and the three diverge.

solid answer

~50 s

Cosine similarity measures the angle between two vectors and ignores their magnitude. The dot product combines angle and magnitude. L2 (Euclidean) distance measures straight-line separation. If every vector is L2-normalized to length 1, these collapse into one ordering: cosine similarity **is** the dot product, and squared L2 distance equals `2 - 2 * cosine`, which is monotonically decreasing in cosine. That is why most teams normalize once at write time and then search with the dot product, which is the cheapest of the three to compute. Unnormalized, the dot product is biased towards high-magnitude vectors, so a long chunk can outrank a better-matching short one. Two practical rules follow: use the metric the embedding model was trained and evaluated with, and make sure the index is built with the same metric you query with — a mismatch usually degrades recall silently instead of raising an error.

code

python · 10 lines
python
import numpy as np

a = np.array([3.0, 4.0])
b = np.array([1.0, 0.0])
unit = lambda v: v / np.linalg.norm(v)
au, bu = unit(a), unit(b)

print(float(a @ b))                      # 3.0  raw dot product
print(float(au @ bu))                    # 0.6  cosine similarity
print(float(np.sum((au - bu) ** 2)))     # 0.8  == 2 - 2 * 0.6

go deeper

for a junior

Be able to say that cosine compares direction, the dot product also reflects magnitude, and L2 compares positions — and that normalizing to unit length makes all three agree.

for a middle

Explain the identity squared L2 equals 2 minus twice the cosine, why normalizing then using the dot product is the common production choice, and why the index metric must match the query metric.

for a senior

Show how a metric or normalization mismatch fails silently and how you catch it: recall@k against exact brute-force search on a sample, plus an assertion on stored vector norms in the ingestion path.

for a principal

Own the invariant across the platform — one declared metric per index, re-embedding as a migration with a shadow index and a cutover, and a policy that similarity scores are never used as absolute quality thresholds.

## The three metrics A vector store compares an embedded query against embedded chunks using a similarity or distance function. Three dominate. **Cosine similarity** is the cosine of the angle between two vectors: `dot(a, b) / (|a| * |b|)`. It ranges from -1 to 1 for arbitrary vectors and is scale-invariant — doubling every component of a vector does not change its cosine with anything. **Dot product** (inner product) is `sum(a[i] * b[i])`. It is not scale-invariant: a vector with a large norm scores highly against everything. Search using it is often called MIPS, maximum inner-product search. **L2 (Euclidean) distance** is the straight-line distance, `sqrt(sum((a[i] - b[i])^2))`. It is a *distance*, so smaller is better, while the other two are *similarities*, where larger is better. Engines almost always rank on squared L2 to avoid the square root, which does not change ordering. ## Why normalization collapses them Normalizing a vector means dividing it by its own length so `|a| = 1`. For two unit vectors: - `cosine(a, b) = dot(a, b) / (1 * 1) = dot(a, b)` — the two are literally the same number. - `|a - b|^2 = |a|^2 + |b|^2 - 2 * dot(a, b) = 2 - 2 * dot(a, b)`. The second identity is the important one: squared L2 distance is a strictly decreasing function of the dot product, so sorting ascending by L2 gives exactly the same neighbour list as sorting descending by cosine. Ranking is preserved even though the numbers differ. This is why a store that only supports L2 can still serve cosine semantics, provided you normalize on the way in. ## When they genuinely diverge They diverge whenever magnitudes vary. If your pipeline stores unnormalized vectors and searches by dot product, the ranking mixes "how similar is this in meaning" with "how big is this vector". Some models deliberately encode a notion of confidence or document length in the norm, and for those the dot product is the intended metric. Most general-purpose text embedding models are trained with a cosine objective and either emit unit vectors already or expect you to normalize. A subtler point: the dot product is not a metric in the mathematical sense. It has no triangle inequality, and a vector is not necessarily most similar to itself — some other vector with a larger norm can score higher against it. Graph-based approximate indexes navigate by greedy descent and lean on metric-space intuitions, so raw inner-product search is harder for them than cosine or L2. The standard workaround is to normalize and treat the problem as cosine search. ## Getting it wrong in practice The failure is usually quiet. If the index was built with cosine and the query path computes L2 over unnormalized vectors, nothing errors — you simply get a worse neighbour list, and unless you measure recall against exact brute-force search on a sample, you will not notice. The same is true if half a corpus is normalized and half is not, which happens when a re-ingestion job uses a different code path. A related trap is mixing embedding models. Vectors from two different models occupy different spaces; distances between them are arithmetic noise dressed up as scores. Changing models means re-embedding and rebuilding the whole index, not appending. ## What to do 1. Read the model's documentation for the metric it was trained with, and use that metric end to end. 2. Normalize at write time if you intend cosine semantics, then use the dot product for speed. 3. Assert the invariant — a cheap check that stored norms are within a tolerance of 1.0 catches an unnormalized ingestion path immediately. 4. Configure the index and the query with the same metric, and treat that configuration as part of the schema. 5. Validate with recall@k against exact search on a held-out sample of queries; a metric mismatch shows up there long before a user complains. Finally, remember what a similarity score is not: a high cosine means two texts sit near each other in embedding space, not that the chunk answers the question. Scores are also not comparable across queries — 0.82 for one query may be a strong match and for another a weak one — so a global similarity threshold is a fragile filter.

  • Why is maximum inner-product search harder for a graph index than cosine search?
    Because the inner product is not a metric. There is no triangle inequality, and a vector need not be its own nearest neighbour — a longer vector can score higher against it than it scores against itself. Greedy graph traversal assumes locality that those properties provide, so recall degrades. The usual fix is to normalize and search by cosine, or to apply a transformation that maps the inner-product problem onto a nearest-neighbour one.
  • You switch to a new embedding model but keep the existing index. What breaks?
    Everything, silently. Vectors from two models live in unrelated coordinate spaces, so distances between an old chunk and a new query are meaningless numbers that still sort into a plausible-looking result list. Dimensionality may also differ, which at least fails loudly. A model change means re-embedding and rebuilding the entire corpus, ideally into a new index you can shadow-test against the old one before cutting over.
  • Is a fixed cosine-similarity threshold a good way to drop irrelevant chunks?
    Rarely. Score distributions shift by query, by model and by chunk length, so a cut-off that works for one query set drops good results for another. Prefer relative approaches: take top-k, or rerank and cut on the reranker's calibrated score, or compare a candidate's score against the distribution for that same query. If you must use a threshold, derive it from a labelled sample and re-derive it whenever the model changes.

Comparing two arrows: cosine asks only whether they point the same way, the dot product also rewards the longer arrow, and L2 measures the gap between their tips. Trim every arrow to the same length and the three questions become one.

saying these in an interview costs you the question

  • Cosine and dot product always rank results the same way
  • Normalization is only an optimization and cannot change results
  • Euclidean distance is meaningless for embeddings
  • Vectors from two different embedding models can be compared
  • A high cosine score proves the chunk answers the question

context

open as a page

In an HNSW vector index, what do m, efConstruction and ef trade off?

level: middleimportance: must knowfreq 66%

basics

~20 s

m sets how many neighbour links each node keeps, efConstruction how hard the builder searches while inserting, and ef how wide the search is at query time. Raising them raises recall while costing memory, build time and query latency respectively.

open as a page

Why add BM25 keyword retrieval alongside dense vectors, and how does reciprocal rank fusion combine them?

level: middleimportance: must knowfreq 56%

basics

~20 s

Dense embeddings match meaning but smear rare exact tokens such as an identifier, a part number or a docket number; BM25 matches those literally. Reciprocal rank fusion merges the two result lists by rank position, so the systems' incomparable raw scores never have to be reconciled.

open as a page

Why can a metadata filter make an HNSW vector search return almost nothing?

level: seniorimportance: should knowfreq 44%

basics

~20 s

A selective filter applied after the search leaves almost nothing, because the top-k nearest vectors rarely satisfy a narrow predicate. Applied before the search, it deletes most nodes from the proximity graph, breaking the connectivity that greedy traversal depends on.

open as a page

What does a cross-encoder reranker buy over the vector index's own ranking?

level: seniorimportance: should knowfreq 48%

basics

~20 s

A cross-encoder reads the query and the candidate together, so it can judge fine-grained relevance that independently-encoded vectors cannot. It raises precision at the top of the list but cannot recover anything the first stage failed to retrieve, and its cost grows with the number of candidates scored.

open as a page

Your 30M-chunk vector index no longer fits RAM — what do you change?

level: principalimportance: should knowfreq 34%

basics

~20 s

Start with the arithmetic — dimensions times bytes times chunks — then work the levers in order: fewer dimensions, quantized vectors with a full-precision rescoring pass, a disk-resident index, sharding along a natural tenancy boundary, and finally the option of not storing vectors at all.

open as a page