Why does OpenAI recommend cosine similarity for comparing its embeddings?

## The three candidate metrics Given two vectors a and b: - **Cosine similarity** = (a · b) / (‖a‖‖b‖) — the cosine of the angle between them, in [-1, 1]. Magnitude-invariant. - **Dot product** = a · b — the same numerator without the normalisation. Sensitive to magnitude. - **Euclidean (L2) distance** = ‖a − b‖ — straight-line distance. Sensitive to magnitude. For text embeddings, the semantic signal lives in *direction*, not length. Two documents about the same subject should point the same way regardless of how long they are. That is the general argument for cosine over raw dot product in embedding search. ## The specific reason for OpenAI's models OpenAI's embedding endpoints return vectors already normalised to unit L2 length. That single fact collapses the distinction: - With ‖a‖ = ‖b‖ = 1, the cosine formula's denominator is 1, so **cosine similarity equals the dot product exactly**. - Expanding the squared Euclidean distance gives ‖a − b‖² = ‖a‖² + ‖b‖² − 2(a · b) = 2 − 2(a · b). Distance is a strictly decreasing function of the dot product, so **ranking by smallest Euclidean distance and ranking by largest cosine similarity produce the identical ordering**. So the honest answer to "which metric should I use" is: for these embeddings it does not change your results. Cosine is recommended because it is the conventional default, its scores are interpretable on a fixed scale, and computing it as a bare dot product is the cheapest option — you skip two norm computations per comparison, which matters when you are scanning millions of vectors. ## Configuring a vector store Most vector databases ask you to pick a distance metric at index creation. For OpenAI embeddings, choose cosine — or the store's "inner product" / "dot" metric, which for unit vectors is the same thing and is often marginally faster because the engine skips normalisation. What you must not do is mix: if the store normalises on ingest but you feed it vectors you truncated yourself without re-normalising, or you build with one metric and query assuming another, results degrade in ways that look like a bad model rather than a bad configuration. ## Reading the scores A classic trap. Cosine similarity is mathematically bounded by [-1, 1], but real OpenAI text embeddings rarely produce negative or near-zero scores between natural-language strings. Unrelated sentences commonly land somewhere in the 0.1–0.4 region and closely related ones in the 0.5–0.9 region, with the exact bands depending on the model and the domain. Consequences: - **There is no universal relevance threshold.** A hard-coded "0.8 means relevant" cutoff copied from a blog post will silently under- or over-retrieve on your corpus. Calibrate against a labelled sample, or avoid absolute thresholds and take top-k plus a reranking step. - **Scores are not probabilities** and should not be shown to users as confidence percentages. - **Scores are not comparable across models or widths.** A 0.62 from `text-embedding-3-small` and a 0.62 from `text-embedding-3-large` mean different things, and comparing vectors of different widths is not defined at all. ## Where the equivalence breaks The cosine/dot equivalence holds only while your vectors are unit length. Two ways to lose that: 1. **Client-side truncation.** Slicing a vector to its first N coordinates yields something shorter than unit length, by an amount that varies per vector. Dot product then partly measures how much norm survived truncation rather than semantic similarity — which distorts ranking, not just scale. Re-normalise after any manual truncation. 2. **Post-processing.** Averaging several chunk embeddings to represent a document, or adding vectors together, produces a non-unit result. Re-normalise before storing. In both cases cosine similarity still behaves correctly because it normalises internally; it is the dot-product shortcut that becomes wrong. That is an argument for configuring cosine rather than inner product in a store where you cannot guarantee ingest hygiene. ## What an interviewer is checking That you know the vectors are normalised and can say what follows from it, rather than reciting "cosine is standard for embeddings" as folklore. The follow-on judgment — no magic threshold, scores incomparable across models, re-normalise anything you transform — is what separates someone who has run a retrieval system from someone who has read about one.