How do you normalize BM25 and cosine scores onto one scale before blending them?

## Why the raw scores cannot be added The two legs produce numbers that share no axis. A BM25 score is a **sum of per-term contributions**, each shaped by inverse document frequency, term frequency saturation and document-length normalization. It has no upper bound in practice, it grows with the number of query terms, and its magnitude depends on corpus statistics — the same document, same query, in a different index, scores differently. A score of 12 means nothing on its own. A cosine similarity is **bounded** (in [-1, 1], and effectively in a narrow band near the top for most trained encoders). It does not grow with query length, and it carries no notion of term rarity. Add them and the lexical leg dominates by sheer magnitude on multi-term queries and is drowned on single-term ones — the weighting you *intended* is replaced by whatever the scales happen to be that day. ## Per-query min-max, and its failure The usual normalization is min-max over each leg's returned candidate window (say the top 100): ``` norm(s) = (s - min_window) / (max_window - min_window) ``` Both legs now land in [0, 1] and a weighted sum `α·norm(dense) + (1-α)·norm(lexical)` behaves as intended *within* one query. The failures are systematic: **It manufactures confidence.** The best result of every query becomes exactly 1.0. A query where the top hit is barely relevant is scored identically to one with a perfect match. Any downstream threshold, any "did we find anything?" logic, any confidence surfaced to a user or a generator, is now meaningless. **It is outlier-sensitive.** One dominant top score stretches the denominator and compresses every other document into the bottom of the range, so the second through hundredth results become near-indistinguishable — the ordering survives but the *weighting* against the other leg collapses. **It depends on the window.** Normalize over the top 10 and over the top 1000 and you get different numbers for the same document, so the fused order changes with a parameter that has nothing to do with relevance. The window must be fixed and identical between offline evaluation and production, or your tuned α does not transfer. **It breaks on short lists.** With two or three candidates, min and max are almost arbitrary; with one candidate the formula divides by zero. ## Z-score and other options **Z-score**: `(s - mean_window) / stdev_window`. More robust to a single outlier than min-max, and it preserves relative spacing better, but it is still computed per query over a window, so it also carries no cross-query meaning, and it produces negative values that a naive weighted sum handles badly. **Theoretical or calibrated bounds**: lexical scoring does admit per-term score upper bounds — the same bounds that dynamic pruning algorithms use to skip postings — so a query-dependent maximum can be computed rather than observed. Dividing by that gives a value with a stable interpretation ("how close to the best achievable score for this query"). It is more work and less common, but it is the honest way to get a number that means something. **Distribution fitting**: the classic information-retrieval approach models the score distribution of relevant and non-relevant documents offline and maps raw scores to an estimated probability of relevance. Done properly this is the only normalization that is genuinely comparable across queries, and it needs labelled data and periodic refitting. ## The missing-leg problem A document that the lexical leg found and the dense leg did not has no dense score. This is not a corner case — it is most of the union. You must choose a fill value, and every choice biases the result: - **Fill with the normalized floor (0)**: treats "not in the top 100" as "maximally dissimilar", which over-penalizes a document that was at rank 101. - **Fill with the window's minimum score**: gentler, but still window-dependent. - **Re-score the document with the missing leg**: exact, and usually affordable only for a small union because it means computing a similarity per document. - **Keep only the intersection**: destroys recall, which is the entire reason you built a hybrid system. Rank fusion sidesteps this cleanly — absence simply contributes nothing. ## When to normalize anyway Despite all of the above, score fusion beats rank fusion in specific situations: when you need a **relevance threshold** (rank fusion cannot give you one), when one leg's *magnitude* carries real information you want to preserve (an exact identifier match should not be one rank step ahead of a fuzzy one — it should be far ahead), or when you want a continuous confidence value for downstream logic. The discipline that makes it work: fix the candidate window and use the same one everywhere; prefer a calibrated or theoretical bound over per-query rescaling if the score must mean something across queries; choose the fill value deliberately and document it; tune the blend weight on a judgment set rather than by eye; and re-tune after any change to the embedding model, the analyzer or the corpus, because every one of those moves the distributions you calibrated against.