Why can a learned sparse retriever like SPLADE make inverted-index queries slower than BM25?

## What a learned sparse representation is A learned sparse retriever uses a language model to produce, for a piece of text, a **weighted vector over the vocabulary** — typically the model's subword vocabulary. Most entries are zero; the non-zero ones are terms the model considers important for retrieving or being retrieved by this text. Two things distinguish it from classic keyword weighting: 1. **The weights are learned**, not derived from term frequency and document frequency. 2. **The representation is expanded**: terms that never appear in the text can receive non-zero weight. A document about "myocardial infarction" may carry weight on "heart" and "attack"; a query for "laptop battery drains fast" may carry weight on "power", "discharge", "quickly". That expansion is the point. It attacks vocabulary mismatch — the core weakness of lexical retrieval — while keeping the representation *interpretable* and, crucially, *indexable in an inverted index*, since the units are still terms. SPLADE is the best-known family of such models; Elastic's ELSER is another learned sparse model. Retrieval is a dot product between the query's weighted term vector and the document's, which is structurally the same accumulation an inverted index already performs. ## Why the cost goes up **More postings lists.** Keyword query cost is roughly proportional to the number of query terms times the work per list. A natural-language query has maybe three to eight content terms after stopword handling. Its learned sparse expansion may have on the order of a hundred non-zero entries. Each is a list to open, decode and merge. **Longer postings lists.** Expansion terms are chosen for semantic association, and semantically central words tend to be common words. A rare, highly discriminative term has a short list; "power", "system" or "time" have enormous ones. So the added terms are disproportionately the expensive kind. **Dynamic pruning stops working.** This is the deep reason and the one interviewers are usually probing for. Fast keyword search is not brute-force accumulation; it uses algorithms such as WAND, block-max WAND and MaxScore, which precompute an upper bound on each term's possible contribution (and, in the block-max variants, per block of the postings list). Knowing the score of the k-th best result so far, the engine can prove that whole stretches of postings cannot enter the top k and skip them entirely. That proof is powerful when there are few query terms and their maximum contributions are very unequal — one rare high-IDF term dominates and everything else is a tie-breaker. Learned sparse queries invert both conditions: many terms, and comparatively flat weights, so no single term dominates and the sum of the remaining upper bounds stays above the threshold for much longer. Far fewer postings can be skipped, and the query drifts back toward exhaustive evaluation. **Bigger index.** If documents are expanded too, every document contributes more postings entries. The index grows, and with it the I/O and memory footprint. ## Why do it anyway Learned sparse retrieval sits between keyword and dense retrieval and inherits useful properties from both. It handles paraphrase and vocabulary mismatch much better than raw term matching, while remaining **exact-match-capable** — an unusual identifier still lands on its own term and still matches — which is precisely where dense retrieval is weakest. It needs no separate vector index, no approximate structure, and no second serving system: the filtering, faceting and pagination machinery of the existing engine applies unchanged. And its output is inspectable: you can look at the terms and weights and see why a document matched, which no dense retriever offers. ## Making it affordable **Expand one side only.** Document-side expansion with an unexpanded query keeps query cost close to ordinary keyword search and pushes the work into indexing, where it is offline and parallelizable. Query-side-only expansion has the opposite profile. Asymmetric variants of these models exist specifically for this trade. **Prune statically.** Drop non-zero entries below a weight threshold, or cap the number of terms per document or per query. The tail of a learned sparse vector contributes little score and a lot of postings; truncating it costs surprisingly little effectiveness for a large efficiency gain, and the operating point should be chosen on a judgment set. **Quantize impacts.** Store weights as small integers rather than floats. This shrinks the index and makes impact-ordered organisations of the postings practical, which restores some early-termination ability. **Use it as a second stage.** Generate candidates cheaply with ordinary keyword retrieval and rescore the shortlist with the learned sparse dot product. You lose the recall gains on documents keyword retrieval never surfaced — which may be most of the benefit — so this is a compromise, not a free lunch. **Regularize sparsity at training time.** The models are trained with an explicit sparsity penalty; picking a checkpoint trained for a sparser representation is an efficiency decision made before you ever index anything. ## The interview framing The expected answer is not "it is slower because it does more work". It is: *the representation changed shape in exactly the way the engine's fast path assumes it will not*. Query evaluation was optimised for few terms with skewed importance, and learned sparse retrieval delivers many terms with flat importance.