In Sentence Transformers, when do you call CrossEncoder.predict and when CrossEncoder.rank, and what does each one return?
answer
- pairs in, scores out
- one query against many documents
- input order against sorted order
- corpus_id, score, optional text
- top_k and return_documents on rank
basics
~20 sCrossEncoder.predict scores a list of (query, passage) pairs and returns one score per pair in input order. CrossEncoder.rank takes one query and a list of documents and returns them sorted by score as corpus_id/score dicts.
solid answer
~40 s`predict(pairs)` is the general call: you build the pairs yourself, it batches them (`batch_size=32` by default) and returns a numpy `float32` array aligned with your input order, or a tensor with `convert_to_tensor=True`. `rank(query, documents, top_k=None, return_documents=False)` is a convenience for reranking: it pairs the one query with every document, calls `predict` internally, and returns a list of dicts with `corpus_id` (the position in the `documents` list you passed) and `score`, sorted highest first, plus `text` if you ask for it. In Sentence Transformers v6 the `score` values from `rank` are plain Python floats. `rank` only works for single-score models (`num_labels=1`); classifier checkpoints such as NLI models go through `predict`.
code
python · 16 linesfrom sentence_transformers import CrossEncoder
model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
query = "How many people live in Berlin?"
passages = [
"Berlin had a population of 3,520,031 registered inhabitants.",
"Berlin is well known for its museums.",
]
# predict: you build the pairs, scores come back in input order
scores = model.predict([(query, p) for p in passages])
print(scores) # numpy float32 array, one score per pair
# rank: one query against many documents, sorted best first
for hit in model.rank(query, passages, top_k=1, return_documents=True):
print(hit["corpus_id"], hit["score"], hit["text"])go deeper
Recall the two shapes: predict takes pairs and returns scores in your order; rank takes one query plus documents and returns sorted dicts with corpus_id and score.
Explain that rank wraps predict, that top_k and return_documents shape its output, and that corpus_id indexes the list you passed rather than any wider corpus.
Show you choose predict for multi-label models, mixed query batches and tensor output, and that you know rank's v6 Python-float scores and its num_labels=1 restriction.
Frame the choice as API ergonomics against control: rank is the right default for single-query reranking, while predict is the seam for batching heterogeneous pairs across devices.
## Two calls over the same model A **CrossEncoder** in Sentence Transformers reads a query and a passage *together* in one forward pass and outputs a score for that pair. It never produces a vector for a single text, so everything you do with it is pair scoring. The library gives you two entry points: - **`predict(inputs, ...)`** — the general-purpose call. `inputs` is a list of pairs (or one pair). You decide what the pairs are. - **`rank(query, documents, ...)`** — a reranking convenience. You hand over one query and a list of candidate documents; it builds the pairs for you, scores them and sorts the result. `rank` is implemented on top of `predict`: it creates `[[query, doc] for doc in documents]`, calls `predict` on that list, and then sorts. The options `rank` forwards to `predict` — `batch_size`, `activation_fn`, `apply_softmax`, `device`, `show_progress_bar` and the prompt arguments — behave the same way in both calls. ## What each one returns | | `predict` | `rank` | |---|---|---| | Input | list of (a, b) pairs, or a single pair | one query + a list of documents | | Order of output | same order as the input pairs | sorted by score, highest first | | Default type | numpy `float32` array | list of dicts | | Keys / shape | one score per pair (one row of scores per pair for multi-label models) | `corpus_id`, `score`, and `text` when `return_documents=True` | | Truncating the output | slice it yourself | `top_k` (default `None` = return all) | | Models supported | any `num_labels` | only `num_labels=1`, otherwise it raises `ValueError` | A few details worth knowing: 1. **`predict` keeps your order.** Internally it sorts the pairs by length so that batches pad less, then restores the original order before returning. You can zip the result with your input list safely. 2. **A single pair returns a single value.** `model.predict(("q", "p"))` gives one score, not an array of length one. 3. **Output type is controlled on `predict`.** `convert_to_numpy=True` is the default; `convert_to_tensor=True` returns one stacked tensor. In Sentence Transformers v6, `rank` no longer has these switches: its scores are Python floats, and passing `convert_to_numpy` or `convert_to_tensor` to `rank` only logs a deprecation warning. 4. **`corpus_id` in `rank` is a position in the list you passed**, not an id from any wider corpus. If the documents came from an earlier retrieval step, you map back through that step's hits yourself. ## When to use which Use **`rank`** for the common reranking shape: one user query, a short list of candidates from a first-stage retriever, and you want the best few in order. `top_k=10` trims the result and `return_documents=True` saves a lookup when you only need the text. Use **`predict`** when: - the pairs are not one query against many documents — for example, scoring a batch of *different* queries each with its own candidate in one call, or scoring duplicate-question pairs; - the model is a **classifier with several labels**, such as `cross-encoder/nli-deberta-v3-base`, which returns three logits per pair (use `apply_softmax=True` to turn them into probabilities that sum to 1 per pair); - you want an **array or tensor** to feed into further computation rather than a list of dicts. ## Loading the model Both calls hang off the same object: ```python from sentence_transformers import CrossEncoder model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2") scores = model.predict([("How many people live in Berlin?", "Berlin is well known for its museums.")]) top = model.rank("How many people live in Berlin?", passages, top_k=3, return_documents=True) ``` The constructor takes the Hub id (or a local path) and keyword-only options such as `max_length`, `activation_fn`, `num_labels` and `device`. Passing those positionally was deprecated in Sentence Transformers v5.4 and still works with a warning. Larger community checkpoints such as `BAAI/bge-reranker-base` load the same way. ## Common mistakes - Treating `rank`'s `corpus_id` as an id into the full collection rather than into the candidate list. - Calling `rank` on a multi-label classifier and being surprised by the `ValueError`. - Re-sorting `predict` output and losing the alignment with the input pairs. - Expecting `rank` to accept `convert_to_tensor=True` and return tensors in v6 — call `predict` for that. The short version for an interview: **`predict` is "score these pairs, in my order"; `rank` is "sort these documents for this query".**
- How would you spread CrossEncoder.predict over two GPUs?Pass a list of devices, for example `device=["cuda:0", "cuda:1"]`; `predict` then starts a multi-process pool automatically and splits the pairs into chunks. For repeated calls you can create the pool once with `start_multi_process_pool()`, pass it as `pool=`, and stop it with `stop_multi_process_pool()` when done.
- Why might predict on a three-label NLI cross-encoder need apply_softmax=True?A multi-label CrossEncoder uses an identity activation by default, so `predict` returns raw logits per class. `apply_softmax=True` normalises each pair's row so the class scores sum to 1, which you need if you want probabilities rather than just the argmax label.
- What does batch_size change in a CrossEncoder call?Only how many pairs go through the model per forward pass; the default is 32 on both `predict` and `rank`. It trades memory for throughput and does not change the scores. The docs advise benchmarking a few values on your hardware, model and typical input length.
- rank returns scores in the same order as the input documents
- rank's corpus_id is the document's id in the whole collection
- predict returns a list of dicts with corpus_id and score
- rank works for any CrossEncoder, including multi-label NLI classifiers
- A CrossEncoder can embed each passage once and reuse the vectors
Assumes Sentence Transformers v6; rank returns Python float scores and its convert_to_numpy / convert_to_tensor arguments are deprecated no-ops since v6.0.