skip to content

Encoding & Similarity APIs

Load a model, encode a batch, compare vectors with encode, encode_query, encode_document and model.similarity. Interviewers look for normalization, batch size and re-encoding the corpus per query.

on this pageshow

questions

5

In Sentence Transformers, what do model.encode() and model.similarity() return when you embed a list of sentences and compare them?

level: juniorimportance: must knowfreq 40%

answer

  1. one row per input
  2. numpy by default, tensor on request
  3. a single string drops the batch axis
  4. all pairs, not row by row
  5. cosine unless similarity_fn_name says otherwise

basics

~20 s

encode() returns a numpy array with one row per input (a 1-D vector for a single string); similarity() returns a torch tensor of shape [len(a), len(b)] scoring every pair, with cosine as the default function.

solid answer

~40 s

`SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2").encode(sentences)` returns a numpy array of shape `[n, 384]` for that model: `convert_to_numpy=True` is the default, and `convert_to_tensor=True` gives one stacked torch tensor on the model's device instead. Pass a single string and you get a 1-D vector, not a `[1, 384]` matrix. `model.similarity(a, b)` accepts numpy arrays or tensors and returns a torch tensor of shape `[len(a), len(b)]` — every row of `a` scored against every row of `b` — using the model's `similarity_fn_name`, which becomes `"cosine"` when neither you nor the saved config set one. For scores of aligned pairs only, call `similarity_pairwise`. The vectors themselves are not unit length unless you pass `normalize_embeddings=True` or the checkpoint ends in a `Normalize` module.

code

python · 14 lines
python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

sentences = ["The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium."]
emb = model.encode(sentences)
print(type(emb).__name__, emb.shape)   # ndarray (3, 384)

one = model.encode("A single sentence")
print(one.shape)                        # (384,)

scores = model.similarity(emb, emb)     # torch tensor, all pairs
print(scores.shape)                     # torch.Size([3, 3])
print(model.similarity_fn_name)         # cosine

go deeper

for a junior

Recall the two shapes: encode gives one row per input as numpy, similarity gives an all-pairs tensor. Know that cosine is the default and a single string returns a 1-D vector.

for a middle

Explain the arguments that change the output: convert_to_tensor keeping results on the device, output_value for token vectors, normalize_embeddings, and how similarity_fn_name is resolved from the saved config.

for a senior

Show you know where vectors come from: the module pipeline, which pooling a raw checkpoint gets, and why output type and device placement matter once embeddings feed a GPU search step.

for a principal

Frame the defaults as contracts a team must pin: the checkpoint, its pooling and similarity function determine every stored vector, so changing any of them means re-embedding, not reconfiguring.

## Loading a model A **Sentence Transformers** model is a pipeline of modules: an input module (usually `Transformer`, which wraps a Hugging Face Transformers checkpoint and its tokenizer), a **`Pooling`** module that turns per-token vectors into one vector per input, and sometimes `Dense` or `Normalize` modules after it. `SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")` downloads the checkpoint, reads its saved module list, and places the model on the most capable device it finds — CUDA or MPS when available, otherwise CPU — unless you pass `device=`. If the repository has no Sentence Transformers configuration (no `modules.json`), the library builds the pipeline itself: a `Transformer` module plus a `Pooling` module using **mean pooling**, or, since Sentence Transformers v5.4, **last-token pooling** (`"lasttoken"`) when the checkpoint's architecture is a causal language model. How pooling works is a topic of its own; at this layer the point is that a published Sentence Transformers model brings its pooling with it, and a raw checkpoint gets a default. ## What `encode()` returns `encode()` turns text into **embeddings**: fixed-size float vectors. Its return type depends on a handful of arguments: | Call | Returns | |---|---| | `model.encode(["a", "b", "c"])` | numpy array, shape `[3, dim]` (float32 for a model loaded at full precision) | | `model.encode("a")` | numpy array, shape `[dim]` — the batch axis is dropped for a single input | | `model.encode(texts, convert_to_tensor=True)` | one stacked torch tensor, left on the model's device | | `model.encode(texts, output_value="token_embeddings")` | one tensor per input with a vector per token | | `model.encode(texts, output_value=None)` | one dict per input with every output (embedding, token vectors, attention mask) | Other defaults worth knowing: - `batch_size=32` inputs per forward pass. - `normalize_embeddings=False`: vectors come back at whatever length the model produces, unless its last module is `Normalize`. - `precision="float32"`: the quantized options are a separate concern. - `show_progress_bar=None`: a bar appears only when logging is at INFO or DEBUG level. `convert_to_tensor=True` overrides `convert_to_numpy`, which matters when the next step is a GPU computation: the tensor stays on the device instead of being copied to CPU memory. ## What `similarity()` returns `model.similarity(a, b)` scores **every embedding in `a` against every embedding in `b`** and returns a torch tensor of shape `[len(a), len(b)]`, even when you hand it numpy arrays. A 1-D input is treated as a batch of one, so a single query against 1,000 passages gives `[1, 1000]`. The function comes from **`similarity_fn_name`**: `"cosine"`, `"dot"`, `"euclidean"` or `"manhattan"` (the last two are negated distances, so higher still means closer). It is read from the saved `config_sentence_transformers.json` when the checkpoint has one; otherwise it is set to `"cosine"` the first time you call `similarity`. With cosine, two copies of the same sentence score `1.0` on the diagonal of `model.similarity(e, e)`. `model.similarity_pairwise(a, b)` is the row-by-row variant: equal-length inputs, one score per index, shape `[n]`. In Sentence Transformers v6, `similarity` and `similarity_pairwise` are ordinary methods, and half-precision embeddings are scored in float32 so nearby scores do not tie. ## A minimal end-to-end call 1. Load: `model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")`. 2. Encode: `emb = model.encode(sentences)` — shape `[n, 384]` for this model. 3. Compare: `scores = model.similarity(emb, emb)` — shape `[n, n]`. 4. Read: `scores[i][j]` is the similarity of sentence `i` and sentence `j`. `util.cos_sim(a, b)` computes the same all-pairs cosine matrix as a free function, without a model, which is handy when the embeddings were loaded from disk. ## Reading the output in practice - `emb.shape[1]` matches `model.get_embedding_dimension()` — the Sentence Transformers v5.4 name for what used to be `get_sentence_embedding_dimension()`, which still works with a deprecation warning. - A score tensor converts to plain Python numbers with `.tolist()` when you need to log it or return it as JSON. - To find the best match per row, `torch.topk` over a row of the score matrix returns the top values and their indices; the indices point back into the second argument's input list. - Similarity values are only comparable when both sides came from the same model with the same settings; a score from one checkpoint means nothing next to a score from another. ## Common slips - **Expecting a list of lists.** The default output is a numpy array; call `.tolist()` only if you need JSON. - **Indexing a single-string result as a matrix.** `model.encode("a")[0]` is the first number, not the first vector. - **Assuming pairwise output.** `similarity(a, b)` is all-pairs; for three-by-three inputs you get nine scores, not three. - **Assuming unit length.** Whether vectors are normalized depends on the checkpoint and on `normalize_embeddings`, not on `encode()` itself.

  • How do you get similarity scores for aligned pairs rather than a full matrix?
    Call `model.similarity_pairwise(a, b)` with two equal-length collections: it returns a 1-D tensor where entry `i` scores `a[i]` against `b[i]`, using the same `similarity_fn_name`. `model.similarity` would build the full `[n, n]` matrix and you would read its diagonal, wasting quadratic work.
  • What pooling does SentenceTransformer apply when you load a plain Transformers checkpoint with no Sentence Transformers config?
    With no `modules.json`, it builds a `Transformer` module plus a `Pooling` module itself: mean pooling for encoder models, and since Sentence Transformers v5.4 last-token pooling for checkpoints whose architecture is a causal language model. A checkpoint published as a Sentence Transformers model keeps the pooling saved in its own configuration.
  • Where does encoding run if you pass no device?
    The constructor picks the most capable device it finds — CUDA or MPS when available, otherwise CPU — and `encode` uses it. Pass `device="cpu"` or `device="cuda"` to the constructor, or `device=` to a single `encode` call, to choose explicitly.

  • encode() returns a Python list of float lists by default
  • model.similarity(a, b) only compares a[i] with b[i]
  • encode() always returns unit-length vectors
  • Encoding a single string returns a [1, dim] matrix
  • similarity() uses dot product unless you configure cosine

Assumes Sentence Transformers v6 (6.1.0): similarity() is a plain method since v6, and raw causal-LM checkpoints default to last-token pooling since v5.4.

open as a page

A Sentence Transformers semantic-search prototype re-encodes the whole corpus on every query; how do you restructure it around util.semantic_search?

level: middleimportance: must knowfreq 25%

basics

~10 s

Encode the corpus once with encode_document() and cache the result; per request encode only the query with encode_query(), call util.semantic_search(query_emb, corpus_emb, top_k=k), and map each hit's corpus_id back to your texts.

open as a page

How does batch_size affect Sentence Transformers' encode() when embedding a few million passages, and how do you choose a value?

level: middleimportance: should knowfreq 12%

basics

~20 s

batch_size (default 32) is how many inputs share one forward pass. Larger batches raise GPU throughput until memory runs out; encode() length-sorts inputs so batches pad little, then restores your order. Benchmark a few sizes on a sample.

open as a page

In Sentence Transformers v6, how do normalize_embeddings and similarity_fn_name work together, and how do you switch a model to dot-product scoring?

level: middleimportance: should knowfreq 15%

basics

~10 s

encode(normalize_embeddings=True) L2-normalizes each vector; similarity_fn_name picks what model.similarity() computes. In v6 switch with model.similarity_fn_name = "dot" or the constructor argument — assigning a function to model.similarity now silently shadows the method.

open as a page

In Sentence Transformers, a retrieval prototype on an asymmetric checkpoint calls encode() for both queries and passages; what goes wrong, and how do encode_query, encode_document and prompts fix it?

level: seniorimportance: should knowfreq 12%

basics

~20 s

Plain encode() applies only a prompt or prompt_name you pass, or default_prompt_name, so an asymmetric checkpoint embeds queries without its trained prefix. encode_query() and encode_document() pick the model's query and document prompts, and set the routing task, when you pass none.

open as a page