Vector

create_index fixes an index's vector dimension and distance metric permanently. Every vector you upsert must have exactly that length or the write is rejected, and switching either setting means creating a new index and re-upserting the data.

A Pinecone record is an id string, a values list whose length must equal the index dimension, and optional metadata. index.upsert() takes a list of records, so send batches of roughly 100 to stay under the request size cap.

Pinecone accepts sparse values only in indexes created with metric="dotproduct". Dot product is linear, so scaling the dense and sparse query vectors weights their contributions predictably; cosine and euclidean indexes reject sparse vectors, and the metric cannot be changed later.

A hybrid record carries the dense embedding in values and a sparse vector in sparse_values as {"indices": [...], "values": [...]}. The query passes vector= and sparse_vector= together, and Pinecone returns one ranked list with a single combined score per match.

A namespace is a logical partition inside a Pinecone index. Each upsert and each query names one namespace, and a query searches only that partition — there is no cross-namespace search in a single call. Namespaces are created implicitly on first write.

A collection is Weaviate's typed container for objects of one kind, much like a table. collections.create names it and declares its properties with data types, plus per-collection settings such as the vector index, replication and multi-tenancy.

alpha weights the two halves of a Weaviate hybrid search: alpha=0 is pure BM25 keyword search, alpha=1 is pure vector search, and values in between blend them. The v4 Python client's default of 0.7 leans toward the vector side.

near_text sends a raw string and lets the collection's configured vectorizer embed it server-side. near_vector takes an embedding you computed yourself. near_object finds the neighbours of an object already stored, referenced by its UUID.

A configured vectorizer makes Weaviate call the embedding model itself: you insert plain properties and the server produces the vector, and a text query is embedded server-side too. Without one you must supply every vector yourself.

Auto-schema is on by default and invents property definitions from whatever object arrives first, so types are guessed from one sample, stray keys become permanent indexed properties, and the resulting schema differs between environments. Property types cannot be changed afterwards.

A Qdrant point is the unit of storage: an id, one or more vectors, and an arbitrary JSON payload. Ids must be an unsigned integer or a UUID — arbitrary strings are rejected. Upserting an existing id replaces that point.

Pass a filter to the search call. In the Python client that is query_points(..., query_filter=models.Filter(must=[models.FieldCondition(key="lang", match=models.MatchValue(value="en"))])). Only points whose payload satisfies the filter come back, still ranked by vector distance.

VectorParams pins a Qdrant collection's vector dimension and its distance metric for the collection's lifetime. Every upserted point must match that size exactly, and neither value can be changed later — switching embedding models means creating a new collection.

m is how many graph links Qdrant keeps per vector (default 16); ef_construct is how wide the neighbour search is while building (default 100). Raising m costs RAM permanently, raising ef_construct costs indexing time only.

Clauses are ANDed with each other: a point must satisfy every condition in must, at least one in should, and none in must_not. min_should raises the should bar from one to N. Nesting Filter objects inside should expresses OR-of-ANDs.

Chroma runs each document through the collection's embedding function and stores the resulting vector next to the text, its id and its metadata. ids are required and must be unique; documents, metadatas and embeddings are parallel lists.

EphemeralClient keeps collections in memory only and loses everything when the process exits. PersistentClient(path="./chroma") writes to that directory and reloads it on the next run. Plain chromadb.Client() is ephemeral, which is why prototype data disappears.

query_texts hands raw strings to Chroma, which embeds them with the collection's embedding function. query_embeddings takes vectors you already computed, so Chroma skips embedding entirely. Use it when the vector comes from elsewhere or is cached.

Distances are only meaningful between vectors from the same model. Mixing embedding functions in one Chroma collection puts incomparable vectors in one index, so nearest-neighbour results become arbitrary — and if the dimensions happen to match, nothing errors.

where filters on the structured metadata dictionary attached to each record, using operators like $eq, $in and $gte. where_document filters on the stored document text itself, with $contains and $not_contains substring matching. They are separate arguments and combine with AND.

IndexFlatL2 stores every vector uncompressed and compares a query against all of them, returning exact L2 nearest neighbours with no training step. It is the right choice for small collections and as a recall baseline, but its cost grows linearly with the dataset.

Create a faiss.StandardGpuResources() object, then call faiss.index_cpu_to_gpu(res, device, index). The resources object owns the GPU scratch memory and cuBLAS handles for that device, and you must keep a Python reference to it for as long as the index lives.

It builds an IVFPQ index: a coarse quantizer that splits the space into 4096 inverted lists, with each vector stored as a product-quantized code of 64 one-byte values instead of the raw floats. Both stages must be trained before adding data.

IVF and PQ indexes learn structure from data — k-means centroids for IVF, codebooks for PQ. Until train() runs, is_trained is False and add() raises. Train on a representative sample of the vectors you will actually search.

Replicate when the index fits on one GPU and you need more queries per second — each device holds a full copy and queries are split between them. Shard when the index does not fit on one GPU: each device holds a slice, every query touches all of them, and results are merged.

Call table.search(query_vector), chain .limit(n) to say how many neighbours you want, and finish with .to_pandas(). Every returned row carries the table's own columns plus a _distance column holding that row's distance from the query vector.

db.create_table(name, data=...) infers an Arrow schema from a pandas DataFrame, PyArrow table or list of dicts. Passing schema= with a PyArrow schema or a lancedb.pydantic LanceModel defines it explicitly and allows creating an empty table.

num_partitions sets how many clusters the vectors are split into, so it governs how much data one query touches per probe. num_sub_vectors sets how many pieces each vector is chopped into for compression, so it governs index size and how lossy stored distances are.

It runs an exhaustive brute-force scan, comparing the query against every vector in the table. Results are exact, but latency grows linearly with row count. Calling create_index builds an approximate IVF_PQ index that trades that exactness for sub-linear search.

Every write to a LanceDB table — append, update, delete, overwrite, index build — commits a new immutable version instead of mutating data in place. table.list_versions() enumerates them, table.checkout(n) pins the table object to version n for reading, and table.checkout_latest() returns to the newest.