How do you choose a Weaviate vectorization strategy for a production workload?

## Frame the decision correctly The question is not "which module is best". It is: **who owns the embedding model, and where does inference run?** Everything else — cost, latency, data residency, migration pain — falls out of that answer. Three positions are available, and mature systems often occupy more than one at a time. ## Option 1: hosted module Weaviate calls a third-party embedding API for you. Configured in one line, credentialed by a request header, working immediately. *Strengths.* Zero infrastructure. Access to strong general-purpose models without an ML team. And a guarantee that is easy to undervalue: because the same module embeds both the corpus and the query, the two can never drift apart. That entire class of silent relevance bug does not exist here. *Costs.* Every import and every text query is an external call, so the provider's availability is your availability on both paths, and their p99 is inside yours. Spend is per-token and scales with traffic rather than with data size, which makes it hard to cap. Your text goes to a third party, which may simply be disqualifying. And model versions move under you unless the module lets you pin one — a relevance evaluation from six months ago may no longer describe the system you are running. ## Option 2: self-hosted inference container Weaviate calls a model server you run — `text2vec-transformers` pointed at your own container via `TRANSFORMERS_INFERENCE_API`. *Strengths.* Text never leaves your network. Cost becomes hardware you already budget rather than a per-token meter. You keep the query-corpus matching guarantee. The model version is pinned by the image you deploy, so it changes only when you deploy. *Costs.* You are operating an inference service: sizing it for import bursts (which are far spikier than query load), keeping it available (its downtime is failed writes and failed text queries), monitoring it, and paying for accelerators that sit idle between import runs. On CPU, throughput may be low enough that bulk import becomes a scheduling problem. This option trades a vendor bill for engineering time, and only pays when you have the team to spend it. ## Option 3: self-provided vectors Weaviate stores and searches; your pipeline embeds. `Configure.Vectorizer.none()`. *Strengths.* One embedding pipeline for every consumer — the vector store, the reranker, the classifier, the cache — so identical text is embedded once. Batching, deduplication by content hash, retry policy and GPU scheduling are yours to optimise, and they are where the real cost savings live at scale. Any model works, including fine-tuned and multi-vector ones no module wraps. No third party on the read path. The model version is a dependency in your own build. *Costs.* The matching guarantee disappears: Weaviate validates dimension, not provenance, so a query embedded with a different model of the same width returns confident nonsense with no error. You must enforce that invariant yourself with pinned versions, recorded metadata and relevance regression tests. And you own the migration — changing models means re-embedding the whole corpus, evaluating, and cutting over. ## The dimensions that actually decide it - **Data sensitivity.** If text cannot leave the network, option 1 is out before any other argument is heard. - **Existing pipeline.** If something already embeds this text, embedding it again in the database is duplicated spend and a second model version to keep in sync. Choose option 3. - **Model specificity.** A fine-tuned domain encoder forces option 3. - **Team.** No ML operations capacity means option 1 or 3-with-a-managed-inference-provider, not option 2. - **Traffic shape.** High query volume against a small corpus makes per-query external calls expensive and puts weight on options 2 and 3. Bulk ingest of a large corpus with light querying inverts that. - **Relevance maturity.** The moment you are running relevance evaluations and comparing models, you need the model pinned and swappable on your own terms — option 3. ## The migration path, and why it is cheap These are not one-way doors. A collection's vectorizer is fixed at creation, but named vectors let one collection carry a module-generated vector and a self-provided vector simultaneously. So the transition is: add a self-provided named vector, backfill it from your new pipeline, evaluate both spaces on the same corpus, shadow-read, switch `target_vector`, then drop the old name. No dual-write application logic, no separate collection, no downtime. ## What I would actually do Start on a hosted module — the fastest path to knowing whether semantic search solves the problem at all, and the cheapest thing to throw away. Set a trigger for the move: the first of (a) an embedding pipeline appears elsewhere in the architecture, (b) relevance tuning starts, (c) embedding spend becomes a line item someone asks about, or (d) a data-residency requirement lands. Then migrate to self-provided vectors through a named vector, and treat the model identifier as a versioned contract from that day on. Self-hosted inference is the right middle only when residency rules them out of hosted APIs and the team can genuinely run the service.