What breaks when you change a CrewAI crew's embedder after memory exists?
answer
- vectors from two models in one collection
- one failure is loud, one is silent
- dimensionality is the loud one
- treat it like a data migration
- new store path gives you rollback
basics
~20 sExisting vectors were produced by the old model, so the new one either fails outright on a dimension mismatch or, at equal dimensions, silently returns poor matches. Recovery is to reset the affected stores and re-index, or point the crew at a fresh storage directory.
solid answer
~50 sCrewAI's `embedder` config decides how memory and knowledge text becomes vectors, and the persisted collections keep whatever the previous model produced. Swap the model and you hit one of two failure modes. If the new model's dimensionality differs, writes and queries against the existing collection error out — a hard, obvious break. If the dimensionality happens to match but the model is different, nothing errors: the query vector lives in a different semantic space from the stored vectors, so retrieval quality collapses quietly and the crew just gets worse. The second is far more dangerous because it looks like a prompt problem. The fix is to treat an embedder change as a re-index: clear the affected stores with `crew.reset_memories(...)` or `crewai reset-memories`, or point `CREWAI_STORAGE_DIR` at a clean location, then let the next kickoff rebuild knowledge and start memory fresh.
go deeper
Know that the embedder decides how text becomes vectors, and that stored vectors and query vectors must come from the same model for retrieval to mean anything.
Distinguish the two failure modes — a dimension mismatch that errors versus an equal-width swap that silently ruins matches — and say that the remedy is resetting the vector-backed stores and re-indexing.
Show the diagnosis path: correlate the quality drop with a config change, inspect what context is actually being injected, and cut over by storage directory so the change is reversible and does not serve traffic against a half-built index.
Set the rule that embedder identity and store location are versioned together, budget re-indexing cost and latency into the change plan, and make sure no provider-side model default can shift the embedding function without a deliberate migration.
## What the embedder setting controls `Crew(embedder={"provider": "openai", "config": {"model": "text-embedding-3-small"}})` tells CrewAI which embedding model turns text into vectors for the vector-backed stores: short-term memory, entity memory, and knowledge collections. (Long-term memory is a SQLite record store and is untouched by this.) The default path uses OpenAI embeddings, which is why enabling memory introduces an OpenAI key dependency even when your generation model is elsewhere; switching provider is a common and legitimate reason to set `embedder` in the first place. ## Why a swap is not free A vector store is only meaningful if every vector in a collection was produced by the same function. Similarity is computed between the query vector and the stored vectors; if they came from different models, the geometry is unrelated. Two distinct failures follow. **Hard failure — dimension mismatch.** Collections are created with a fixed vector width. If the old model produced 1536-dimension vectors and the new one produces 768, the store rejects the operation. You get an error at kickoff or on the first retrieval, which is annoying but honest: the system tells you what you did. **Silent failure — same width, different space.** Plenty of models share a dimensionality. Nothing rejects the write. The collection now contains a mix of vectors from two incompatible spaces, and queries embedded by the new model match old entries essentially at random. Retrieval degrades, the agent's injected context becomes irrelevant, and output quality drops without a single stack trace. Teams typically spend days blaming the prompt, the model temperature or the task descriptions before someone checks what changed in the embedder config. A related quieter variant: providers version their embedding models. "Same model name" is not always "same vectors forever", so long-lived stores can drift even without an intentional config change. ## Detecting it - **Correlate with deploy history.** Sudden quality drop with no prompt change, immediately after a config or dependency bump, points here. - **Inspect retrieval.** Run the crew verbose and look at the context being injected. If knowledge chunks come back topically unrelated to the task, the retrieval layer — not the prompt — is broken. - **Bisect on a clean store.** Reset memory and re-index knowledge with the new embedder, then rerun the same inputs. If quality returns, mixed vectors were the cause. ## Recovering An embedder change should be treated as a schema migration for your vector data: 1. Clear the affected stores. `crew.reset_memories(command_type="all")` in code, or the `crewai reset-memories` CLI, drops the accumulated state; the narrower command values let you target individual stores. 2. Re-index knowledge. The next kickoff re-chunks and re-embeds every knowledge source with the new model — budget for that embedding cost and the extra kickoff latency, especially if the corpus is large. 3. Alternatively, cut over by location: set `CREWAI_STORAGE_DIR` to a new path for the new embedder. This gives you an instant rollback — flip the variable back and the old store is intact. The cutover approach is what you want in production, because it makes the change reversible and avoids a window where the crew is serving traffic against a half-rebuilt index. ## Preventing it - **Pin the embedding model explicitly** rather than relying on a provider default, so an upgrade elsewhere cannot change it under you. - **Tie the storage location to the model.** Encoding the embedder identity in the storage path makes mixed-space stores structurally impossible. - **Note that long-term memory survives.** It is SQLite rows, not vectors, so it carries across an embedder change; that is a small consolation, and it also means a reset of vector stores alone leaves task-evaluation history intact. ## Interview framing The answer that lands names both failure modes and stresses that the silent one is worse, then gives a concrete recovery — reset and re-index, or cut over by storage directory for reversibility — and closes with the preventive rule that the embedder and the store are a single versioned unit. Saying only "you have to re-embed" is correct but shows no operational experience.
- Which of CrewAI's memory stores is unaffected by an embedder change?Long-term memory. It is a SQLite record of past task executions with their quality evaluations and suggestions, not embeddings, so no vector geometry is involved and the rows remain valid across an embedder swap. Short-term memory, entity memory and knowledge collections are all vector-backed and therefore all affected.
- Why is the equal-dimension case more dangerous than a dimension mismatch?Because nothing errors. The store happily accepts vectors of the right width, so the mixed collection looks healthy while queries in the new model's space match old entries near-randomly. Retrieval quality degrades with no exception, no log line and no failed health check, and the investigation usually starts at the prompt rather than at the embedding config.
- How would you roll out an embedder change with a rollback path?Point CREWAI_STORAGE_DIR at a new location for the new embedder and let that store index from scratch, leaving the old directory intact. You can compare quality between the two, and rolling back is flipping one environment variable rather than re-embedding an entire corpus under incident pressure.
saying these in an interview costs you the question
- Says any embedder works with an existing store
- Assumes a mismatch always raises a clear error
- Blames the prompt when retrieval quality suddenly drops
- Deletes the whole storage directory without budgeting re-indexing cost
- Thinks matching vector dimensions means matching semantics