How do you migrate a production vector index to a new OpenAI embedding model?

## Why this is a migration and not a config change An embedding model defines a coordinate system. Two models trained separately place "quarterly revenue" at unrelated points, and there is no published transform between OpenAI's embedding spaces — no projection matrix, no adapter. The same applies to a change of the `dimensions` parameter within one model: a 1536-wide vector and a 3072-wide vector cannot even be compared, since cosine similarity is undefined across differing lengths. So the moment you change the model or the width, every stored vector becomes dead weight. This is what makes the embedding model choice one of the stickier decisions in a retrieval system: it is cheap to make and expensive to revisit. ## The failure mode you are avoiding The dangerous version of this migration is the incremental one. A team switches the model in the ingestion code, deploys, and new documents start arriving as `text-embedding-3-large` vectors in an index full of `ada-002` vectors. Nothing errors — the dimensions may even match if widths coincide. Queries embedded with the new model retrieve only the new documents and score the old ones near-randomly; queries embedded with the old model do the reverse. Recall degrades quietly, users report "the search got worse", and because there is no exception anywhere the cause takes days to find. The structural defence is metadata: every vector row carries the model identifier and the dimension width, ingestion asserts they match the index's declared configuration, and a mismatch is a hard failure at write time rather than a soft failure at query time. ## The dual-index playbook **1. Decide with evidence, not novelty.** Before committing, embed a labelled evaluation set — queries paired with the chunks that should be retrieved — under both configurations and compare recall@k and MRR. Newer and larger is not automatically better on a specific corpus, and if the gain is inside the noise, the correct decision is not to migrate. **2. Provision a parallel index.** Separate namespace, collection or table, configured for the new width and metric. Do not mutate the live one. **3. Backfill offline.** Submit the corpus through the Batch API rather than the synchronous endpoint, so a multi-million-chunk job runs against separate queues and cannot consume the rate-limit budget your live query path depends on. Content-hash keys make the job resumable and idempotent. **4. Dual-write new documents.** From the moment the backfill starts, ingest writes to both indexes. Otherwise documents created during the backfill window exist in the old index only, and you cut over to a stale corpus. **5. Shadow-read before cutting over.** Route a sample of real production queries to both indexes, log the top-k from each, and compare — offline evaluation sets are small and synthetic, real query distributions are neither. This is also where you catch latency regressions from a wider vector. **6. Cut over behind a flag,** with the old index still populated and warm. A flag flip is an instant rollback; a deleted index is not. **7. Delete last.** Keep the old index for a defined soak period — long enough to be confident, short enough that you are not paying for two copies forever. ## Costs to put on the table before starting - **Re-embedding tokens.** Knowable exactly in advance: tokenise the corpus once and multiply by the model's rate. There is no output-token component. - **Double storage** for the duration. At 3072 dimensions and float32 a vector is 12 KB; ten million chunks held twice is real money in a memory-resident index. - **Wall-clock time.** The Batch API's completion window is up to 24 hours per job, and a large corpus may take several. Plan the migration in days, not hours. - **Ongoing serving cost** if the new width is larger: more index memory and more work per query, permanently. ## The judgment an interviewer is probing Three things. First, whether you understand that vector spaces are model-specific and that no conversion exists — the technical core. Second, whether you reach for a dual-index, shadow-read, flag-controlled cutover rather than an in-place mutation, which is generic migration discipline applied to a new substrate. Third, whether you insist on measuring before and after: the entire premise of the migration is that retrieval improves, and a migration that ships without a recall comparison has not demonstrated its own reason for existing. The strongest answers also mention the preventative half — recording model and width alongside every vector, and asserting them on write — because that is what turns this from an archaeology project into a scheduled job.