How do you handle a base-model upgrade for a compiled DSPy program in production?

## Why a model upgrade is a compile-invalidating event Compilation optimizes prompt state *against a specific model's behaviour*. Everything the optimizer discovered is a correction for how that model failed on your data: an instruction clause that stops it inventing catalogue identifiers, four demonstrations that teach a format it did not follow naturally, a phrasing that suppresses a particular verbosity. Change the model and those corrections are addressed to someone who is no longer in the room. The outcomes are not symmetrical, which is what makes this a judgment question rather than a procedure: - The artifact may still **help**, because most of it encodes task structure rather than model quirks. - It may become **neutral**, wasting tokens on instructions the new model no longer needs. - It may actively **hurt**. This is the case people miss. Demonstrations that scaffolded a weaker model can anchor a stronger one to a shallower answer style than it would produce unprompted, and defensive instruction clauses can suppress capability. Because all three are plausible, the answer is never "recompile" or "keep it" — it is "measure the three candidates." ## The pin A compiled artifact without provenance is unmanageable. Store, next to the artifact: the base model identifier and version, the optimizer and its configuration, the random seed, the sampling temperature used during compilation, the training-set version or hash, and the held-out score it achieved. That turns "why did quality drop last Tuesday" from an archaeology project into a diff. Note what pinning can and cannot buy you. It gives you a **fixed artifact** — the same prompts ship every time, which is the important half. It does *not* give you a bit-reproducible compile: the compile ran a stochastic model many times, and providers change model behaviour under a stable name. Recompiling with identical inputs yields a similar artifact, not the same one. Claiming full reproducibility here is a red flag; claiming a pinned, versioned output is correct. ## The upgrade procedure 1. **Freeze.** The currently-deployed artifact stays deployed while you evaluate. 2. **Evaluate the pinned artifact on the new model** using the held-out set. This is the cheapest experiment and it often answers the question outright. 3. **Recompile against the new model** with the same optimizer configuration and training set, producing a candidate artifact. 4. **Evaluate the uncompiled program on the new model** as the third arm. On a genuinely stronger model this sometimes wins, and discovering that saves you a recurring token bill. 5. **Compare on the quality-cost frontier.** Include prompt token count per request, latency and metric score. A compiled artifact that adds four demonstrations to every production call is a permanent cost line. 6. **Ship one, keep the loser.** Rollback means redeploying the previous artifact, which only works if you kept it. ## Budgeting the recompile The recompile bill is the part principals are expected to quantify rather than wave at. For a demonstration-only optimizer, the cost is roughly the training-set size times a few rollouts. For a joint instruction-and-demonstration optimizer like MIPROv2, the optimizer proposes instruction candidates for **each** predictor, pairs them with demonstration sets, and scores combinations on batches of training data — so cost scales with the number of predictors in the program, the number of candidates explored, and the evaluation-batch size, multiplied by however many trials the search runs. A two-module program with a fifty-item training set is inexpensive; a six-module program with a thousand-item set and a heavy search setting is a real budget item, and it recurs on every model upgrade. That recurrence is the strategic point. If your provider ships a new model quarterly, you have signed up for a quarterly recompile-and-evaluate cycle. That argues for keeping the training set small and high-quality, keeping the program shallow, and automating the three-arm comparison in CI rather than doing it by hand each time. ## Contested ground, stated honestly There is no settled industry consensus in 2026 on how much prompt optimization survives a model generation. Reported experience varies by task: structured extraction tends to transfer well, while instructions that patched a specific model's reasoning weakness often do not. The defensible position in an interview is not a number — it is the process: pin everything, hold out an evaluation set, always evaluate the uncompiled arm, and treat the artifact as disposable rather than precious.