In DSPy, what changes when you swap Predict for ChainOfThought on a step?

middleshould knowfreq 34%

answer

signature is the interface
module is the implementation
one line, then recompile
extra reasoning output field
tokens and latency move too

basics

~20 s

Swapping the module keeps the same signature but changes how that step is executed: ChainOfThought extends the declared outputs with a reasoning field the model fills before answering. Calling code, training data and metric are untouched — you change one line and recompile.

solid answer

~50 s

Modules are the implementations that sit behind a signature. dspy.Predict asks for the declared outputs directly; dspy.ChainOfThought takes the same signature and extends it with an extra reasoning output that the model produces before the declared fields; dspy.ReAct runs a tool loop against the signature until it can fill them. Because all three accept the same declaration, swapping one for another is a one-line edit — the field names your program reads do not move, the training examples stay valid, and the metric is unchanged. What does change is the prompt shape, so any demonstrations a previous compile attached were selected for the old shape. The honest workflow is: swap the line, recompile, evaluate both on a held-out set, and keep whichever wins. Reasoning modules cost more tokens and latency per call, so the comparison must be quality against cost, not quality alone.

Modules are the *implementations* that sit behind a signature. `dspy.Predict` asks for the declared outputs directly; `dspy.ChainOfThought` takes the same signature and extends it with an extra reasoning output that the model produces before the declared fields; `dspy.ReAct` runs a tool loop against the signature until it can fill them. Because all three accept the same declaration, swapping one for another is a one-line edit — the field names your program reads do not move, the training examples stay valid, and the metric is unchanged. What does change is the prompt shape, so any demonstrations a previous compile attached were selected for the old shape. The honest workflow is: swap the line, recompile, evaluate both on a held-out set, and keep whichever wins. Reasoning modules cost more tokens and latency per call, so the comparison must be quality against cost, not quality alone.

code

python · 11 lines

python

import dspy

class ArchiveQA(dspy.Module):
    def __init__(self, search):
        super().__init__()
        self.search = search
        self.answer = dspy.Predict("context, question -> answer")

    def forward(self, question):
        context = self.search(question, k=5)
        return self.answer(context=context, question=question)

go deeper

for a junior

Know that Predict, ChainOfThought and ReAct all accept the same signature, so changing between them is a one-line edit rather than a rewrite of your prompts.

for a middle

Explain what the swap actually moves — the rendered prompt gains a reasoning output field — and what it leaves alone: field names, training examples and the metric.

for a senior

Demonstrate the discipline: recompile after the swap because old demonstrations were bootstrapped for the old shape, then compare on a held-out set with token cost and latency in the comparison.

for a principal

Frame it as interface-versus-implementation across a whole pipeline, and be willing to say that reasoning modules are a measured bet, not a default upgrade, on lookup-shaped or latency-sensitive steps.

Signature versus module

A signature says what a step takes and returns. A module says how that step is carried out. Keeping those separate is the reason a DSPy program can change its reasoning strategy without a refactor.

The three modules people actually name in interviews are:

dspy.Predict — the baseline. It renders the signature and asks the model for the declared output fields, nothing more.
dspy.ChainOfThought — takes the same signature and extends it with an additional reasoning output field, so the model writes its working before the declared outputs. In current DSPy (3.x) that field is called reasoning; older releases named it rationale.
dspy.ReAct — takes a signature plus a list of tools and runs an interleaved loop of thought, tool call and observation until it can produce the declared outputs, bounded by an iteration cap.

All three consume the same declaration. That is the whole trick.

What the swap actually changes

Four things stay fixed and one thing moves.

Fixed: the field names downstream code reads, so pred.answer still resolves. The training set, because examples are keyed to the declared input and output fields, not to the module. The metric, for the same reason. And the rest of the program — sibling modules do not know or care which implementation this step uses.

Moved: the prompt that gets rendered, and therefore the behaviour, the token count and the latency. A reasoning module emits a block of intermediate text on every call, which you pay for on every request in production, not just during development.

Composition is the reason this matters

A single-step swap is a small convenience. The payoff appears in a multi-step program — say a catalogue pipeline that retrieves passages and then answers from them. That program is a module too: it holds sub-modules as attributes and calls them in its forward method, exactly like a small neural-network module holding layers. Because the parent is itself a module, an optimizer can compile the whole thing end to end: it treats every predictor inside the program as a tunable site and tunes them together against one metric on the final output.

So when you decide the answering step needs explicit reasoning, you change its constructor line and recompile the program. You do not touch the retrieval step, its field names, or the metric — and the optimizer re-derives demonstrations for both steps under the new shape.

Why you must recompile

This is the part candidates skip. Demonstrations attached by a previous compile were selected while the step had the old prompt shape. After a swap, those demonstrations are stale in a specific way: they were bootstrapped without a reasoning field, so they no longer show the model the format it is now being asked to produce. Reusing them is not catastrophic, but it forfeits the reason you compiled. Treat a module swap as a change that invalidates the artifact, the same way changing a source file invalidates a compiled binary.

When reasoning does not help

The interviewer's real question is usually judgment, not mechanics. Adding a reasoning module is not a free upgrade:

On lookup-shaped tasks — extract a field, classify into three buckets, copy an identifier out of a passage — a reasoning preamble often adds tokens and nothing else, and can hurt by giving the model room to talk itself out of the obvious answer.
On models that already reason internally, an explicit reasoning field can duplicate work you are already paying for.
On latency-sensitive paths, the extra output is felt directly by the user.
And the benefit is task-dependent enough that you cannot predict it — which is precisely why the swap-and-recompile loop exists. You measure.

The interview-ready summary

Say it as an interface story: the signature is the interface, the module is the implementation, and the compile step is what fills in the wording for whichever implementation is currently plugged in. Then add the cost sentence — you compare reasoning against direct prediction on a held-out set with cost and latency in the comparison, because a one-line swap is cheap to try and expensive to leave in unmeasured.

## Signature versus module A signature says *what* a step takes and returns. A module says *how* that step is carried out. Keeping those separate is the reason a DSPy program can change its reasoning strategy without a refactor. The three modules people actually name in interviews are: - **`dspy.Predict`** — the baseline. It renders the signature and asks the model for the declared output fields, nothing more. - **`dspy.ChainOfThought`** — takes the same signature and extends it with an additional reasoning output field, so the model writes its working before the declared outputs. In current DSPy (3.x) that field is called `reasoning`; older releases named it `rationale`. - **`dspy.ReAct`** — takes a signature plus a list of tools and runs an interleaved loop of thought, tool call and observation until it can produce the declared outputs, bounded by an iteration cap. All three consume the same declaration. That is the whole trick. ## What the swap actually changes Four things stay fixed and one thing moves. Fixed: the **field names** downstream code reads, so `pred.answer` still resolves. The **training set**, because examples are keyed to the declared input and output fields, not to the module. The **metric**, for the same reason. And the **rest of the program** — sibling modules do not know or care which implementation this step uses. Moved: the **prompt that gets rendered**, and therefore the behaviour, the token count and the latency. A reasoning module emits a block of intermediate text on every call, which you pay for on every request in production, not just during development. ## Composition is the reason this matters A single-step swap is a small convenience. The payoff appears in a multi-step program — say a catalogue pipeline that retrieves passages and then answers from them. That program is a module too: it holds sub-modules as attributes and calls them in its `forward` method, exactly like a small neural-network module holding layers. Because the parent is itself a module, an optimizer can compile the whole thing end to end: it treats every predictor inside the program as a tunable site and tunes them together against one metric on the final output. So when you decide the answering step needs explicit reasoning, you change its constructor line and recompile the program. You do not touch the retrieval step, its field names, or the metric — and the optimizer re-derives demonstrations for both steps under the new shape. ## Why you must recompile This is the part candidates skip. Demonstrations attached by a previous compile were selected while the step had the old prompt shape. After a swap, those demonstrations are stale in a specific way: they were bootstrapped without a reasoning field, so they no longer show the model the format it is now being asked to produce. Reusing them is not catastrophic, but it forfeits the reason you compiled. Treat a module swap as a change that invalidates the artifact, the same way changing a source file invalidates a compiled binary. ## When reasoning does not help The interviewer's real question is usually judgment, not mechanics. Adding a reasoning module is not a free upgrade: - On **lookup-shaped tasks** — extract a field, classify into three buckets, copy an identifier out of a passage — a reasoning preamble often adds tokens and nothing else, and can *hurt* by giving the model room to talk itself out of the obvious answer. - On **models that already reason internally**, an explicit reasoning field can duplicate work you are already paying for. - On **latency-sensitive paths**, the extra output is felt directly by the user. - And the benefit is **task-dependent enough that you cannot predict it** — which is precisely why the swap-and-recompile loop exists. You measure. ## The interview-ready summary Say it as an interface story: the signature is the interface, the module is the implementation, and the compile step is what fills in the wording for whichever implementation is currently plugged in. Then add the cost sentence — you compare reasoning against direct prediction on a held-out set with cost and latency in the comparison, because a one-line swap is cheap to try and expensive to leave in unmeasured.

After that swap, can you keep the demonstrations the previous compile produced?
Technically yes, but you should not rely on them. They were bootstrapped while the step had a different output shape — without the reasoning field — so they no longer demonstrate the format the model is now asked to produce. Recompile so the optimizer re-derives demonstrations under the new shape, then compare the new artifact against the old one on a held-out set before shipping.
Where does ReAct fit, given it takes the same signature?
ReAct is the implementation you choose when the step cannot be answered from its inputs alone and needs tools. It runs a bounded thought-action-observation loop and stops when it can fill the declared output fields. The interface is unchanged, so the swap is still one line, but the cost profile is far larger — multiple model calls plus tool latency per step — so it is a different order of decision from adding a reasoning field.
Does an optimizer tune a multi-module program one step at a time?
No — that is the point of compiling the program rather than the prompt. The optimizer scores the program's final output with a single metric and tunes every predictor inside it against that score, so improvements at one step are only kept if they help the end result. That also means a step can end up with demonstrations that look odd in isolation but serve the pipeline.

saying these in an interview costs you the question

Saying a reasoning module always improves accuracy
Assuming the swap requires editing the signature or the training data
Ignoring the extra tokens and latency a reasoning field costs per call
Reusing an old compiled artifact after changing a module's implementation
Describing modules as prompt templates rather than implementations of a signature

In DSPy, what changes when you swap Predict for ChainOfThought on a step?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Signature versus module

What the swap actually changes

Composition is the reason this matters

Why you must recompile

When reasoning does not help

The interview-ready summary

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions3 questions

▾related topics2 topics

Signature versus module

What the swap actually changes

Composition is the reason this matters

Why you must recompile

When reasoning does not help

The interview-ready summary

answer

go deeper

context