What makes a looping Haystack pipeline terminate, and what caps a runaway loop?

seniorshould knowfreq 42%

answer

Graphs here may contain cycles
Routing decides when to leave
There is a per-component execution cap
Default is generous, not a budget
Overrun aborts rather than returning

basics

~20 s

A loop terminates because a router inside it has an exit route that eventually fires. As a backstop, Pipeline(max_runs_per_component=...) caps how many times any single component may run in one call — default 100 — and exceeding it aborts the run with an error instead of spinning forever.

solid answer

~50 s

Haystack pipelines may contain cycles, and the canonical shape is validate-and-retry: a joiner accepts both the first input and the feedback edge, a generator produces a candidate, a validator checks it, and a router either emits on the exit socket or sends the failure back into the joiner with an error message appended. The loop ends when the router takes the exit route — that is the real termination condition, and it is your job to make it reachable. The framework's backstop is max_runs_per_component on the Pipeline constructor (default 100 in current 2.x/3.x): it counts executions per component within a single run(), and when a component exceeds it the whole run aborts with an error naming that component rather than silently returning a half-answer. Set it low — three to five — for LLM loops, because every extra pass is another paid call and another slice of your latency budget.

Haystack pipelines may contain cycles, and the canonical shape is validate-and-retry: a joiner accepts both the first input and the feedback edge, a generator produces a candidate, a validator checks it, and a router either emits on the exit socket or sends the failure back into the joiner with an error message appended. The loop ends when the router takes the exit route — that is the real termination condition, and it is your job to make it reachable. The framework's backstop is `max_runs_per_component` on the `Pipeline` constructor (default 100 in current 2.x/3.x): it counts executions per component within a single `run()`, and when a component exceeds it the whole run aborts with an error naming that component rather than silently returning a half-answer. Set it low — three to five — for LLM loops, because every extra pass is another paid call and another slice of your latency budget.

code

python · 10 lines

python

from haystack import Pipeline
from haystack.components.joiners import BranchJoiner

# cap each component at 3 executions per run() call
pipe = Pipeline(max_runs_per_component=3)

# the joiner is what lets the loop's first component take two senders:
# the initial input and the feedback edge coming back from the router
pipe.add_component("loop_entry", BranchJoiner(str))
print(pipe.get_component("loop_entry"))

go deeper

for a junior

Know that a Haystack pipeline may contain a loop and that the Pipeline constructor takes max_runs_per_component as a safety limit on how often one component runs.

for a middle

Explain the loop shape: a variadic joiner at the entrance, work, a validator, and a router whose exit route ends the cycle — and that exceeding the cap raises rather than returning.

for a senior

Treat the cap as a cost budget: three to five passes for an LLM correction loop, expensive deterministic work kept outside the cycle, and the cap-exceeded error handled as a real fallback path in the caller.

for a principal

Decide where the retry lives. In-graph loops belong to pure data feedback; anything needing rate limiting, model escalation or circuit breaking is better run from calling code that can see the whole request budget.

Cycles are legal

The graph is not required to be acyclic. connect() will happily close a loop, and the scheduler runs a component again whenever fresh inputs arrive on it. This is what makes self-correction expressible as wiring rather than as a Python while outside the pipeline.

The canonical loop shape

A validate-and-retry loop has four roles:

A joiner at the loop entrance. The prompt builder needs input from two places: the initial request and the feedback edge coming back from the validator. A plain input socket takes one connection, so a variadic joiner sits at the entrance and forwards whichever arrived. Without it you cannot even wire the cycle.
The work. A prompt builder and a generator produce a candidate — structured JSON, SQL, a plan.
A validator. A custom component that parses the candidate and either passes it through or produces an error description. Its declared output sockets are what the router branches on.
A router with an exit route. One route emits the accepted value out of the loop; another feeds the candidate plus the error text back to the joiner so the next prompt contains the complaint.

Each pass is a full LLM call. The loop is a retry mechanism with a learning step — the error text goes into the next prompt — which is the only reason it beats a plain retry.

What actually stops it

The honest answer is: the router. Termination is a property of your routing logic, not of the framework. Two failure modes recur:

The exit condition can never be true (a Jinja expression referencing a field the validator does not emit, so the exit route is dead wiring).
The model deterministically produces the same invalid output, so the feedback edge fires forever.

Both would run without limit, which is why the framework has a cap.

max_runs_per_component

Pipeline(max_runs_per_component=5) sets, per pipeline instance, how many times any single component may execute within one run() call. The counter is per component and resets for the next run(). Default is 100 in current versions — fine as an infinite-loop guard, far too high as an LLM budget. When a component exceeds the cap, the run aborts with an error identifying the component; it does not quietly return the best-so-far result. That is the right default for correctness (you never ship a half-validated answer as if it were fine) and it means your calling code must handle the exception as a real outcome: fall back to a canned response, return the last candidate with a warning, or surface a 503.

In earlier 2.x releases the equivalent knob was named max_loops_allowed; if you meet that spelling you are reading old material. Answers here assume haystack-ai 2.5 or newer, including 3.0.

Choosing the number

For an LLM correction loop, three to five is the usual band. Reason about it as a budget rather than a safety net: with three passes you have paid three generations and three validations before failing, and the p99 latency of the endpoint is roughly three times the single-pass latency. If your validator's pass rate on the second attempt is low, more passes rarely help — the model is failing systematically, and the fix is prompt or schema work, not a bigger cap.

Also remember the cap applies to every component, not just the ones in the cycle. A retriever placed inside the loop is re-queried on every pass and burns the same counter; keeping expensive, deterministic work outside the loop is both cheaper and clearer.

Observability

A loop makes the run non-linear, so "what happened" is a real question. The validator's output is the one you want on a bad run — name it in include_outputs_from to see the errors that were fed back. A component-level callback or a tracing integration gives you per-pass timing. And pipe.draw() is worth running once on any cyclic pipeline: a loop you cannot see in the diagram is a loop nobody will maintain.

When not to loop in the graph

If the retry logic needs to touch anything outside the pipeline — a rate limiter, a different model on the second attempt, a circuit breaker, a queue — running the pipeline twice from calling code is usually simpler than encoding all of that in edges. In-graph loops are best when the whole feedback cycle is data flowing between components, which is exactly the case for schema validation and self-critique.

## Cycles are legal The graph is not required to be acyclic. `connect()` will happily close a loop, and the scheduler runs a component again whenever fresh inputs arrive on it. This is what makes self-correction expressible as wiring rather than as a Python `while` outside the pipeline. ## The canonical loop shape A validate-and-retry loop has four roles: 1. **A joiner at the loop entrance.** The prompt builder needs input from two places: the initial request and the feedback edge coming back from the validator. A plain input socket takes one connection, so a variadic joiner sits at the entrance and forwards whichever arrived. Without it you cannot even wire the cycle. 2. **The work.** A prompt builder and a generator produce a candidate — structured JSON, SQL, a plan. 3. **A validator.** A custom component that parses the candidate and either passes it through or produces an error description. Its declared output sockets are what the router branches on. 4. **A router with an exit route.** One route emits the accepted value out of the loop; another feeds the candidate plus the error text back to the joiner so the next prompt contains the complaint. Each pass is a full LLM call. The loop is a retry mechanism with a *learning* step — the error text goes into the next prompt — which is the only reason it beats a plain retry. ## What actually stops it The honest answer is: the router. Termination is a property of your routing logic, not of the framework. Two failure modes recur: - The exit condition can never be true (a Jinja expression referencing a field the validator does not emit, so the exit route is dead wiring). - The model deterministically produces the same invalid output, so the feedback edge fires forever. Both would run without limit, which is why the framework has a cap. ## max_runs_per_component `Pipeline(max_runs_per_component=5)` sets, per pipeline instance, how many times any single component may execute within one `run()` call. The counter is per component and resets for the next `run()`. Default is 100 in current versions — fine as an infinite-loop guard, far too high as an LLM budget. When a component exceeds the cap, the run **aborts with an error identifying the component**; it does not quietly return the best-so-far result. That is the right default for correctness (you never ship a half-validated answer as if it were fine) and it means your calling code must handle the exception as a real outcome: fall back to a canned response, return the last candidate with a warning, or surface a 503. In earlier 2.x releases the equivalent knob was named `max_loops_allowed`; if you meet that spelling you are reading old material. Answers here assume haystack-ai 2.5 or newer, including 3.0. ## Choosing the number For an LLM correction loop, three to five is the usual band. Reason about it as a budget rather than a safety net: with three passes you have paid three generations and three validations before failing, and the p99 latency of the endpoint is roughly three times the single-pass latency. If your validator's pass rate on the second attempt is low, more passes rarely help — the model is failing systematically, and the fix is prompt or schema work, not a bigger cap. Also remember the cap applies to *every* component, not just the ones in the cycle. A retriever placed inside the loop is re-queried on every pass and burns the same counter; keeping expensive, deterministic work outside the loop is both cheaper and clearer. ## Observability A loop makes the run non-linear, so "what happened" is a real question. The validator's output is the one you want on a bad run — name it in `include_outputs_from` to see the errors that were fed back. A component-level callback or a tracing integration gives you per-pass timing. And `pipe.draw()` is worth running once on any cyclic pipeline: a loop you cannot see in the diagram is a loop nobody will maintain. ## When not to loop in the graph If the retry logic needs to touch anything outside the pipeline — a rate limiter, a different model on the second attempt, a circuit breaker, a queue — running the pipeline twice from calling code is usually simpler than encoding all of that in edges. In-graph loops are best when the whole feedback cycle is data flowing between components, which is exactly the case for schema validation and self-critique.

When the cap is hit, does the pipeline return its best result so far?
No — the run aborts with an error naming the component that exceeded its cap. That is deliberate: a validate-and-retry loop that ran out of attempts has, by definition, no validated answer, and returning the last invalid candidate as a normal result would hide the failure. Your calling code should treat the exception as a first-class outcome and fall back explicitly.
Why does a loop need a joiner at its entrance?
Because the first component in the cycle receives input from two senders — the initial pipeline input and the feedback edge from the router — and a plain input socket accepts exactly one connection. A variadic joiner collapses both into a single edge. Without it you cannot even wire the cycle; connect() rejects the second connection.
What does putting a retriever inside the loop cost you?
Every pass re-runs it, burning its own share of the per-component cap and adding a query plus latency to each attempt — usually for identical results, since the underlying query rarely changes between correction attempts. Keep deterministic, expensive work upstream of the joiner and let only the generate-validate-route segment cycle.

saying these in an interview costs you the question

Says Haystack pipelines must be acyclic
Thinks max_runs_per_component returns the last result
Reads the cap as a total component count
Leaves the default 100 as an LLM retry budget
Believes the framework detects non-terminating loops itself

What makes a looping Haystack pipeline terminate, and what caps a runaway loop?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Cycles are legal

The canonical loop shape

What actually stops it

max_runs_per_component

Choosing the number

Observability

When not to loop in the graph

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions3 questions

▾related topics3 topics

Cycles are legal

The canonical loop shape

What actually stops it

max_runs_per_component

Choosing the number

Observability

When not to loop in the graph

answer

go deeper

context