You're designing the communication for a saga spanning five services with compensating actions on failure. Under what conditions would you choose choreography versus orchestration, and what does each cost you in observability and evolvability as the system grows?

## The axis that actually decides it The choice between choreography and orchestration doesn't stay fixed as a system grows; the right answer at five services with two failure paths is often the wrong answer at fifteen services with a dozen failure paths, and reasoning about that trajectory is the actual principal-level question here, more than picking one pattern as universally correct. The axis that matters most is not the number of services involved but the **density and volatility of the failure and compensation logic**: - how many of the steps in the saga need a defined "undo" action; - how often does the sequence or its error handling change; - how many separate teams own pieces of that sequence. ## What choreography scales gracefully Choreography scales gracefully along one specific dimension: **adding participants** that react to existing events costs nothing to the services already in the flow. If five services already form a saga via events and a sixth team wants to add a new side effect triggered by one of those existing events, no existing service's code changes at all. This is a genuine, durable advantage as an organization grows, because it means the core saga's owning teams never become a bottleneck for other teams wanting to hook into the process. But this advantage specifically applies to independent side effects; it does not extend to changes in the core sequence or compensation logic itself. If the failure handling for the saga grows more complex, say a new rule that "if shipping fails after inventory is reserved, only partially refund and issue a store credit for the difference," that logic in choreography has to be implemented as new event subscriptions and handlers distributed across whichever services are involved in that specific failure path, and there is no single place to write, review, or test that rule as a coherent unit; it exists as an emergent property of several services' independently-deployed event handlers agreeing (hopefully) on the right behavior. ## What orchestration scales gracefully Orchestration scales gracefully along the opposite dimension: **complex, changing, branching failure and compensation logic**. Because the whole sequence, happy path and every compensation path, lives in one component, adding a new conditional branch to the saga, like the partial-refund rule above, means writing it once in the orchestrator's state machine, where it can be code-reviewed, unit-tested, and reasoned about as a whole. This becomes increasingly valuable as the number of failure paths grows, since the alternative, choreography, would require that same growing complexity to be correctly distributed and kept consistent across multiple independently-deployed services. What doesn't scale gracefully in orchestration is **participant growth from unrelated teams**: every team that wants their service to be part of the saga's sequence, even for something that doesn't need real compensation logic, has to get a change into the shared orchestrator, which means code review, deploy coordination, and shared on-call risk for a component that now sits in the critical path of a growing number of unrelated features. ## How observability degrades on each side Observability degrades differently under the two patterns as they grow. | Pattern | What growth does to it | |---|---| | **Choreographed** | Systems at five services are often still debuggable by reading five services' worth of event handlers, but at fifteen services, understanding "what is supposed to happen when an order is placed" stops being something any one engineer can hold in their head, and teams typically respond by building dedicated tooling: an event-sourced saga tracker that subscribes to everything purely to reconstruct state, or heavy investment in distributed tracing with consistent correlation IDs propagated through every event, essentially reconstructing, at significant tooling cost, the single-source-of-truth view that orchestration gives you for free. | | **Orchestrated** | Systems keep a clear state machine view as they grow, but the orchestrator's own internal complexity grows instead, its state machine can become a sprawling, deeply-nested set of conditional branches that's readable in principle but increasingly hard to safely modify, and its blast radius as a single component grows too: an orchestrator outage or bug doesn't degrade one step of the process, it can halt every in-flight saga across the whole system, which is a materially larger failure domain than any single choreography participant going down. | ## The hybrid mature organizations converge on What mature organizations converge on, and this is the practical resolution most principal engineers land on, is a **hybrid**: - **orchestration** for the genuinely transactional core of the saga, the steps that need real compensation, ordering guarantees, and a single source of truth for "is this order in a consistent state"; - **choreography** for everything hanging off that core that's a true independent side effect with no compensation needs, like notifications, analytics, and recommendation-model updates. Concretely, an orchestrator might own payment, inventory, and shipping, including their compensating actions, and simply publish `OrderFulfilled` or `OrderFailed` events at the end, which any number of unrelated services can then choreograph off of without ever needing to touch or even be known to the orchestrator. This gets the observability and correctness benefits of orchestration exactly where compensation complexity actually lives, while preserving choreography's frictionless extensibility exactly where it's genuinely low-risk to do so, and it's the pattern you'll see described in most serious writing on saga design (including the standard saga-pattern literature that treats choreography and orchestration as implementation strategies for the same underlying pattern, not competing architectures) once a system has grown past the point where one approach cleanly fits the whole workflow.