A service updates its own database and then needs to publish an event so the next saga step can run. What is the 'dual write' problem this creates, and how does the transactional outbox pattern solve it?

## The dual-write problem The problem the outbox pattern solves starts from a very ordinary piece of code: a service handles a request, writes a row to its database, and then publishes an event to a message broker so downstream saga participants know to proceed. The trouble is that "write to the database" and "publish to the broker" are two entirely separate systems with no shared transaction manager between them. Either way, there is a gap: - **Commit first, publish second.** If the code commits the database write first and then the process crashes, or the broker is unreachable, before the publish call completes, the database now reflects a change that nobody downstream will ever hear about — the saga silently stalls. - **Publish first, commit second.** If the code publishes first and the subsequent database commit then fails or the process crashes before committing, downstream services react to an event describing a state change that never actually happened. Retrying doesn't fully fix this either: if you publish first and then write, and the write fails, you can't simply un-publish the event once other services may have already consumed it. This is the **"dual write"** problem — two writes to two independent systems that need to be atomic together but have no way to be, because a two-phase commit across a database and a message broker is exactly the kind of distributed transaction the saga pattern exists to avoid in the first place. ## What the outbox makes atomic The **transactional outbox pattern** resolves this by changing what actually gets committed atomically. Instead of writing to the business table and calling the broker, the service writes to the business table and inserts a row into a second table — the **outbox** — in the same local database transaction. The outbox row typically contains: - the event type; - a serialized payload; - metadata like an aggregate ID and timestamp. Because both writes are part of one ACID transaction against a single database, the database's own commit/rollback guarantee makes them atomic: either the business row and the outbox row are both persisted, or neither is. This turns a cross-system atomicity problem into an ordinary single-database transaction, which every relational (and many NoSQL) database already knows how to do reliably. ## Getting the event onto the broker Getting the event out of the outbox table and onto the broker is a second, separate concern, handled by a **relay** process. There are two common implementations. | Relay | Mechanism | Trade-off | |---|---|---| | Polling — the simpler one | a background job periodically queries the outbox table for unsent rows (ordered by insertion time or an autoincrement ID), publishes each to the broker, and marks it sent or deletes it | cheap to build, but it adds polling latency and load on the table | | Change-data-capture (CDC) — the more robust one | a tool like Debezium tails the database's write-ahead log (the same mechanism the database uses internally for replication) and streams new outbox rows to the broker without querying the table at all | giving lower latency and less load, at the cost of an extra infrastructure component (Kafka Connect plus Debezium, or an equivalent) that itself needs to be operated and monitored | ## At-least-once, not exactly-once The trade-off this introduces is that the relay only guarantees at-least-once delivery, not exactly-once: if the relay crashes after publishing to the broker but before marking the outbox row as sent, it will republish the same event on restart. Consumers of these events must therefore be idempotent — able to safely process the same event twice without double-effects, typically by tracking already-processed event IDs (this is where the companion **inbox pattern** comes in on the consumer side). The outbox table also needs housekeeping — old sent rows must be purged or archived — or it grows unbounded. ## Failure modes Failure modes show up as either: - **"ghost" delays** — a stuck relay means events pile up in the outbox and downstream saga steps stall, even though the source data is correct; - **duplicate processing** — a relay retry after a network blip between it and the broker causes a consumer to see the same event twice, and if that consumer isn't idempotent, it double-charges or double-reserves. Because the outbox row and the business row commit together, an operator can always audit the outbox table against the business table to confirm no dual-write gap exists — that auditability is one of the pattern's practical benefits beyond just correctness. ## A reference implementation A concrete widely used example is Debezium's outbox event router, purpose-built to read outbox rows from services (written with plain JDBC/JPA) and republish them onto Kafka topics per aggregate type, which is the reference implementation many teams start from rather than hand-rolling their own polling relay.