skip to content

Design a reliable event-publishing mechanism that fixes the dual-write gap left by AFTER_COMMIT. Explain the transactional outbox pattern, its delivery semantics, and how it relates to Spring Modulith's event publication registry.

level: principalimportance: should knowfreq 45%

answer

  1. INSERT outbox row in the same tx as the data
  2. relay: poll (SKIP LOCKED) or CDC/Debezium
  3. at-least-once ⇒ idempotent consumers
  4. avoid XA/2PC — local tx only
  5. Modulith event_publication table = in-process outbox

basics

~20 s

Write the event into an outbox table in the same transaction as the business data, so both commit atomically. A separate relay reads the outbox and publishes to the broker, marking rows done and retrying failures. This gives at-least-once delivery; consumers must be idempotent. Spring Modulith's event publication registry is an in-process version of this.

solid answer

~50 s

The outbox pattern replaces the non-atomic 'write DB then publish' with a single atomic DB write plus reliable async delivery. In the business transaction you INSERT the message into an outbox table alongside the domain change, so they commit or roll back together — no dual write. A separate relay then reads unpublished outbox rows and sends them to the broker, marking each published (or leaving it for retry). Because the relay can crash after publishing but before marking done, delivery is at-least-once, so consumers must be idempotent (dedupe by message id). The relay is either a poller (SELECT unpublished rows on a schedule) or CDC-based (Debezium tailing the DB log for lower latency). Spring Modulith implements exactly this for in-process events: @ApplicationModuleListener events are stored in an event_publication table in the same transaction; completed ones are marked, and incomplete publications are republished on startup — an outbox for module-to-module events. For external brokers you add a CDC or polling relay over your own outbox table.

code

java · 19 lines
java
// 1) Business tx writes data + outbox row atomically
@Transactional
public void place(Order o) {
    orderRepository.save(o);
    outboxRepository.save(new OutboxMessage(
        UUID.randomUUID(), "Order", o.getId(),
        "OrderPlaced", toJson(o), OutboxStatus.NEW));
} // both rows commit together -- no dual write

// 2) A separate relay publishes and marks sent (retries on failure)
@Scheduled(fixedDelay = 1000)
@Transactional
public void relay() {
    // SKIP LOCKED lets multiple relay instances run without contention
    for (OutboxMessage m : outboxRepository.lockNextBatch(100)) {
        kafkaTemplate.send("orders", m.getId().toString(), m.getPayload());
        m.markSent(); // if we crash before this, the row is re-sent later
    }                // -> at-least-once; consumers dedupe by m.getId()
}

go deeper

for a junior

Not expected to design this; awareness that a durable table can back reliable delivery is a bonus.

for a middle

Can describe the outbox at a high level but may miss at-least-once/idempotency and relay options.

for a senior

Explains atomic write + relay + idempotent consumers and why XA is avoided.

for a principal

Full design: relay strategies (poll vs CDC), ordering, pruning, delivery semantics, and mapping to Spring Modulith's event publication registry and @Externalized events.

## The problem being solved AFTER_COMMIT stops phantom events but leaves the **dual-write** gap: commit succeeds, then the publish is lost to a crash/outage. The **transactional outbox** pattern removes the dual write entirely. ## Core idea: make the intent-to-publish part of the transaction Instead of writing to the DB and separately to the broker, you write **only to the DB** — twice, atomically: 1. The business change (e.g. `INSERT order`). 2. A row in an **outbox** table describing the message to send (e.g. `INSERT outbox(id, aggregate, type, payload, status='NEW')`). Both are in the **same local transaction**, so they commit or roll back **together**. There is now no moment where the order exists but the intent to publish does not. The dual write has become a **single atomic write**. ## The relay (message relay / dispatcher) A separate process/thread moves outbox rows to the real broker: - **Polling publisher** — periodically `SELECT * FROM outbox WHERE status='NEW'`, publish each to Kafka/Rabbit, then `UPDATE status='SENT'` (or delete). Simple, DB-agnostic; latency = poll interval; watch for hot-table contention (use `SKIP LOCKED`, indexes, batching). - **CDC / transaction-log tailing** — a tool like **Debezium** reads the database's write-ahead log and streams outbox inserts to Kafka. Lower latency, no polling load, but more infrastructure. ## Delivery semantics: at-least-once The relay itself can fail between **publishing** and **marking the row sent**: ``` publish(msg) // succeeds <crash> UPDATE status='SENT' // never runs -> row reprocessed -> duplicate sent ``` So the outbox gives **at-least-once** delivery, not exactly-once. **Consumers must be idempotent** — deduplicate by the message's unique id, or make handlers naturally idempotent (upserts). Exactly-once end-to-end across systems is generally impractical; at-least-once + idempotency is the pragmatic target. ## Ordering and multiplicity - Preserve order per aggregate if consumers need it (sequence column, single partition key = aggregate id). - Clean up sent rows (archival/TTL) to keep the table small. ## Why not XA/2PC? A distributed transaction spanning DB + broker would make the two writes atomic, but XA is heavyweight, has fragile recovery, throttles throughput, and Kafka in particular is a poor XA participant. The outbox achieves the needed guarantee using only **local** transactions — which is why it's the industry default. ## Spring Modulith's event publication registry Spring Modulith gives you an outbox for **in-process, module-to-module** events out of the box: - An event handled by `@ApplicationModuleListener` (= `@TransactionalEventListener` AFTER_COMMIT + `@Async` + `@Transactional` REQUIRES_NEW) has a corresponding row written to an **`event_publication`** table **within the original transaction**. - When the listener completes successfully, the publication is **marked completed**. - On application **restart**, **incomplete** publications (listener never finished) are **resubmitted**, so an event is not lost to a crash between commit and handling. This is precisely outbox semantics — durable intent + retry — scoped to the modular monolith. For **cross-service** delivery to an external broker you still layer a CDC/polling relay over an outbox table (Modulith can externalize events via `@Externalized`, backed by broker integrations). ## Design checklist 1. Outbox table written in the business transaction (no separate connection). 2. Idempotent consumers keyed on message id (at-least-once). 3. Choose relay: polling (simple) vs CDC/Debezium (low latency). 4. Handle ordering per aggregate if required. 5. Prune/archive delivered rows. 6. In a Spring Modulith monolith, reuse the event publication registry rather than hand-rolling. ## One-sentence summary Make the decision to publish part of the same transaction as the data, then deliver it reliably and asynchronously — trading dual-write inconsistency for at-least-once delivery plus idempotent consumers.

  • Your outbox gives at-least-once delivery. How do consumers avoid processing duplicates?
    Make consumers idempotent: dedupe on the message's unique id (a processed-ids table or unique constraint), or design handlers as natural upserts so reprocessing is harmless. Exactly-once end-to-end is impractical, so at-least-once + idempotency is the standard.
  • Polling relay vs Debezium/CDC — how do you choose?
    Polling is simplest and DB-agnostic but adds query load and latency equal to the poll interval; use SKIP LOCKED and indexes. CDC (Debezium tailing the WAL) gives near-real-time latency with no polling load but adds operational complexity and infrastructure. Choose CDC when latency/scale justify the extra moving parts.
  • How does Spring Modulith's event publication registry differ from a broker outbox?
    The registry is an in-process outbox for module-to-module @ApplicationModuleListener events, persisted in an event_publication table and resubmitted on restart. It reliably delivers within the monolith; for external broker delivery you still add a CDC/polling relay (or use Modulith's @Externalized event support).

saying these in an interview costs you the question

  • Claiming the outbox gives exactly-once delivery (it's at-least-once; consumers must be idempotent).
  • Writing the outbox row on a separate connection/transaction — that reintroduces the dual write.
  • Proposing XA/2PC as the standard fix without noting its practical drawbacks.
  • Forgetting a relay/retry mechanism, or forgetting to prune sent rows.
  • Not knowing Spring Modulith already provides an in-process outbox.

context

resolving links…