Transactional Outbox and Change Data Capture
The transactional outbox pattern and change data capture: avoiding dual writes, publishing events reliably, and Debezium-style CDC.
Services often need to update their database and publish an event about it: save the order, then tell the world OrderPlaced. Doing these as two separate steps (a dual write) is unsafe, because a crash or failure between them leaves the database and the event stream disagreeing. The transactional outbox pattern and change data capture (CDC) solve this by making the event a product of the database write itself.
Why dual writes break
| Order of steps | Failure between steps | Result |
|---|---|---|
| Write DB, then publish | Crash or broker down after the DB commit | Order exists, but no service hears about it: no email, no shipment |
| Publish, then write DB | DB write fails or rolls back | Services react to an order that doesn't exist |
| Both, without a transaction | Any partial failure | Silent, permanent inconsistency that's hard to detect |
You can't put a database and a message broker in one ordinary transaction (that would need two-phase commit, which most brokers don't support and which couples their availability).
The transactional outbox
Write the event into an outbox table in the same database transaction as the business change. A separate relay reads the outbox and publishes to the broker, marking or deleting rows once published.
BEGIN;
INSERT INTO orders (id, user_id, total, status) VALUES ('o_981', 'u_42', 1499, 'placed');
INSERT INTO outbox (id, aggregate_id, type, payload)
VALUES ('evt_5521', 'o_981', 'OrderPlaced', '{"order_id":"o_981","total":1499}');
COMMIT;- Guarantee: if the order is committed, the event will eventually be published; if the transaction rolls back, there's no event.
- Delivery is at-least-once: the relay can publish and then crash before marking the row, so it republishes. Consumers must be idempotent, using the event ID.
- Ordering: publish in outbox order per aggregate (key the topic by
aggregate_id) to keep events for one entity in order. - Cleanup: delete or archive published rows so the table stays small.
Relay options: polling vs CDC
| Polling publisher | Change data capture | |
|---|---|---|
| How | Query SELECT … FROM outbox WHERE published = false every few hundred ms | Read the database's replication log (Postgres WAL, MySQL binlog) |
| Latency | Polling interval | Near real time |
| DB load | Repeated queries | Minimal: reads the log the database writes anyway |
| Tools | A small background job | Debezium, AWS DMS, Postgres logical replication |
Change data capture on its own
CDC can also stream changes from any table, not just an outbox: every insert, update and delete becomes an event. That's ideal for keeping derived systems in sync (search indexes, caches, data warehouses) without the application doing anything. The trade-off is that table-level change events expose your internal schema to consumers; an outbox lets you publish deliberate, stable domain events instead.
Rule of thumb: use the outbox for events other services depend on (a public contract), and raw CDC for replicating data into your own derived stores like search and analytics.
Where it’s used
Open-source CDC that tails database logs into Kafka, with a dedicated outbox event router for the outbox pattern.
CDC streams product and user changes into Elasticsearch and cache invalidation, avoiding app-level dual writes. See full-text search.
Use outbox tables so domain events are published exactly when business transactions commit.
CDC replicates OLTP changes into Snowflake, BigQuery or a lakehouse in near real time.
In an interview
- Whenever you write to a database and publish an event, call out the dual-write problem.
- Propose the transactional outbox (or CDC) and explain the at-least-once publish and idempotent consumers.
- Use CDC to keep search indexes, caches and warehouses in sync with the source of truth.
- Key events by aggregate ID to preserve per-entity order.