Outbox Pattern: Writing to the Database and the Queue Without Losing Either

The outbox pattern: writing to the database and the queue without losing either

Most web apps reach the same moment. A customer pays, the order flips to paid in the database, and something downstream needs to know: send the receipt, notify the fulfilment system, update the invoice. The naive version commits the order, then publishes an event somewhere. It works, until the day the two steps land in the wrong order in production: the money state changed, the notification didn't go out, and nothing in the logs says anything went wrong.

The gap is easy to miss in testing because both halves work when the server is healthy. It shows up when the queue connection drops right after commit, or the process dies between the two lines. The outbox pattern closes that gap with less machinery than most descriptions suggest.

The failure window

Take the Laravel-shaped version:

DB::transaction(function () {
    $order->update(['status' => 'paid']);
    Bus::dispatch(new SendReceipt($order));
});

It looks safe wrapped in a transaction, and it isn't. Queue backends like Redis and SQS aren't transactional with your database. If the worker connection dies after the commit lands and before Redis accepts the job, the order is paid and the receipt never enters a queue. If dispatch lands first and the commit rolls back, a receipt goes out for an order that was never paid. Both directions lose.

The standard advice is "dispatch after commit", and Laravel's afterResponse/->afterCommit() does reduce one half of the problem. What it can't do is make the two writes atomic, because they happen in two systems. You need a record of the intent to publish, stored where the order is stored, committed in the same transaction.

The outbox

The idea: instead of sending the message, write a row saying you owe the message.

Schema::create('outbox', function (Blueprint $t) {
    $t->id();
    $t->string('event');
    $t->jsonb('payload');
    $t->timestamp('processed_at')->nullable();
    $t->timestamps();
});

// inside the same transaction as the business change:
DB::transaction(function () {
    $order->update(['status' => 'paid']);
    Outbox::create([
        'event' => 'order.paid',
        'payload' => ['order_id' => $order->id],
    ]);
});

The order.paid event and the paid status now commit together or not at all. The event isn't sent yet, but the fact that it must be sent is as durable as the order itself. A separate worker picks up unprocessed rows, publishes each one, and marks it processed:

Outbox::whereNull('processed_at')
    ->orderBy('id')
    ->chunkById(100, function ($rows) {
        foreach ($rows as $row) {
            Bus::dispatch(new DeliverOutboxEvent($row));
            $row->update(['processed_at' => now()]);
        }
    });

If the publisher crashes mid-batch, the unprocessed rows are still there, and the next pass retries them. Delivery becomes "at least once", which is the honest trade: duplicate events are far easier to deal with than missing ones.

Making duplicate delivery harmless

At-least-once delivery means consumers must tolerate the same event twice, so the handler has to be idempotent:

• A dedupe key in the payload. Consumers keep a table of processed event IDs and skip repeats.
• Natural idempotency in the handler. SendReceipt checks whether the receipt was already sent before sending.
• Stable event IDs. The outbox row's own ID can be the dedupe key, generated at commit time, never regenerated by retries.

One subtlety: mark processed_at only after the dispatch call returns. Marking first and dispatching second reintroduces the same two-systems problem with a smaller blast radius and cheaper retries.

Why this beats afterCommit for anything financial

Laravel's afterCommit is fine for low-stakes side effects. An email that occasionally doesn't send, or a cache warm-up that can wait for the next request, doesn't need an outbox. Once a side effect is money-adjacent (receipts, invoices, payout notifications), a database row is the cheapest reliable store you already run.

The cost side is small: one table, one publisher worker, one retry policy you already need for queues anyway. No Kafka, no Change Data Capture. The heavyweight versions of this pattern exist for high-throughput event systems; a web app publishing a few thousand events a day needs about a hundred lines.

Checklist

• Write the event row in the same transaction as the state change.
• Publish from a separate worker, keyed on processed_at IS NULL, in id order.
• Give every event a stable ID, and make consumers dedupe on it.
• Mark processed only after the publish call succeeds.
• Add an alert on outbox rows older than a few minutes: a growing backlog is the first signal the publisher is down.

The outbox isn't clever, and that's the point. It moves a cross-system consistency problem onto the one store that's already transactional, and lets everything else retry in peace.