Receiving Webhooks: Verification, Idempotency, and the Replay Problem

Sooner or later your app has to accept webhooks: Stripe telling you an invoice was paid, a Git provider announcing a push, a customer's system notifying yours. The sending side gets all the attention: signing, retries, delivery dashboards. Receiving is where production incidents actually start, because you don't control any of it. The delivery arrives whenever it wants, sometimes more than once, occasionally in the wrong order, and from systems you can't debug.

The good news: four habits cover almost everything that goes wrong.

Verify the signature, every single time

The first and non-negotiable rule: a webhook endpoint with an unauthenticated URL is a public endpoint that happens to create paid subscriptions. Verify the sender's signature before touching the payload.

Most providers follow the same pattern: an HMAC over the raw body with a shared secret:

// Stripe-style: verify the signed header against the raw body
$event = \Stripe\Webhook::constructEvent(
    $request->getContent(),
    $request->header('Stripe-Signature'),
    config('services.stripe.webhook_secret')
);

The detail that bites people: signers compute the HMAC over the raw request body, before any framework or middleware re-encodes, pretty-prints, or normalizes the JSON. Verify before your JSON parsing layer touches the body, or you'll chase a signature mismatch that only shows up for payloads containing non-ASCII characters.

One more failure mode worth knowing: some senders sign with a timestamp and you should reject deliveries older than a few minutes, so a captured payload can't be replayed at an attacker's leisure.

Never trust the delivery, trust the event ID

Webhook documentation everywhere says "at-least-once delivery," and people read it as "usually once." It isn't. Retries happen when your endpoint responded too slowly, when your provider's network blipped, and when your own redelivery button gets pressed. The same invoice.paid event can arrive three times, and your code should treat that as routine.

The fix is a deduplication step you've already built if you followed our piece on the sending side: store the event ID, reject duplicates.

$stored = DB::transaction(function () use ($event) {
    return WebhookEvent::firstOrCreate(['provider' => 'stripe', 'event_id' => $event->id]);
});
if ($stored->wasRecentlyCreated === false) {
    return response()->noContent(); // already handled, ack and move on
}

Marking the row inside the same transaction that does the work keeps two concurrent deliveries of the same event from racing each other through the check.

Out-of-order delivery is the quieter sibling: an invoice.paid can arrive after its own invoice.voided, especially when retries stretch over hours. Compare the event timestamp against the state you already store and refuse to apply stale events; the timestamp in the payload exists for exactly this.

Hand off the work, respond fast

The handler has one job: verify, deduplicate, enqueue, return 2xx. Every second it spends calling a payment gateway or rebuilding a PDF is a second closer to the sender's timeout, and a timeout means a retry, which means a duplicate, which means the "processed twice" bug you already solved by deduplicating.

ProcessStripeEvent::dispatch($event->id);
return response()->noContent();

If the queue worker then fails, the event is already recorded and a job retry picks it up. The webhook stays fast and idempotent, and the sender has nothing to retry for.

This is also why "test mode works but production doesn't" happens: in test mode you click the button and watch it work synchronously. In production the real work happens somewhere else, where nobody is watching. Log every event you accept (type, ID, delivery status) before you need to answer "did we receive the refund notification from Tuesday?"

The 404 you should be sending

When an event arrives that matches no known type (a new Stripe event, a renamed webhook, a payload from a deleted endpoint) respond with 200 and a log entry, or carefully with 410 Gone. What you must not do is let it fall into a 500, because the sender will dutifully retry an event you can never process, forever, and your dead-letter queue fills with noise that hides real failures.

Unknown events aren't errors. They're tomorrow's features arriving early.

The checklist

Verify the HMAC signature against the raw body, on every request.
Reject or flag events older than your freshness window.
Store the event ID and deduplicate inside the transaction that does the work.
Enqueue, acknowledge, get out. Under a second.
Handle unknown event types gracefully and log them.
Endpoints belong in your secrets management, not in a .env that's copy-pasted between environments.

None of this is clever. It's just the difference between a webhook that works when the demo runs and one that still works during a payment provider's bad afternoon.