Race conditions: when two users spend the same money
Every developer learns to check the balance before charging it. The bug shows up the first time two requests run at once: both read the same balance, both see enough money, both write. The check passed in both requests, and the data is now wrong. Nothing crashed, no error appeared in any log, and the corruption is discovered weeks later during a reconciliation nobody enjoys.
Concurrency bugs are quiet because each individual request looks correct. The failure lives in the timing between requests, so a test that passes will keep passing until the code meets real users acting at the same time.
Where these bugs actually happen
Not in exotic load scenarios. In ordinary features:
- A "Buy" button double-clicked, so two checkout requests process one cart.
- A customer opens an invoice twice in two tabs and pays both.
- Two admins edit the same settings record; the last save wins and erases the first.
- A webhook arrives twice while a user is clicking through the same flow, and both paths credit an account.
None of these look like a race to the person who reports them. They look like "the button charged me twice" or "our stock numbers don't add up."
Transactions are the container, not the fix
A transaction gives you all-or-nothing, but it does not give you isolation the way most people assume. Under MySQL's default REPEATABLE READ, two transactions can both read the same balance, both decide it's sufficient, and both commit. The transaction prevented half-written writes; it did nothing about the logic racing itself.
// WRONG: the check and the write can interleave with another request
DB::transaction(function () use ($account, $amount) {
if ($account->balance < $amount) {
throw new InsufficientFunds();
}
$account->decrement('balance', $amount);
});
The check must lock out the competing request between reading and writing. In MySQL and PostgreSQL, SELECT ... FOR UPDATE does that: the second transaction blocks until the first commits, then re-reads the row and sees the new balance.
DB::transaction(function () use ($accountId, $amount) {
// Locks this row; a concurrent transaction waits here
$account = DB::table('accounts')->where('id', $accountId)
->lockForUpdate()->first();
if ($account->balance < $amount) {
throw new InsufficientFunds();
}
DB::table('accounts')->where('id', $accountId)
->decrement('balance', $amount);
});
Two rules come with it. Keep the transaction short: the lock is held while your transaction runs, so never put HTTP calls inside one. And every path that touches the balance has to lock it, including whichever controller it runs from. One code path missing the lock reinstates the bug.
Constraint beats lock when a constraint fits your rule
Some rules express themselves as uniqueness, and a unique index catches the race without any locking code at all. A coupon that may be used once per user is a unique index on (user_id, coupon_id); if the insert violates it, the coupon was used, and PostgreSQL lets you tell those violations apart with ON CONFLICT DO NOTHING. A limited-stock sale is a conditional write: WHERE reserved < total and checking the affected row count. The database arbitrates in a single statement, so there is no gap for a second request to slip through.
Losing a race this way is not an error path to apologize for. It's the correct behavior rendered as an exception, which means the failure is visible, countable, and testable.
Idempotency keys for the flows that repeat
Retries and double-clicks are not anomalies; they are what happens to any endpoint long enough. A client-side disabled attribute helps, but every browser can break it, and a network timeout guarantees the same request arrives twice.
Give mutation endpoints a client-supplied idempotency key (a UUID generated once per attempt, header or field). Server side it becomes a unique constraint:
Payment::create([
'idempotency_key' => $request->header('Idempotency-Key'),
// ... the rest of the payment attributes
]);
The second attempt fails the unique check. Returning the original result instead of the duplicate is then a lookup away. This is the same trick our billing article applies to provider webhooks, pointed at your own API: every request carries evidence of whether you already did it.
Test it as a race, not a sequence
The naive test passes: run the charge once, assert the balance. A real test makes both requests run before either finishes. The reliable version is a small set of concurrent workers, each posting the same endpoint with the same idempotency key, run against a real database. Then assert one charge, one non-negative balance. In Go, t.Parallel() plus a WaitGroup covers the same shape. If a test suite has never made two requests fight, every lockForUpdate in the codebase is unfalsified.