---
title: "The charge that almost happened twice"
description: "A payment webhook with no idempotency guard, one retry away from double-charging real customers. How it surfaced and what the fix actually required."
publishedAt: 2026-07-26
author: Ena Pragma
url: https://enapragma.co/blog/case-study-the-charge-that-almost-happened-twice
tags: ["case-study", "verification"]
---

A payment webhook handler looked like it was working. It processed events, updated records, moved on. Nobody had checked what happened when the same event arrived twice.

It does, eventually. Networks retry. Providers resend. Any handler that isn't built for that will, sooner or later, treat one payment as two.

## The setup

The client's platform received webhook events from a payment processor to confirm charges and update order state. The processing code assumed each event would arrive once. That assumption doesn't hold in production. Providers resend events on timeout, dropped acknowledgments, or their own retry logic, and a handler with no protection against replays processes the resend exactly like a new event.

For a payment webhook, that gap has one direction of failure: money. A replayed event either double-charges a customer or corrupts the order state trying to reconcile two "successful payment" signals for one real payment.

## What we caught

Reviewing the handler, we checked it against the case that doesn't show up in a demo: the same event ID arriving twice. It wasn't idempotent. There was no check for "have I already processed this exact event," so a resend would run the full charge-confirmation logic a second time.

The failure mode wasn't hypothetical. Webhook retries are normal provider behavior, not an edge case. Any integration without idempotency protection will eventually see a duplicate, and when it does, the customer either gets charged twice or the order shows a state that doesn't match what actually happened.

## What we did about it

We added idempotency handling to the webhook processing layer:

1. Every incoming event is checked against its unique event ID before any charge-confirmation logic runs.
2. A duplicate event is acknowledged (so the provider stops retrying) but not reprocessed.
3. The fix was verified against real duplicate-delivery scenarios, not just the happy path, before it shipped.

No customer was double-charged and no order was left in an inconsistent state as a result of this fix. It closed the gap before a real duplicate delivery could exercise it.

## Why this is the whole point

This isn't glamorous work. Idempotency checks don't show up in a product demo. But payment infrastructure fails quietly until the moment it doesn't, and the moment it doesn't is a customer's bank statement, not a bug tracker. Verifying that a payment handler does the right thing on the second delivery of the same event, not just the first, is the same discipline behind everything we build: don't ship on the assumption that the happy path is the only path.

That's what "we run what we build" means in practice for anyone taking a payment online. We don't just wire up the integration and walk away. We check what it does under the conditions it will actually see.