The Same Webhook Arrived Twice. Your System Processed It Twice.

The webhook arrived, your system created the fulfilment record and the sender did not receive a successful response. A retry followed. Your system created another fulfilment record.

From the sender’s perspective, retrying was reasonable. From the warehouse’s perspective, there are now two instructions for one business event. The integration failed at the point where receiving a message became permission to repeat an action.

The first attempt may have worked

Consider this hypothetical sequence:

1. An order event reaches your endpoint.
2. The application commits a local dispatch request.
3. The connection fails before the sender receives the acknowledgement.
4. The sender delivers the event again.
5. The handler creates another dispatch request because it treats every delivery as new work.

The error does not require the first database operation to fail. It requires uncertainty about the result to be mistaken for evidence that nothing happened.

Stripe’s webhook documentation explicitly warns about duplicate deliveries and recommends tracking processed event IDs. It also warns that event delivery order is not guaranteed. Those are useful design inputs: a delivery attempt is neither a unique business action nor a reliable position in a timeline.

Verify the sender using its supported signature-verification process before trusting an event. Then establish the event’s identity in the appropriate provider and account context. Authentication and duplicate handling solve different problems; a valid signed retry can still repeat work.

The race hidden inside a duplicate check

A common first fix is to ask the database whether an event has been processed. If not, perform the action and save the event ID. That looks sensible in a single request. Two workers can both ask before either has saved anything, both receive “not found” and both proceed.

The important decision must be enforced where concurrent workers cannot independently win it. A database uniqueness constraint on the receipt identity is one building block. The receipt and a local business effect should be committed in the same transaction when the effect lives in that database.

Here is the shape of a small SQLite demonstration, not a complete production webhook endpoint:

BEGIN IMMEDIATE;
INSERT INTO receipts(event_id) VALUES (:event_id)
  ON CONFLICT(event_id) DO NOTHING;
-- Inspect whether this statement inserted a row.
-- Only if it did: insert the local business effect.
COMMIT;

The application must inspect the insertion result. It must roll back on a failed effect rather than leave a receipt that falsely says the work is complete. Database errors are not duplicates, and should not be swallowed as successful processing.

SQLite serialises writers in this example. Other database engines have different transaction and locking behaviour. The design needs to be tested on the database and isolation settings you actually deploy.

A small local replay test

For this article, a local Python and SQLite lab used a receipt table with a unique event ID and a separate effect table. The effect table deliberately had no unique event constraint: the transaction and receipt logic had to prevent repeated inserts.

Four scenarios were executed against isolated test data. Two separate connections were synchronised before the concurrent-delivery attempt. The measured results were:

Across the four distinct event IDs, the database contained four receipts and four effects. The response-loss case was simulated after the commit; no HTTP connection or payment provider was involved. This proves the local transaction behaviour of the demonstration. It does not prove a deployed integration can fulfil orders exactly once.

The boundary where the database stops helping

Now replace the local effect with a request to an external dispatch service. Your database transaction cannot roll back a parcel label already created in another system. If that remote request succeeds but its response is lost, retrying without an appropriate idempotency mechanism can create another label.

Use a stable business-operation key with the downstream service when it supports suitable idempotent requests. Understand its retention window and what happens when the same key is reused with different parameters. If the remote outcome remains unknown, reconcile against the remote record before issuing another irreversible action.

A durable outbox can keep a pending dispatch instruction in the same database transaction as the local state change. A worker then delivers it. That closes one gap between local commit and later processing, but the worker’s remote call still needs duplicate protection or reconciliation. Naming a table “outbox” does not remove uncertainty at the next boundary.

Event identity and business identity also differ. Two distinct events may describe the same fulfilment obligation. Conversely, two legitimate updates to one customer are not necessarily duplicates. Define the business operation you are protecting before choosing a key; do not permanently suppress every event that mentions the same object.

Give the operator a recovery path

For asynchronous handling, acknowledge receipt only after the event is durably accepted, then let a worker perform the longer operation. Keep the endpoint responsive without returning success for work that exists only in memory.

Record enough state to distinguish received, processing, completed, retryable failure and uncertain remote outcome. Operators need the event identity, operation identity, attempt history and last useful error. A “retry” button should follow those states rather than blindly repeat everything.

This is where App Development and AI and Automation meet: the business workflow needs reliable effects, not merely a successful connector run. Repair starts by following one event through receipt, persistence, action and recovery.

Before trusting the integration, repeat an event in staging, race two deliveries and interrupt the response after a committed action. Compare delivery attempts with actual business effects. If two attempts still create two dispatches, adjusting the retry delay has not solved the problem.