The webhook that arrived before the one it depended on

Stripe does not promise to deliver events in order. A handler that decides from the event's type ships goods for payments that have not settled, and ships them again when the payment does.

The idempotency advice everyone follows is about duplicates: store the event id, skip the event if you have seen it. It is correct, and it is aimed at the wrong failure for the case in this piece. The two events below are not duplicates. They have different ids and different types, and they describe the same order at two moments in time. Nothing in an idempotency store stops a handler from acting on both, and the order they arrive in is not up to you.

The setup

A Checkout session finishes. For a card payment, one event carries the news: checkout.session.completed, with payment_status already paid. For the payment methods that settle later, bank debits and vouchers among them, the same event fires when the customer finishes checkout with payment_status still unpaid. The money arrives hours or days later, and Stripe then sends checkout.session.async_payment_succeeded, or async_payment_failed if it never does.

Stripe's documentation says two things about this that are easy to read past. Delivery order is not guaranteed. And the completed event is not a promise of payment; the fulfilment guide says to check payment_status. Most handlers were written against a card-only test account where the first event always says paid, and never meet the second shape until a customer pays by bank debit.

Here is the handler most people ship, reduced to what matters. It is the naive example in second-run:

export default function build({ provider, crashOnce, state }) {
  const done = new Set();
  const orders = new Map();
  // ...
  return async function handle(event) {
    if (done.has(event.id)) return { kind: "duplicate" };
    const r = await provider.fulfil(event.orderId);          // the expensive part
    orders.set(event.orderId, { status: "provisioned", r });
    done.add(event.id);
    return { kind: "provisioned" };
  };
}

It has the idempotency check. It fulfils on every event it has not seen. It never asks whether the payment settled.

The failure, in both directions

The out-of-order scenario in second-run plays two orders through the handler. The first order gets its events in the order Stripe intends. The second gets them reversed, which Stripe also permits.

// Phase A: the unpaid completed event arrives first. Nothing may be bought yet.
const unpaidFirst = ev("evt_4a", "ord_4", { type: "checkout.session.completed", paid: false });
await deliver(handle, unpaidFirst, rejections);
const boughtUnpaid = provider.calls.length;
// Then the payment succeeds. Exactly one purchase now.
await deliver(handle, ev("evt_4b", "ord_4", { type: "checkout.session.async_payment_succeeded", paid: true }), rejections);
const afterPaid = provider.calls.length;
// Phase B, a second order: paid first, then a stale unpaid completed event. Still exactly one purchase.
await deliver(handle, ev("evt_5a", "ord_5", { type: "checkout.session.async_payment_succeeded", paid: true }), rejections);
await deliver(handle, ev("evt_5b", "ord_5", { type: "checkout.session.completed", paid: false }), rejections);

Run against the handler above, the report is:

FAIL  out-of-order — an unpaid completed event arrives before the payment succeeds, and again after it
      observed: 1 purchase(s) while only an unpaid event had arrived, 2 after payment,
                4 across both orders; final status "provisioned" / "provisioned"
      expected: 0 purchases before payment, exactly 1 per order after it, both orders "provisioned"

Read the numbers slowly, because the last line hides the damage. Both orders end provisioned. Nothing threw. No status is wrong. Four items were bought for two orders.

Phase A: the unpaid event arrives and the handler buys. If that bank debit fails three days later, the customer has the goods and you have the invoice. When the payment does succeed, the second event has a new id, the idempotency check waves it through, and the handler buys again. Two items, one payment.

Phase B: the paid event arrives first and the handler buys, correctly. Then the completed event, delayed somewhere between Stripe and you, arrives carrying its snapshot of the session from before the money moved. New id. The handler buys again. Two items, one payment, this time triggered by an event that is telling the truth about the past.

The idempotency store did exactly what it was built to do on all four deliveries. None of them was a duplicate.

Why the test suite passed

Three reasons, and they compound.

The fixtures were all cards. Every checkout.session.completed fixture had payment_status: "paid", because that is what the Stripe CLI produces on a test account by default and it is what the developer saw in the Dashboard. No fixture had an unpaid completed event, so no test could show that fulfilling on it was wrong.

The tests delivered one event per order. A test that sends one event and asserts one purchase cannot say "the second event about this order must not buy again", and it cannot express order at all. Out-of-order is a property of a sequence, and the suite had no sequences.

The idempotency test proved the wrong thing. It sent the same event twice, asserted one purchase, and passed. That test is about redelivery. It says nothing about two different events that both look like a reason to fulfil. Passing it produced confidence that covered a case it never touched.

The fix: gate on the state, not the type

The order has exactly one transition that buys: paid, and not yet provisioned. Every event, of any type, in any order, either carries evidence of that state or it does not. The handler's first line becomes a question about the payment, and the event type stops mattering:

return async function handle(event) {
  if (event.paid === false) return { kind: "unpaid" };       // gate on payment status, not event type
  const c = claim(event.orderId);
  if (c.kind !== "claimed") return c;                        // provisioned, or another attempt holds the lease
  try {
    const r = await provider.fulfil(event.orderId);
    const w = settle(event.orderId, c.claimId, { status: "provisioned", r });
    return w;
  } catch (e) {
    // ...mark failed only if nothing was bought; never swallow, let the retry happen
    throw e;
  }
};

Trace the two phases through it. Unpaid completed first: unpaid, nothing bought. Paid event: the claim succeeds, one purchase, order provisioned. Paid event first: the same, one purchase. Stale unpaid completed afterwards: unpaid, nothing. Two orders, two purchases, and second-run's verdict on this version is one line: PASS out-of-order.

The claim is doing a second job here that is easy to miss. If two paid events about one order ever arrive, a retry of async_payment_succeeded, or a completed event that was paid because the card settled instantly followed by its own redelivery, the gate lets both through and the claim makes sure only one buys. The payment gate and the claim answer different questions: "should anything happen for this order yet?" and "is this attempt the one that gets to do it?" You need both.

One refinement for a case the scenario does not cover. The event carries a snapshot of the session at the moment the event was created. If the paid event is lost entirely, a late unpaid completed event is the only news you will ever get, and its snapshot says unpaid. A handler that retrieves the session from Stripe at handling time and gates on the live payment_status fulfils correctly no matter which event brings the news and no matter how late. It costs one API call per event. For anything that ships goods or credits an account, it is worth it.

Five checks, one hour

1. Find every place fulfilment is decided by event.type. For each, ask what happens when that event has payment_status: "unpaid". If the answer is "it fulfils", that is this bug.
2. Grep your fixtures for "unpaid". If nothing comes back, your suite has never seen a delayed payment method.
3. Look at what your idempotency key is. If it is the event id, it protects against redelivery only. The protection against two different events buying twice is the claim on the order, not the key.
4. Replay two real events reversed. From the Dashboard's webhook log, resend an async_payment_succeeded and then its completed event to a test endpoint. Count purchases.
5. Run the scenario. node src/cli.js your-handler.js --only out-of-order in a checkout of second-run reports what your handler actually did and can emit the regression test.

What this generalises to

Any system that receives its state as a stream of notifications has this shape: an order-status feed, a carrier's scan events, a payment processor's dispute lifecycle, a Kafka topic with at-least-once delivery and more than one partition. The notifications are true about the moment they were made. They are not promised to arrive in that order, and each one looks like a reason to act. The handler that survives this decides from the resource's state, takes a claim before acting, and treats the notification as a hint to go and look, not as the instruction itself.

The code

The scenario, the naive handler, and the fixed one: github.com/GG5533/second-run. The claim and settle pattern in full, with both storage backends and the fencing tests from the previous piece: github.com/GG5533/stripe-fulfilment-kit. Neither needs a Stripe account or a network to run.

Sami Habbal fixes the failures that only show up on run number two. Charlotte, NC. github.com/GG5533