Meet Modi
Back to Blog
·5 min

The reward that fired twice

A challenge creator got a bonus for winning. They also got it for winning again, on the same challenge, because a retry doesn't know it's a retry.

By Meet Modi
IdempotencyBackendConcurrency

A challenge creator gets a bonus in in-app currency when their opponent's score crosses a threshold. Most creators got the bonus once. A few got it twice, on the same challenge, for the same win.

Nobody replayed anything on purpose. The completion handler just ran more than once.

The bug: the award logic trusted that completion only happens once. Completion doesn't work that way.

Take 1: award inline, every time

This ran on a background task queue. Background tasks in this system get retried after a crash, and the mobile client separately retries its own completion call after a timeout. Both paths call the same handler.

First run: score check passes, wallet gets incremented, status flips to completed. Second run, triggered by a retry a few seconds later: score check passes again, because nothing about the challenge row says this already happened. The wallet gets incremented a second time.

Take 2: a flag that lives with the award

The fix wasn't to stop retries from happening. Retries are the reason the system is reliable when a task crashes mid-flight. The fix was to make the award itself remember it already ran.

async function onChallengeCompleted(challengeId: string) {
  const challenge = await db.challenges.findById(challengeId);
  if (challenge.opponentScore < challenge.threshold) return;

  const result = await db.challenges.updateOne(
    { id: challengeId, creator_xp_awarded: false },
    { $set: { creator_xp_awarded: true, status: "completed" } }
  );

  if (result.matchedCount === 0) {
    // Already awarded on a previous run. Nothing to do.
    return { xpGranted: 0 };
  }

  await db.wallets.increment(challenge.creatorId, {
    xp: challenge.bonusXp,
  });

  return { xpGranted: challenge.bonusXp };
}

The match condition creator_xp_awarded: false and the flag flip happen as one write against the challenge row. A retried handler runs the same update, matches nothing because the flag is already true, and returns a zero delta instead of touching the wallet.

What I should have done first

I should have asked, before writing the handler, how many times this function can run for the same challenge. The answer was never zero or one, it was always "at least once, possibly more," and every retry-based system answers that question the same way.

Once I started asking that question first, the flag-on-the-record pattern stopped feeling like a fix and started feeling like the default way to write this kind of function.

More Posts