A referral credit that had to fire exactly once

The mobile app calls an endpoint I'll call /app/info during onboarding. Not once. During a single signup I saw it hit four times, once before the user's account record had fully populated, once right after, and twice more from a client on a bad connection retrying a call it thought had failed.

Buried in that endpoint was a referral bonus: if the new user came in through a referral link, credit the new user and whoever referred them. Exactly once. The naive version paid out once per call, so a flaky connection turned one referral into two or three payouts.

The fix: a flag on the referral record, not on the request

The tempting fix is to dedupe by request, some kind of idempotency key the client sends. That only works if the client is well-behaved, and the client here wasn't the one with the bug, the network was. So the flag went on the data instead of the request.

async function maybeCreditReferral(userId: string) {
  const referral = await db.referrals.findByReferredUser(userId);
  if (!referral) return { credited: false, reason: "no_referral" };

  const result = await db.referrals.updateOne(
    { id: referral.id, referral_credited: false },
    { $set: { referral_credited: true } }
  );

  if (result.matchedCount === 0) {
    return { credited: false, reason: "already_credited" };
  }

  const rewardAmount = await rewardsService.getReferralAmount();

  await Promise.all([
    walletService.credit(referral.newUserId, rewardAmount),
    walletService.credit(referral.referrerId, rewardAmount),
  ]);

  return { credited: true, amount: rewardAmount };
}

Every call to /app/info runs this. Three of the four calls during that onboarding matched nothing, because the flag was already true, and returned already_credited without touching a wallet. Only the first call that won the race actually paid out.

The reward amount itself isn't hardcoded here. It's fetched from a rewards service at the moment of credit, so if that service's referral value changes, this code doesn't need to know or care. Two systems agreeing on a number by both asking a third system beats two systems trying to stay in sync.

Questions people actually ask

Why not just dedupe by request instead of a flag on the record?

Because the duplicate calls in this case weren't duplicate requests in any sense the client controlled. A retry after a dropped connection looks like a brand new request with a new ID. Deduping by request ID would have caught none of them. The only thing that was actually singular here was the referral record itself, so that's what got the flag.

What happens if two requests race on the exact same flag check?

The check and the flip happen as one conditional update against the database, matched on referral_credited: false. If two requests hit that update at nearly the same instant, the database serializes them. One gets a match and proceeds to credit. The other gets zero matches, because by the time its write lands the flag is already true, and it returns already_credited. There's no window where both see false.

Why fetch the reward amount live instead of hardcoding it?

We had a rewards service that already owned the source of truth for what a referral pays out, because that number changes with promotions. If I hardcoded it here, the day someone ran a promo doubling referral value, this endpoint would keep paying the old amount until someone remembered this code existed. Fetching it live means the two systems can't drift.

How did the mutual-exclusivity bug get caught?

Product wanted referred users to get either the referral credit or a generic signup bonus, never both. Someone on the team noticed a test user's wallet balance was higher than either reward on its own, filed it, and when I traced it, the signup bonus code ran unconditionally regardless of whether the referral credit had just fired. The fix was an explicit order: try the referral credit first, and only fall through to the generic signup bonus if the referral path reported no_referral or the user genuinely had none. If the referral path reported already_credited, that still counts as "this user has a referral," so the signup bonus still doesn't fire.

What I'd do differently

I'd write the mutual-exclusivity rule into the same function as the referral check instead of leaving it as a second branch someone has to remember to write correctly at the call site. The bug wasn't in the idempotency logic, that part held up fine under retries. It was in a completely separate rule living one level up, with nothing enforcing it structurally.