Meet Modi
Back to Blog
·5 min

An endpoint built to be called twice

A reward endpoint has no way to tell a genuine retry from a genuinely repeated action. So we stopped pretending it could, and made it credit every call, on purpose.

By Meet Modi
Distributed SystemsAPI DesignBackend

A reward-granting endpoint sits behind a service boundary. Another internal service calls it whenever a user finishes some action worth a small amount of in-app currency. That calling service cannot guarantee it only calls this endpoint once per completion. Its own retries and network hiccups produce duplicate calls sometimes, and there's nothing about those duplicate calls that looks different from the outside.

The instinct when you see duplicate calls granting duplicate rewards is to add dedup logic on the receiving end. We looked at that and stopped, because the receiving endpoint has no context to do it well. It doesn't know if two calls five seconds apart are one completion retried or two separate completions that happen to look alike.

The decision: stay stateless, move the responsibility

The calling service, unlike the receiving endpoint, does have the context. It knows whether it already successfully called this endpoint for a given completion event, because it's the one that decided the event happened. So the contract became explicit: this endpoint stays stateless and credits every call it receives, full stop. Not calling it twice is entirely the caller's job.

// This endpoint is intentionally NOT idempotent.
// It credits every call it receives, no exceptions.
//
// Do not add dedup/idempotency logic here. The calling
// service is the only party with enough context to know
// whether a call is a genuine retry or a new completion
// event, and it already guarantees single-delivery on its
// side. Deduping here would silently eat legitimate retries
// the caller relies on for reliability.
async function grantCompletionReward(req: GrantRewardRequest) {
  await walletService.credit(req.userId, req.amount);
  return { credited: true, amount: req.amount };
}

That comment isn't decoration. It exists specifically to stop a future engineer, possibly me in six months, from looking at this function, seeing no guard against duplicate calls, and "fixing" it. Adding dedup here wouldn't fix anything. It would break the caller's actual retry-for-reliability behavior, silently, in a way that only shows up as underpaid rewards weeks later.

Questions people actually ask

Isn't a stateless reward endpoint dangerous?

It's dangerous if you assume statelessness means no one is responsible for correctness. Here it means one specific party is responsible, and that party is the one with the information to actually do it right. The danger isn't in the endpoint being stateless. It's in a stateless endpoint pretending it can also be correct about dedup with no context to do it.

Why not just make the receiving endpoint idempotent by default?

Idempotency needs a key, something that identifies "this exact logical operation" so a repeat of it can be recognized and no-op'd. This endpoint doesn't receive anything that reliably identifies the completion event versus a retry of it, because the caller's own retry logic doesn't attach one. Building idempotency without a trustworthy key just means guessing, usually by time window or by matching recent calls, and guessing wrong either double-pays or silently drops a legitimate second reward for a legitimate second action.

How does the calling service actually guarantee it only calls once?

It tracks, on its own side, whether it has already successfully received a response for a given completion event before it considers that event done. If its call to the reward endpoint times out or fails, it retries, because from its perspective the call might not have landed. If its call succeeds, it marks the event as rewarded and never calls again for that event. The reward endpoint doesn't need to know any of this. It just needs to trust that a call means "credit this," every time.

What would go wrong if someone added dedup here later?

Say someone adds a rule that ignores a call if an identical-looking call came in within the last few seconds. The calling service's legitimate retry, sent because it genuinely didn't get a response the first time even though the first call actually succeeded, would now get silently swallowed. The caller thinks the reward didn't go through and might alert or escalate, or worse, just move on assuming failure, while the receiving end thinks everything's fine because it deduped "correctly." Both sides end up with a different, wrong story about what happened, and the user is underpaid with no error anywhere.

What I'd do differently

I'd write the comment in the code before writing the function, not after. The design decision here is the interesting part, and it's easy for that decision to exist only in a design doc or a Slack thread while the code itself looks like an oversight. A future reader shouldn't have to go find the conversation to learn that the missing guard is the point.

More Posts