Every payment API integration eventually hits the same bug: the user taps
“Pay” twice, the network stutters, a timeout fires, a retry runs, and somehow
your customer is charged twice. The fix is an idempotency key
— the contract that lets a client safely retry a request without duplicating
its side-effect. This article covers the patterns NxtBanking, Stripe, Razorpay
and most mature payment APIs use, and the five mistakes we still find in
production code reviews.
1. What idempotency actually guarantees
A request is idempotent if sending it two or three or N times
results in the same server state as sending it once. For payment APIs that
means: same merchant_txn_id + same idempotency key + same payload
= the same transaction, not a new one.
2. Who generates the key — and from what?
The client generates the idempotency key, not the server.
This is non-negotiable; the server can’t tell two retries apart from two
genuinely separate requests without a client-supplied correlation token.
Good sources of entropy:
- UUID v4 — 122 random bits; collision risk is effectively zero. Our default.
- UUID v7 (time-ordered) — same guarantees plus DB-friendly ordering.
- Deterministic hash — sha256(user_id + order_id + amount + day). Lets you reconstruct the key from inputs; useful when the client crashes before it can persist its generated UUID.
Anti-patterns:
- Timestamps only — two users in the same ms collide.
- Database auto-increment IDs — leak ordering, predictable.
- Server-generated keys returned to the client — defeats the point.
3. Where to send it
Industry-standard is a dedicated header: Idempotency-Key:
Keep it out of the body so your gateway / proxy / log-aggregator can route on
it without parsing JSON.
curl -X POST "https://api.nxtbanking.com/payouts/v1/transfer"
-H "Authorization: Bearer $TOKEN"
-H "Idempotency-Key: 9f8e7d6c-5b4a-4938-a7b6-c5d4e3f21098"
-H "Content-Type: application/json"
-d '{
"amount": 1000.00,
"account": "HDFC0001234567890",
"ifsc": "HDFC0000001",
"remarks": "Payout for invoice #5432"
}'
4. Server-side storage — the interesting part
The server needs to remember, for each (client_id, idempotency_key) pair,
either: “I’m working on it” (in-flight), “done — here’s the canonical response”
(completed), or “never seen this key” (new).
A minimal Postgres schema:
CREATE TABLE idempotency_records (
client_id TEXT NOT NULL,
key TEXT NOT NULL,
request_hash BYTEA NOT NULL, -- sha256(method + path + body)
response_code INT, -- NULL while in-flight
response_body JSONB, -- NULL while in-flight
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
completed_at TIMESTAMPTZ,
PRIMARY KEY (client_id, key)
);
-- Index to expire old records (see TTL section).
CREATE INDEX idx_idemp_created ON idempotency_records (created_at);
5. The request lifecycle
- Request arrives with Idempotency-Key.
- INSERT … ON CONFLICT DO NOTHING a row with request_hash and the current user.
- If the insert created a row → we’re the first; go process.
- If the insert found an existing row:
If request_hash matches and response is present → return the stored response verbatim.
- If request_hash matches and response is null → 409 “request still processing” (or poll-and-wait depending on your API contract).
- If request_hash differs → 422 “idempotency key reused with different payload” (hostile-client signal).
- On completion, UPDATE the row with response_code, response_body, completed_at.
6. TTL — how long to remember keys
24 hours is the sweet spot for retail payments. A day is long enough for any
reasonable retry window (including a flaky mobile user re-opening the app an
hour later). Beyond 24 h, your storage grows without giving you meaningful
safety. Stripe uses 24 h. Razorpay uses 24 h. Follow the herd.
-- Nightly cleanup
DELETE FROM idempotency_records WHERE created_at < now() - interval '24 hours';
7. The five mistakes we see in code reviews
- Storing the key in memory only. One pod restart and your idempotency guarantee is gone. Use a durable store (Postgres / Redis with AOF).
- Computing request_hash over the raw body including whitespace. Clients frequently re-serialise JSON with different key ordering. Hash canonicalised JSON (sorted keys, minified) or hash structural data.
- Returning the in-flight record’s (empty) response on retry. The correct response is 409 or a poll — never an empty 200.
- Using the user’s email / phone / order-id as the key. Breaks when the same user places a second genuine order within the TTL window. The key is a request identifier, not a resource identifier.
- No alarm on “key reused with different payload.” This is a real signal — either a client bug or an attacker. Alert the on-call engineer.
8. Idempotency vs safe retry strategy
Idempotency gives you the right to retry. It doesn’t tell you
when to retry. Use exponential backoff with full jitter, cap at 6-8
attempts, and never retry on 4xx (except 408, 409, 429). We cover the retry
machinery in
Payout API Failure Modes
& Retry Strategy.
9. Production checklist
- ☐ All mutating endpoints accept Idempotency-Key
- ☐ Durable store with 24-h TTL
- ☐ Request-hash over canonicalised JSON
- ☐ In-flight vs completed states distinguishable
- ☐ Metrics: idempotent-hits, hash-mismatch, in-flight-wait-time
- ☐ Alerts on hash-mismatch rate > 0.01%
- ☐ Runbook for “key reused with different payload”