Idempotency keys done right: the pattern each implementation misses

A duplicate charge or a double-sent email usually traces back to the same root cause: the server ran an operation it should have recognised as already done. The fix most teams reach for is an idempotency key. It works — right up until two retries arrive within the same 50-millisecond window, and neither one knows the other exists.

This article covers the three patterns for building idempotency into an API endpoint, the specific failure mode inside each, and the database-level technique that closes the most common gap. It is written for engineers who have already read the introductory explainers and want to know what those explainers quietly skip.

Three patterns, one goal

An idempotent operation produces the same outcome whether it runs once or ten times. In API terms: when a client retries a request, the server skips the operation and returns the earlier result.

There are three places in a system where that guarantee can live:

• Header-based key: the client generates a UUID per intended operation and sends it in every request. Stripe calls this header `Idempotency-Key`. The server stores the key alongside the result and returns the cached result on any retry.
• Natural key: the domain model already carries a unique constraint that encodes the operation. An `order_id` or `invoice_id` that is unique in your database is already an idempotency key.
• Body-hash key: the server hashes the request body and uses the hash as the key. No client coordination required.

Each approach solves the duplicate-operation problem. Each has a distinct failure mode. The mistake is treating them as equivalent and reaching for whichever feels simplest in the moment.

Header-based keys: Stripe's pattern and its gap

Stripe's model is the reference implementation most teams copy. The client sends an `Idempotency-Key` header. The server stores the key in a table, runs the operation, and stores the result. On any subsequent request with the same key, it returns the stored result without running the operation again.

The implementation most tutorials show looks like this:

-- Step 1: check if the key already exists
SELECT status, result FROM idempotency_keys WHERE key = $1;

-- If not found: run the operation, then:
INSERT INTO idempotency_keys (key, status, result, created_at)
VALUES ($1, 'done', $2, now());

The race is between the SELECT and the INSERT. Two requests carrying the same key can both execute the SELECT before either has committed the INSERT. Both read 'not found'. Both run the operation. You have charged the customer twice.

This is not a theoretical concern. It is the failure you hit at moderate traffic when a mobile client retries on a network timeout before the first response has returned. The gap between check and commit is typically tens of milliseconds — long enough for a second request to slip through on a busy endpoint.

Wrapping both statements in a transaction does not help. The SELECT does not lock a row that does not yet exist. Another transaction reads the pre-insert state concurrently regardless of isolation level, unless you go all the way to serialisable isolation — which most production databases absorb poorly under load.

Natural keys: no extra table, but limited scope

When an operation maps cleanly to a single row in your database, the primary key or a unique index on that row is already an idempotency key.

Consider a `payments` table with a unique constraint on `(invoice_id, idempotency_ref)`. Attempting to insert a duplicate raises a conflict. Handle it with `ON CONFLICT DO NOTHING` and return the stored row. No separate idempotency table, no TTL management, no header protocol. The database enforces the uniqueness semantics you already need for business reasons.

Three specific cases where this breaks down:

• Amendment: if the same invoice legitimately needs two separate charges (a partial charge followed by the remainder), the natural key does not distinguish a retry from a new operation. You need a separate idempotency reference alongside the domain key.
• Multi-step operations: some operations write to several tables or call external services. There is no single row to constrain. A unique index on one table does not protect the other steps of the flow.
• External effects: sending an email, dispatching a webhook, or calling a third-party API cannot be constrained by a database index. The side effect happens outside your transaction boundary entirely.

Natural keys handle the simple case cleanly and are almost always underused in teams that reach immediately for an idempotency table. For anything that crosses a single row or a single system, they leave gaps.

Body-hash keys: less obvious than it looks

A body hash removes the coordination requirement. The server computes an HMAC-SHA-256 of the request body (or a canonical subset of it) and uses the result as the idempotency key. The client sends no special header. Deduplication is transparent to the caller.

The appeal is real in contexts where clients are third-party systems or legacy code that cannot be updated to send an explicit key. Three problems follow.

• What goes into the hash. Include a timestamp or a request ID and every retry at a different moment is treated as a new operation. Exclude too much and two genuinely different operations with the same amount and recipient hash identically. The canonical hash input is hard to define correctly and fragile to change after deployment.
• Disambiguation. If a client legitimately sends the same amount to the same recipient twice in quick succession, the body hash treats the second as a retry and suppresses it. You need out-of-band disambiguation — a client-supplied intent flag, a time-window exemption — which re-introduces the client coordination you were trying to avoid.
• The same race. A body hash is still just a key. The check-and-insert gap that affects header-based keys exists here in exactly the same form. Changing how the key is generated does not change the concurrency problem at all.

Idempotency keys: table schema and atomic claim

The fix for the TOCTOU gap is a single INSERT statement that atomically claims the key. Before showing it, the table:

CREATE TABLE idempotency_keys (
  id           BIGSERIAL    PRIMARY KEY,
  key          TEXT         NOT NULL,
  status       TEXT         NOT NULL CHECK (status IN ('processing', 'done', 'failed')),
  result       JSONB,
  created_at   TIMESTAMPTZ  NOT NULL DEFAULT now(),
  completed_at TIMESTAMPTZ
);

-- Partial unique index: only enforce uniqueness within the active window.
-- Keys older than 24 hours are ignored by this index and can be reused.
CREATE UNIQUE INDEX idempotency_keys_active_key
  ON idempotency_keys (key)
  WHERE created_at > (now() - interval '24 hours');

The partial index handles expiry without a background job. Rows older than the window are invisible to the constraint and do not prevent new insertions on the same key.

The atomic claim:

-- Returns a row if this request is first; returns nothing if another claimed the key first.
INSERT INTO idempotency_keys (key, status)
VALUES ($1, 'processing')
ON CONFLICT ON CONSTRAINT idempotency_keys_active_key DO NOTHING
RETURNING id;

The semantics are precise. Postgres evaluates the INSERT and the conflict check as a single atomic step. There is no window in which another transaction can observe the row as absent and also claim it.

• Returned a row: your request is the first to claim this key. Proceed with the operation.
• Returned nothing: another request claimed the key first. Either wait and return the stored result, or return a 409 with a Retry-After hint.

After the operation completes, update the row:

UPDATE idempotency_keys
SET    status       = 'done',
       result       = $2,
       completed_at = now()
WHERE  key = $1;

Handling the 'processing' state

The atomic claim introduces a state you have to reason about explicitly: a key that has been claimed but whose operation has not yet finished. A second request with the same key that arrives during this window will receive no row from the INSERT and needs to decide what to do.

Three options, with the tradeoffs:

• Return 409 immediately. The client sees a conflict, backs off, and retries after a delay. Simple to implement. Works well when the operation is fast (< 1 second) and clients retry with exponential backoff. Breaks down for long-running operations where a retry storm is a real cost.
• Return 202 with a status URL. Tell the client the operation is in progress and give it a URL to poll for the result. This is the correct answer for operations that take more than a few seconds. It requires you to model the operation as an async job, which most teams underinvest in early.
• Block and long-poll. Hold the connection open and wait for the first request's transaction to commit, then return its result. Works for fast operations on low-concurrency endpoints. Risky on high-concurrency ones: held connections consume server resources and can cascade into a backlog.

The 409 approach is the pragmatic default for synchronous endpoints with fast operations. The 202 approach is the correct long-term answer for anything that can take seconds.

Expiry and external effects: what remains open

Two gaps survive the atomic-write fix.

The first is handled by the partial index above, but the implementation detail matters. Storing idempotency keys in Redis with a Redis TTL creates a silent race: the key can expire while the operation is still processing. A new request arrives, finds the key absent, and proceeds as though it is the first — while the original is still running. Application-layer expiry via `created_at` and a server-side check is safer. The partial index approach shown above achieves this without a background cleanup process.

The second gap is genuine and cannot be closed by idempotency-key design alone. Idempotency keys protect your database writes. They do not protect operations that cross a network boundary: card charges, email sends, webhook dispatches.

If the process sends an email at step 3 and crashes before updating the idempotency key status at step 4, the retry sees a 'processing' key, waits or returns 409, but the email is already gone. The idempotency key correctly prevented a duplicate write to your database. It did not prevent a duplicate email.

For third-party APIs that support their own idempotency keys — Stripe, Twilio, Braintree — pass the original client key (or a deterministic derivation like `HMAC(client_key, operation_name)`) as the downstream idempotency key. If the downstream call already ran, the vendor returns the previous result without repeating the operation.

For systems that offer no such guarantee, the honest answer is an outbox pattern: write the intended side effect to a database table in the same transaction as your business write, then process the outbox asynchronously. The outbox entry is idempotent by virtue of being a database row. The downstream system still needs to handle at-least-once delivery, but at least your intent is durably recorded before the network call happens.

The idempotency key buys you safety within your own system boundary. Crossing that boundary cleanly is a separate problem, and conflating the two is where most double-sends ultimately originate.