Idempotency

An operation is idempotent if performing it many times has the same effect as performing it once. Setting a user's email to a@example.com is idempotent; adding $10 to their balance is not. In a distributed system this distinction decides whether a network timeout is a shrug or an incident.

The reason it matters is unavoidable: the client can never know whether a request that timed out was processed. The response may have been lost on the way back. The client's only options are to retry (risking a duplicate) or give up (risking a lost write). Idempotency is what makes "retry" the safe choice — and retries are how every reliable system is built.

TL;DR

Quick Example

A payment endpoint that is safe to call twice:

Core Concepts

Idempotent vs. safe vs. deterministic

Three properties that get conflated:

Every safe operation is idempotent. An idempotent operation need not be safe (DELETE changes state) and need not return identical responses (DELETE may return 204 then 404) — what must be identical is the resulting state.

HTTP method semantics

Intermediaries rely on this. HTTP client libraries and load balancers will silently retry idempotent methods on connection failure — so a GET that mutates state will be executed twice in production.

Idempotency keys

A key is an opaque, client-generated, globally unique string (a UUID v4 is standard) that identifies one logical operation, not one HTTP attempt. The contract:

The critical, frequently botched rule: the client generates the key before the first attempt and reuses it for every retry. Generating a new UUID inside the retry loop makes the whole mechanism decorative.

Natural idempotency

Before reaching for keys, ask whether the operation can just be idempotent. This is almost always cheaper and more robust:

Client-supplied resource IDs make PUT the natural verb and idempotency disappears as a concern:

Implementing an Idempotency Layer

The dedupe table

Scope keys per user or per API credential (PRIMARY KEY (user_id, key) also works) so one tenant's UUID collision can't leak or block another's request.

Three states, three behaviors

Returning 409 for concurrent duplicates is the honest answer. The alternative — blocking until the first attempt finishes — ties up a connection and just moves the timeout.

Why the UNIQUE constraint is non-negotiable

Read-then-write does not work under concurrency:

Let the database arbitrate: INSERT first, and treat the unique-violation as "someone else owns this key." That is the only check-and-act sequence that is atomic. See Database Transactions.

Store the key and the effect atomically

If the charge commits but the key insert fails (or vice versa), you have re-introduced the bug you were fixing. Keep both in one local transaction. When the side effect is in an external system, propagate the key to it — every serious payment API accepts one:

Expiry

Keys are not permanent. 24 hours is the common window (Stripe's default); it comfortably covers any sane retry policy. Sweep expired rows on a schedule so the table stays small — see Background Jobs.

Idempotent Message Consumers

Queues deliver at-least-once. Redelivery happens on consumer crash, visibility-timeout expiry, or rebalance — so every consumer must tolerate seeing the same message twice.

Two subtleties that bite:

When the effect lives in another system that can't join your transaction, use the transactional outbox pattern: write the effect-intent to an outbox table in the business transaction, and have a relay publish it with retries. See Message Queues and Apache Kafka.

⚠️ Warning: Kafka's "exactly-once semantics" applies to Kafka-to-Kafka processing with transactions. The moment your consumer calls an external API, you are back to at-least-once and need idempotency at that boundary.

Best Practices

Make the client own the key's lifetime

Generate the key at the point the user initiates the action — form submit, job enqueue — and thread it through every retry, including retries after a process restart. A key stored alongside the pending operation survives a client crash; a key held in a local variable does not.

Validate that the same key means the same request

Hash the canonical request body and compare on replay. A key reused with a different payload is a client bug, and silently replaying the old response hides it. Return 422.

Return the original status code, not just the body

If the first attempt returned 201 Created with a Location header, the replay should too. Clients branch on status codes.

Design webhooks to be replayed

Your own webhooks will be redelivered — send a stable event_id, and document that consumers must dedupe on it. Reciprocally, dedupe on the provider's event ID when you consume.

Pair idempotency with bounded, jittered retries

Idempotency makes retries safe; it doesn't make them free. Combine with exponential backoff plus jitter and a circuit breaker, or you'll idempotently DDoS your own database during an incident. See Rate Limiting.

Keep read-modify-write off the hot path

balance = balance + 10 executed twice is wrong. INSERT INTO ledger_entries (...) executed twice is caught by a unique constraint, and the balance is derived. An append-only ledger is idempotency-friendly by construction — see Event-Driven Architecture.

Common Mistakes

Regenerating the key inside the retry loop

Check-then-insert instead of insert-then-catch

Deduplicating in a cache that can evict

A GET with side effects

Treating a timeout as a failure

FAQ

Is PUT always idempotent?

By HTTP semantics, yes — that's the contract you're promising. Your implementation must honor it: a PUT that appends to a list or increments a counter violates the spec, and infrastructure that auto-retries PUT will corrupt data.

What's the difference between exactly-once delivery and exactly-once processing?

Exactly-once delivery over an unreliable network is provably impossible — the acknowledgment itself can be lost. Exactly-once processing (or "effectively once") is achievable: deliver at-least-once and make the processing idempotent. Every real system does the latter.

How long should I keep idempotency keys?

Long enough to outlast any client's retry policy — 24 hours is the industry norm and matches Stripe. Extend it for slow human-in-the-loop flows, but bound it, and return a clear error rather than silently re-executing an expired key.

Should the client or server generate the idempotency key?

The client. The server can't distinguish a retry from a genuinely new identical request — only the caller knows its own intent. Server-generated keys defeat the purpose.

Do I need idempotency keys if I use a database transaction?

Yes. A transaction guarantees atomicity of one execution; it says nothing about a second HTTP request arriving after the first committed. Transactions and idempotency keys solve different failure modes and are used together.

How does this interact with distributed transactions?

Idempotency is a prerequisite for the saga pattern: each step and each compensating action must be safely retryable, because a saga orchestrator will retry them.

Related Topics

References