Generating Idempotency Keys in Node.js / Express APIs

This page is part of the Idempotency Key Implementation guide under API Design Fundamentals & Architecture. It covers the specific decisions and code required to generate secure keys on the client side, validate and deduplicate them inside Express middleware, and back the whole flow with an atomic storage layer — so that POST and PATCH endpoints behave safely under client retries, network timeouts, and load-balancer re-routes.

When This Problem Surfaces

Duplicate execution bugs are rarely obvious until they reach production. The following table maps the observable symptom to the underlying failure:

Symptom Root Cause First Check
Two database rows created for one client request Middleware runs after the route handler, or the key is absent Verify middleware order: json() → auth → idempotencyCheck → route
POST /orders returns 201 twice with different order_ids Client regenerated the key on retry SDK interceptor must freeze the key on first send
500 Internal Server Error with deadlock detected under load SELECT then INSERT race between concurrent pods Replace with atomic INSERT … ON CONFLICT DO NOTHING
200 OK returned but Idempotency-Key header is absent Reverse proxy or CORS preflight stripped the header Add Idempotency-Key to Access-Control-Allow-Headers and proxy proxy_pass_header
Silent duplicate charge No idempotency key on the payment endpoint at all Enforce the header as required: true in the OpenAPI spec

The moment any of these symptoms appear in staging load tests or production logs, the fix follows the same path: audit key generation entropy, confirm middleware position, and switch to an atomic storage write.

OpenAPI Spec Snippet

Anchor the contract in the spec before writing any Express code. The snippet below is valid OpenAPI 3.1.0 and should be referenced by every post and patch operation that mutates state:

components:
  parameters:
    IdempotencyKey:
      in: header
      name: Idempotency-Key
      required: true
      description: >
        Client-generated UUIDv4 or UUIDv7. Must be identical across retries
        of the same logical operation. Scoped per endpoint and tenant.
      schema:
        type: string
        format: uuid
        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$'
  responses:
    IdempotencyDuplicate:
      description: Key already processed — cached response returned.
      headers:
        X-Idempotency-Replayed:
          schema: { type: boolean }
        X-Idempotency-Key-Status:
          schema:
            type: string
            enum: [DUPLICATE, PROCESSING]
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Order'   # same payload as the original 201
    IdempotencyInFlight:
      description: A request with this key is still being processed.
      headers:
        Retry-After:
          schema: { type: integer }
        X-Idempotency-Key-Status:
          schema:
            type: string
            enum: [PROCESSING]

Pair this $ref pattern with a Spectral rule (shown in the CI section below) to prevent any future post or patch operation from skipping the parameter reference.

Request Flow: Key Lifecycle

The diagram below traces the path of an Idempotency-Key header from client send through Express middleware to the atomic storage check and back:

Idempotency Key Request Lifecycle in Express Sequence diagram showing a POST request with an Idempotency-Key flowing through Express middleware layers — JSON body parser, auth, idempotency check — before hitting the route handler. On a duplicate key the middleware short-circuits and returns the cached response without touching the route. Client Express Middleware Route Handler Storage Cache Hit? POST + Idempotency-Key 400 if header absent UPSERT / SETNX HIT: return cached payload 200 + X-Idempotency-Replayed MISS: allow through Business logic cache response 201 Created (first execution)

Step-by-Step Implementation

Step 1 — Generate the Key on the Client

The client is responsible for creating and preserving the key. Use crypto.randomUUID() (Node.js 14.17+, all modern browsers):

// TypeScript — axios interceptor that freezes the key across retries
import axios from 'axios';

const api = axios.create({ baseURL: 'https://api.example.com' });

api.interceptors.request.use((config) => {
  if (['post', 'patch'].includes((config.method ?? '').toLowerCase())) {
    // Only generate once — retries arrive with config already set
    config.headers['Idempotency-Key'] ??= crypto.randomUUID();
  }
  return config;
});
# Python — requests Session that auto-injects and preserves the key
import uuid, requests

class IdempotentSession(requests.Session):
    def request(self, method: str, url: str, **kwargs):
        if method.upper() in ('POST', 'PATCH'):
            headers = kwargs.setdefault('headers', {})
            headers.setdefault('Idempotency-Key', str(uuid.uuid4()))
        return super().request(method, url, **kwargs)

Never regenerate the key inside a retry loop. Generate once, attach to the request object, and let the retry library re-send the same config.

Step 2 — Validate the Header in Express Middleware

Wire the validation middleware before any route, after express.json() and authentication:

// middleware/idempotency-validate.ts
import { Request, Response, NextFunction } from 'express';

const UUID_RE =
  /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;

export function validateIdempotencyHeader(
  req: Request,
  res: Response,
  next: NextFunction
): void {
  // Only enforce on state-mutating methods
  if (!['POST', 'PATCH'].includes(req.method)) return next();

  const key = req.headers['idempotency-key'];
  if (typeof key !== 'string' || !UUID_RE.test(key)) {
    res.status(400).json({
      type: '/errors/invalid-idempotency-key',
      title: 'Idempotency-Key header is missing or not a valid UUID.',
      status: 400
    });
    return;
  }
  next();
}

Step 3 — Atomic Storage Check (PostgreSQL or Redis)

This is the critical section. Both options below guarantee exactly-one execution per key:

PostgreSQL (Prisma-compatible raw SQL):

// middleware/idempotency-check.ts  (PostgreSQL path)
import { Request, Response, NextFunction } from 'express';
import { pool } from '../db';   // pg Pool instance

export async function idempotencyCheckPg(
  req: Request,
  res: Response,
  next: NextFunction
): Promise<void> {
  if (!['POST', 'PATCH'].includes(req.method)) return next();

  const key = req.headers['idempotency-key'] as string;
  const tenantId = (req as any).tenantId as string;

  // Atomic: insert only when the row does not yet exist
  const { rowCount } = await pool.query(
    `INSERT INTO idempotency_keys (key, tenant_id, status, created_at)
     VALUES ($1, $2, 'PROCESSING', NOW())
     ON CONFLICT (key, tenant_id) DO NOTHING`,
    [key, tenantId]
  );

  if (rowCount === 0) {
    // Key already exists — return cached response if ready, else 409
    const cached = await pool.query(
      `SELECT status, response_body, response_status
         FROM idempotency_keys
        WHERE key = $1 AND tenant_id = $2`,
      [key, tenantId]
    );
    const row = cached.rows[0];
    if (row?.status === 'COMPLETED') {
      res.status(row.response_status)
         .set('X-Idempotency-Replayed', 'true')
         .set('X-Idempotency-Key-Status', 'DUPLICATE')
         .json(row.response_body);
    } else {
      res.status(409)
         .set('Retry-After', '2')
         .set('X-Idempotency-Key-Status', 'PROCESSING')
         .json({ type: '/errors/idempotency-in-flight', status: 409 });
    }
    return;
  }

  // Key is new — attach a post-response hook to cache the result
  res.on('finish', async () => {
    await pool.query(
      `UPDATE idempotency_keys
          SET status = 'COMPLETED',
              response_body  = $3,
              response_status = $4
        WHERE key = $1 AND tenant_id = $2`,
      [key, tenantId, (res as any)._body, res.statusCode]
    );
  });

  next();
}

Redis (single atomic SET NX EX):

// middleware/idempotency-check.ts  (Redis path)
import { createClient } from 'redis';
import { Request, Response, NextFunction } from 'express';

const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();

const TTL = 86_400; // 24 hours in seconds

export async function idempotencyCheckRedis(
  req: Request,
  res: Response,
  next: NextFunction
): Promise<void> {
  if (!['POST', 'PATCH'].includes(req.method)) return next();

  const key = `idemp:${req.method}:${req.path}:${req.headers['idempotency-key']}`;

  // NX = only set if not exists; EX = TTL in seconds — single round-trip, atomic
  const acquired = await redis.set(key, 'PROCESSING', { NX: true, EX: TTL });

  if (!acquired) {
    const cached = await redis.get(`${key}:response`);
    if (cached) {
      const { status, body } = JSON.parse(cached);
      res.status(status)
         .set('X-Idempotency-Replayed', 'true')
         .json(body);
    } else {
      res.status(409).set('Retry-After', '2').json({
        type: '/errors/idempotency-in-flight',
        status: 409
      });
    }
    return;
  }

  res.on('finish', async () => {
    await redis.set(
      `${key}:response`,
      JSON.stringify({ status: res.statusCode, body: (res as any)._body }),
      { EX: TTL }
    );
    await redis.set(key, 'COMPLETED', { XX: true, KEEPTTL: true });
  });

  next();
}

Step 4 — Wire the Middleware Stack

Order matters. express.json() must parse the body before authentication runs, and the idempotency check must intercept the request before the route handler executes:

// app.ts
import express from 'express';
import { validateIdempotencyHeader } from './middleware/idempotency-validate';
import { idempotencyCheckPg } from './middleware/idempotency-check';
import { requireAuth } from './middleware/auth';
import { orderRouter } from './routes/orders';

const app = express();

app.use(express.json({ limit: '1mb' }));  // 1 — parse body
app.use(requireAuth);                     // 2 — authenticate
app.use(validateIdempotencyHeader);       // 3 — format check
app.use(idempotencyCheckPg);              // 4 — dedup check
app.use('/v1/orders', orderRouter);       // 5 — route

export default app;

Step 5 — Contract Test (Jest + Supertest)

// tests/idempotency.test.ts
import request from 'supertest';
import app from '../app';
import { pool } from '../db';

afterAll(() => pool.end());

test('second request with same key returns cached response', async () => {
  const key = crypto.randomUUID();
  const payload = { sku: 'WIDGET-42', quantity: 1 };

  const first = await request(app)
    .post('/v1/orders')
    .set('Idempotency-Key', key)
    .send(payload);
  expect(first.status).toBe(201);

  const second = await request(app)
    .post('/v1/orders')
    .set('Idempotency-Key', key)
    .send(payload);
  expect(second.status).toBe(201);
  expect(second.headers['x-idempotency-replayed']).toBe('true');
  expect(second.body.id).toBe(first.body.id);   // same order, not a new one
});

test('missing key returns 400 with RFC 7807 body', async () => {
  const res = await request(app).post('/v1/orders').send({ sku: 'X' });
  expect(res.status).toBe(400);
  expect(res.body.type).toMatch(/invalid-idempotency-key/);
});

RFC and Standard Compliance

RFC 9110 (HTTP Semantics, Section 9.2) defines method safety and idempotency at the protocol level but delegates key-based deduplication to the application. The Stripe-style Idempotency-Key header is a widely adopted convention; IETF draft-ietf-httpapi-idempotency-key-header-04 documents it as a proposed standard. The 409 Conflict status code (RFC 9110 §15.5.10) is the correct response for an in-flight collision because the conflict is with the current state of the resource (an active processing lock), not a client format error. Use 400 Bad Request only for missing or malformed headers.

Idempotency and Safety Implications

The error response format used in the validation step above intentionally follows the RFC 7807 Problem JSON structure (type, title, status). This matters beyond aesthetics: SDK generators and API gateway policies that parse application/problem+json can automatically distinguish a format error (400) from a concurrency conflict (409) and route each to the appropriate retry or escalation path. Aligning idempotency error bodies with Problem JSON means your error contracts and your deduplication logic reinforce each other rather than producing inconsistent client behaviour.

SDK / Codegen Downstream Effect

When the OpenAPI spec marks Idempotency-Key as required: true in the parameter list, generators surface it differently depending on the target language. The diff below shows what changes between an absent parameter and a required one in a generated TypeScript Axios client:

// BEFORE — Idempotency-Key not in spec
 export async function createOrder(body: OrderRequest): Promise<Order> {
-  return api.post('/v1/orders', body);
+  // No header — silent duplicates under retries
 }

// AFTER — Idempotency-Key required: true in spec
 export async function createOrder(
   body: OrderRequest,
+  idempotencyKey: string          // compiler-enforced
 ): Promise<Order> {
+  return api.post('/v1/orders', body, {
+    headers: { 'Idempotency-Key': idempotencyKey }
+  });
 }

The required: true flag also causes generators to include the parameter in TypeScript interface types and Python dataclass fields. This compile-time enforcement is far more reliable than runtime logs for catching missing keys before code ships.

Common Mistakes

Mistake Correct Approach
Using Math.random() or Date.now() as the key source Use crypto.randomUUID() (Node.js 14.17+) — cryptographically secure, no collisions under distributed load
Placing idempotency middleware after the route handler Mount it before all routes; once the handler runs, the side-effect has already happened
SELECT then INSERT in two separate statements Use INSERT … ON CONFLICT DO NOTHING (PostgreSQL) or SET NX EX (Redis) — single atomic round-trip
Scoping the key by user ID alone Scope by (endpoint, HTTP method, tenant_id, client-supplied UUID) to prevent cross-resource collisions
No TTL on stored keys Set a 24–72 h TTL; unbounded growth exhausts Redis memory or PostgreSQL disk
Returning 200 OK without distinguishing replays Add X-Idempotency-Replayed: true so clients and observability tools can distinguish first execution from cached replay

FAQ

Should I use UUIDv4 or UUIDv7 for idempotency keys?

UUIDv4 is the safe default: pure randomness, zero risk of timestamp collision. UUIDv7 is preferable when storage is PostgreSQL with a B-tree index on the key column — the monotonically increasing prefix improves insert locality and reduces page splits under high write throughput. Both satisfy the cryptographic entropy requirement.

How do I prevent duplicate processing under concurrent retries?

Use a single atomic operation: INSERT … ON CONFLICT DO NOTHING in PostgreSQL, or SET NX EX in Redis. Never do a SELECT then INSERT — the gap between those two statements is a race window that concurrent pods will exploit.

What is the right TTL for idempotency key storage?

24–72 hours covers the vast majority of client retry windows. Align the TTL with your maximum observable retry horizon and any regulatory requirements for duplicate-charge prevention. Use Redis EXPIRE or a PostgreSQL created_at + interval scheduled cleanup job.

Back to Idempotency Key Implementation