Rate Limiting & Quota Contracts

Part of the Error Contracts & Resilience Mapping reference. Rate limiting is usually treated as an infrastructure concern that happens to clients. Treated as a contract instead — published headers, a documented 429 body, an honest Retry-After — it becomes something clients can cooperate with, which is the difference between a well-paced integration and a retry storm.

Problem framing

An undocumented limit produces a predictable sequence. A client integrates, works fine in testing, then hits production volume and starts receiving 429s with an empty body. Having no information about how long to wait, its retry loop fires again immediately, the limiter rejects it again, and the client’s own latency budget is consumed by rejections. Support tickets follow, and the answer — “you are allowed 600 requests per minute per key” — was never anywhere the client could read it.

Every part of that is fixable in the contract. Publish the limit, the remaining allowance and the reset moment on every response so a client can pace itself. When you do reject, say why in a machine-readable body, and say when to come back in a header every HTTP library already understands. The rejection then becomes a normal, handled outcome rather than an error, which is exactly the classification discipline described in Retryable vs Non-Retryable Errors.

Choosing the limit model

The model you implement determines what your headers can honestly say. Three are common, and they behave very differently at the window boundary.

Fixed windows are trivially implementable and let a client legitimately send double the intended rate across a boundary. Sliding windows fix that at the cost of holding more state. Token buckets express “sustained rate plus burst” directly, which is usually what you actually mean, and they make the reset hint approximate rather than exact — an honest header must say so.

Spec definition

Model the limit surface as response headers plus a 429 response reusing the shared problem schema:

# openapi.yaml (OpenAPI 3.1.0) — the rate limit contract
openapi: 3.1.0
info:
  title: Messaging API
  version: "2.3.0"
components:
  headers:
    RateLimitPolicy:
      description: The policy that applied, e.g. "default";q=600;w=60
      schema: { type: string }
      example: '"default";q=600;w=60'
    RateLimit:
      description: Remaining allowance and seconds until reset for the applied policy.
      schema: { type: string }
      example: '"default";r=417;t=23'
    RetryAfter:
      description: RFC 9110 §10.2.3 — seconds to wait before retrying.
      schema: { type: integer, minimum: 0 }
      example: 23
  schemas:
    RateLimitProblem:
      allOf:
        - $ref: "#/components/schemas/Problem"
        - type: object
          required: [policy, limit, window_seconds, retry_after_seconds]
          properties:
            type:                { const: "urn:api:errors:v1:rate-limited" }
            policy:              { type: string, example: default }
            limit:               { type: integer, example: 600 }
            window_seconds:      { type: integer, example: 60 }
            retry_after_seconds: { type: integer, example: 23 }
            scope:               { type: string, enum: [api_key, user, ip, endpoint] }
paths:
  /messages:
    post:
      operationId: sendMessage
      responses:
        "202":
          description: Accepted for delivery.
          headers:
            RateLimit-Policy: { $ref: "#/components/headers/RateLimitPolicy" }
            RateLimit:        { $ref: "#/components/headers/RateLimit" }
        "429":
          description: The caller exceeded its quota for this policy.
          headers:
            RateLimit-Policy: { $ref: "#/components/headers/RateLimitPolicy" }
            RateLimit:        { $ref: "#/components/headers/RateLimit" }
            Retry-After:      { $ref: "#/components/headers/RetryAfter" }
          content:
            application/problem+json:
              schema: { $ref: "#/components/schemas/RateLimitProblem" }

Documenting the headers is what makes them reachable from a generated SDK — an undocumented header exists on the wire but not in any client type, so nobody uses it.

Standard alignment

The important nuance: the modern RateLimit header fields are a draft, not a finished RFC, and their syntax changed across draft revisions. The stable parts of your contract are 429, Retry-After and the problem body; the header fields are the ergonomic layer on top. That split is explored in RateLimit Header Fields vs X-RateLimit.

Implementation: emitting the contract server-side

// rate-limit.ts — Express middleware emitting the full contract
import type { Request, Response, NextFunction } from "express";

interface Decision {
  allowed: boolean;
  policy: string;      // "default"
  limit: number;       // 600
  window: number;      // 60 seconds
  remaining: number;   // 417
  resetSeconds: number;// 23
  scope: "api_key" | "user" | "ip" | "endpoint";
}

export function rateLimit(check: (req: Request) => Promise<Decision>) {
  return async (req: Request, res: Response, next: NextFunction) => {
    const d = await check(req);

    // Emitted on EVERY response, not only rejections.
    res.setHeader("RateLimit-Policy", `"${d.policy}";q=${d.limit};w=${d.window}`);
    res.setHeader("RateLimit", `"${d.policy}";r=${d.remaining};t=${d.resetSeconds}`);

    if (d.allowed) return next();

    res.setHeader("Retry-After", String(d.resetSeconds));
    res.status(429).type("application/problem+json").json({
      type: "urn:api:errors:v1:rate-limited",
      title: "Rate limit exceeded",
      status: 429,
      detail: `Quota of ${d.limit} requests per ${d.window}s for policy "${d.policy}" is exhausted.`,
      policy: d.policy,
      limit: d.limit,
      window_seconds: d.window,
      retry_after_seconds: d.resetSeconds,
      scope: d.scope,
    });
  };
}

Two details do the heavy lifting. Retry-After is set as an integer number of seconds, which every HTTP client understands without parsing your conventions. And scope tells the client what was limited — an integration hitting a per-endpoint limit can shift work elsewhere, while one hitting a per-key limit must slow down globally.

Implementation: cooperating client-side

A client that only reacts to 429 is already too late. The useful client watches the remaining allowance and paces itself before rejection:

# pacing_client.py — pace requests using the advertised allowance
import asyncio, re, time
import httpx

_RATELIMIT = re.compile(r'r=(\d+).*?t=(\d+)')

class PacedClient:
    """Slows down as the advertised allowance runs low; obeys Retry-After on 429."""

    def __init__(self, client: httpx.AsyncClient, floor: int = 50):
        self._client = client
        self._floor = floor            # start pacing below this many remaining

    async def request(self, method: str, url: str, **kw) -> httpx.Response:
        while True:
            resp = await self._client.request(method, url, **kw)

            if resp.status_code == 429:
                wait = int(resp.headers.get("Retry-After", "1"))
                await asyncio.sleep(wait)
                continue               # a 429 with Retry-After is a schedule, not a failure

            m = _RATELIMIT.search(resp.headers.get("RateLimit", ""))
            if m:
                remaining, reset = int(m.group(1)), int(m.group(2))
                if remaining < self._floor and remaining > 0:
                    # Spread the remaining allowance evenly over the rest of the window.
                    await asyncio.sleep(reset / remaining)
            return resp

Pacing beats retrying because it keeps the client inside its budget instead of spending latency on rejected calls. Where a rejection does happen, the backoff policy should still apply the jitter rules from Configuring Exponential Backoff for 5xx Errors so a fleet of clients does not resynchronise on the same reset boundary.

Edge-case handling

Multiple policies applying at once. A request can be inside its per-key limit but outside a per-endpoint one. Emit one RateLimit-Policy/RateLimit pair per policy (structured fields allow a list) and set Retry-After from the soonest reset the caller must wait for.

Limits behind a shared gateway. When the gateway limits and the service also limits, two independent counters exist and neither header describes the whole truth. Pick one enforcement point, and have the other report only.

Clock skew on absolute resets. A reset expressed as an absolute timestamp is wrong on any client whose clock is off. Prefer relative seconds; if you must expose an absolute time, also send Date so the client can compute the offset.

Validation and testing patterns

Two tests are worth having permanently. The first asserts the headers exist on success, which is the requirement teams forget:

// rate-limit.contract.test.ts
import { describe, it, expect } from "vitest";

describe("rate limit contract", () => {
  it("advertises the allowance on a successful response", async () => {
    const res = await fetch(`${BASE}/messages`, { method: "POST", body: "{}" });
    expect(res.status).toBe(202);
    expect(res.headers.get("RateLimit-Policy")).toMatch(/q=\d+;w=\d+/);
    expect(res.headers.get("RateLimit")).toMatch(/r=\d+;t=\d+/);
  });

  it("rejects with a parseable problem document and Retry-After", async () => {
    let res: Response;
    do { res = await fetch(`${BASE}/messages`, { method: "POST", body: "{}" }); }
    while (res.status !== 429);

    expect(res.headers.get("Content-Type")).toContain("application/problem+json");
    expect(Number(res.headers.get("Retry-After"))).toBeGreaterThanOrEqual(0);
    const body = await res.json();
    expect(body.type).toBe("urn:api:errors:v1:rate-limited");
    expect(body.retry_after_seconds).toBe(Number(res.headers.get("Retry-After")));
  });
});

The final assertion — header and body agreeing — catches the most common production inconsistency, where the two values are computed independently and drift apart under load.

# .spectral.yaml — require the contract wherever a 429 is documented
rules:
  ratelimit-429-needs-retry-after:
    description: A documented 429 must document Retry-After.
    severity: error
    given: "$.paths[*][*].responses['429']"
    then:
      field: headers.Retry-After
      function: truthy

  ratelimit-429-needs-problem-body:
    description: A 429 must return application/problem+json.
    severity: error
    given: "$.paths[*][*].responses['429'].content"
    then:
      field: "application/problem+json"
      function: truthy

SDK generation impact

Documented headers become accessible values; undocumented ones do not exist as far as a generated client is concerned:

  // Generated TypeScript, response envelope
  export interface SendMessageResponse {
    status: number;
    data: MessageAccepted;
+   headers: {
+     "RateLimit-Policy"?: string;   // documented → typed accessor
+     "RateLimit"?: string;
+   };
  }

The 429 is equally consequential: with a documented application/problem+json schema, generators emit a typed error carrying retry_after_seconds, so a caller can branch on it. Without one, the client throws a generic HTTP error and the retry hint is stranded in an untyped response object. The mapping from status code to client behaviour is covered in HTTP Status Code Mapping.

Anti-patterns quick-reference

Where the limiter belongs

Placement decides what the limit actually protects. A limiter at the edge protects everything behind it, rejects before any body is parsed, and can honestly promise that a rejected request had no side effect — which is what makes a 429 safe to retry automatically. A limiter inside the application can see business context the edge cannot, such as a per-tenant plan or a per-object quota, but by the time it runs the request has already consumed a connection, a worker and a body parse.

Most estates need both, and the failure is running them without coordination: two independent counters, two sets of headers, and neither one describing the whole truth. Pick one enforcement point per policy, let the other report only, and make sure the headers a client sees come from whichever component actually decided.

FAQ

Related

• Error Contracts & Resilience Mapping — up-link: the parent reference covering status mapping, retries and fallbacks
• RateLimit Header Fields vs X-RateLimit — choosing between the draft standard fields and the de facto convention
• Returning 429 with Problem+JSON — the rejection body in detail
• Documenting Rate Limits in OpenAPI — making limits visible to generated clients
• Retryable vs Non-Retryable Errors — the sibling section classifying which failures may be retried
• Client Fallback Strategies — what a client does once it is out of allowance