Exposing Pagination Metadata Without Breaking Clients

This page belongs to Response Envelopes & Metadata in the Query Patterns & Data Shaping Strategies reference. Adding cursors and counts to an endpoint that already has consumers is one of the most common evolution tasks in an API’s life, and one of the easiest to get subtly wrong — the failure mode is not an error but silently truncated results.

The decision trigger

Change you want to make Breaking?
Add meta.next_cursor to an enveloped response No — additive
Add meta to a bare array response Yes — top-level type changes
Add an opt-in include_total parameter No — default behaviour unchanged
Change the default limit from unlimited to 50 Yes — silently truncates results
Lower the maximum limit from 1000 to 100 Yes for callers using more
Add a Link header with rel="next" No — headers are ignored if unknown
Change next_cursor from absent to null when exhausted Depends on client strictness
Reorder members in the JSON object No — JSON objects are unordered

Spec snippet: additive by construction

# openapi.yaml — every new member is optional and defaults to today's behaviour
components:
  schemas:
    PageMeta:
      type: object
      required: [has_more]              # only the invariant is required
      properties:
        has_more:     { type: boolean }
        next_cursor:  { type: [string, "null"] }
        total_count:  { type: [integer, "null"], description: Only with include_total=true. }
        page_size:    { type: integer, description: Echo of the applied limit. }
paths:
  /invoices:
    get:
      parameters:
        - name: limit
          in: query
          description: >
            Page size. Omitting it preserves the historical behaviour of this
            endpoint; it does NOT impose a default page size.
          schema: { type: integer, minimum: 1, maximum: 500 }
        - name: include_total
          in: query
          schema: { type: boolean, default: false }

The comment on limit encodes the crucial decision: an endpoint that historically returned everything must keep doing so until its unbounded behaviour is formally deprecated. A default page size introduced quietly is the most damaging “non-breaking” change in this whole area, because clients receive a well-formed 200 containing a fraction of the data and no indication anything is missing.

Why a retroactive default page size is dangerous With a new default limit the client receives 50 of 4000 rows and a successful status, so a nightly job silently processes a fraction of the data. With an opt-in limit the client still receives everything until it asks to paginate. Default limit introduced Limit stays opt-in GET /invoices (unchanged client) 200 OK · 50 of 4000 rows no error, no warning nightly reconciliation misses 3950 discovered weeks later GET /invoices (unchanged client) 200 OK · all 4000 rows behaviour preserved clients adopt ?limit= when ready unbounded form deprecated on a date

Step-by-step

Step 1 — Add metadata as new optional members

// list-invoices.ts — additive metadata, historical default preserved
const MAX_LIMIT = 500;

export async function listInvoices(req: Request, res: Response) {
  const requested = req.query.limit ? Number(req.query.limit) : null;
  const limit = requested ? Math.min(requested, MAX_LIMIT) : null;   // null = unbounded, as before

  const page = await queryInvoices({ cursor: req.query.cursor as string | undefined, limit });

  const meta: Record<string, unknown> = {
    has_more: page.nextCursor !== null,
    next_cursor: page.nextCursor,
  };
  if (limit !== null) meta.page_size = limit;

  if (req.query.include_total === "true") {
    const { count, estimate } = await countInvoices(req.query);
    meta.total_count = count;
    meta.total_is_estimate = estimate;
  }

  res.json({ data: page.rows, meta });      // `data` unchanged for existing clients
}
// link-header.ts — RFC 8288 relations alongside body metadata
export function setPageLinks(res: Response, base: string, nextCursor: string | null) {
  const links: string[] = [`<${base}>; rel="self"`];
  if (nextCursor) {
    const url = new URL(base, "https://api.example.com");
    url.searchParams.set("cursor", nextCursor);
    links.push(`<${url.pathname}${url.search}>; rel="next"`);
  }
  res.setHeader("Link", links.join(", "));
}

Header links cost almost nothing and serve two audiences the body cannot: generic HTTP tooling that follows rel="next" automatically, and any endpoint still returning a bare array where the body has no room for metadata.

Step 3 — Prove the change is additive

# Spec-level: classify the diff before merging
docker run --rm -v "$PWD:/specs" tufin/oasdiff:latest \
  breaking /specs/openapi.previous.yaml /specs/openapi.yaml --fail-on ERR

# Behaviour-level: replay recorded production requests against both builds
python3 - <<'PY'
import json, subprocess, sys

recorded = [json.loads(l) for l in open("recorded-requests.jsonl")]
mismatches = 0
for r in recorded:
    old = subprocess.run(["curl", "-s", f"http://old.local{r['path']}"], capture_output=True).stdout
    new = subprocess.run(["curl", "-s", f"http://new.local{r['path']}"], capture_output=True).stdout
    o, n = json.loads(old), json.loads(new)
    # The invariant: the items each caller sees must be unchanged.
    o_items = o if isinstance(o, list) else o.get("data")
    n_items = n if isinstance(n, list) else n.get("data")
    if o_items != n_items:
        print("MISMATCH", r["path"], file=sys.stderr)
        mismatches += 1
print(f"{len(recorded)} replayed, {mismatches} mismatch(es)")
sys.exit(1 if mismatches else 0)
PY

The replay is what catches the truncation class of bug, because a spec diff sees a new optional parameter and correctly calls it additive — it cannot know that the default changed underneath. The general mechanics of classification are in Detecting Breaking Changes with oasdiff.

Step 4 — Retire unbounded responses deliberately

Introducing a page-size cap without silent truncation Stage one adds opt-in pagination while unbounded requests still return everything. Stage two keeps serving everything but adds a deprecation warning and measures who is affected. Stage three enforces the cap and rejects unbounded requests with an explicit error. 1 · opt-in ?limit= and ?cursor= added no default applied nothing breaks 2 · warn warnings[] on unbounded calls Sunset header set affected callers identified 3 · enforce cap applied after the date explicit 400, never silent truncation an error a client can see beats a page a client cannot detect

During stage two, warn in-band so the signal reaches the running integration rather than a mailing list:

{
  "data": [ "… 4000 items …" ],
  "meta": { "has_more": false },
  "warnings": [
    {
      "code": "unbounded_collection_deprecated",
      "message": "Requests without ?limit will be capped at 500 items from 2026-11-30. Adopt cursor pagination."
    }
  ]
}

Two verification passes are needed because they catch different failures:

What the spec diff catches versus what a replay catches A specification diff sees structural changes such as new members and parameters. A replay of recorded requests catches behavioural changes such as an altered default page size. spec diff traffic replay new optional member: additive new parameter: additive removed member: breaking default value change: invisible compares real response bodies catches a changed default page size catches a reversed sort order only covers recorded paths the dangerous change in this area is precisely the one a diff calls additive

Standard compliance

Element Standard Clause Note
Link relations RFC 8288 §3 next, prev, first, last, self
Tolerant reading RFC 9110 §16.3.1 Unrecognised fields should be ignored
Deprecation signalling RFC 8594 §2, §3 Announce the cap before enforcing it
400 Bad Request RFC 9110 §15.5.1 For a limit above the maximum
JSON object ordering RFC 8259 §4 Members are unordered — reordering is safe

SDK / codegen downstream effect

  export interface PageMeta {
    has_more: boolean;
+   next_cursor?: string | null;
+   total_count?: number | null;
+   page_size?: number;
  }

All three additions are optional properties, so previously generated clients still compile against the new type and newly generated ones gain access to the metadata. That is the whole test for an additive response change in a statically typed language: does the old code still compile against the new type. Publishing that new SDK version follows the semver rules in Semver Ranges for Generated SDK Clients.

Common mistakes

Mistake Correct approach
Introducing a default page size retroactively Keep unbounded behaviour until it is formally deprecated
Lowering the maximum limit without notice Treat it as breaking; announce and measure first
Repurposing an existing field for a cursor Add a new member; never change a field’s meaning
Relying on a spec diff alone Replay recorded requests to catch default-value changes
Putting links only in the Link header for enveloped responses Put them in the body and mirror in the header
Silently truncating instead of erroring An explicit 400 is safer than a partial 200

FAQ

Is adding a field to a response a breaking change?

For the great majority of clients, no: RFC 9110 §16.3.1 expects unrecognised content to be ignored, and every mainstream JSON deserialiser skips unknown members by default. It is breaking for two populations — clients that validate responses against a schema in strict mode, and clients that deserialise into closed types with additionalProperties: false semantics. State tolerant reading as an expectation in your API documentation so the contract is explicit, and if you have a small, known consumer set, check them before relying on the assumption.

Should pagination links go in the body or the Link header?

If the response has an envelope, put them in the body and mirror them in the Link header. The body is where client developers look, where documentation renders them and where a typed SDK can expose them; the header is the standardised location that generic tooling already understands, and it is the only option for endpoints still returning a bare array. Emitting both costs a few dozen bytes and removes the argument entirely. What does not work is header-only links on an enveloped API — they get lost behind proxies, forgotten by client authors, and stripped by well-meaning middleware.

How do I introduce pagination on an endpoint that never had it?

In three stages, and never by adding a default. First add limit and cursor as optional parameters while an omitted limit continues to return everything — that is purely additive. Second, keep returning everything but attach an in-band warning and a Sunset date to unbounded requests, and use your access logs to identify exactly which callers are affected. Third, after the announced date, enforce the cap by rejecting unbounded requests with a 400 that names the parameter to use. The one thing to avoid at every stage is returning a truncated page with a 200, because a client cannot distinguish that from a genuinely small result set.

Back to Response Envelopes And Metadata