Circuit Breaker Thresholds for HTTP Clients

This page belongs to Client Fallback Strategies in the Error Contracts & Resilience Mapping reference. A breaker with default thresholds is worse than none: it either never trips when a dependency is failing, or trips constantly on normal error rates and turns a partial degradation into a total one. The numbers are the design.

The decision trigger

Spec snippet: the five parameters

// breaker-config.ts — every parameter, with defaults that actually work
export interface BreakerConfig {
  /** Fraction of failed calls that opens the circuit. 0.5 = half of all calls. */
  failureRateThreshold: number;
  /** Minimum calls in the window before the rate is evaluated at all. */
  minimumThroughput: number;
  /** Rolling window over which the rate is computed. */
  windowMs: number;
  /** How long the circuit stays open before a probe is allowed. */
  openDurationMs: number;
  /** Consecutive successful probes required to close it again. */
  halfOpenSuccesses: number;
}

export const DEFAULTS: BreakerConfig = {
  failureRateThreshold: 0.5,    // half the calls failing is unambiguous
  minimumThroughput: 20,        // never judge on fewer than 20 calls
  windowMs: 10_000,             // 10s rolling window
  openDurationMs: 5_000,        // probe again after 5s (plus jitter)
  halfOpenSuccesses: 3,         // three good probes before full traffic
};

minimumThroughput is the parameter most often missing, and its absence produces the classic overnight failure: at 3 a.m. the endpoint receives two calls, one fails, the rate is 50%, and the breaker opens for a service that is perfectly healthy.

Step-by-step

Step 1 — Decide what counts as a failure

// failure-classification.ts — what trips the breaker, and what does not
export function countsAsFailure(outcome: CallOutcome): boolean {
  if (outcome.kind === "transport") return true;          // connect/read timeout, reset
  const s = outcome.status;

  if (s >= 500) return true;                              // the dependency is unwell
  if (s === 429) return true;                             // systemic: we are over quota
  if (s === 401 || s === 403) return true;                // systemic: credentials wrong

  // 400, 404, 409, 412, 422 — THIS request was wrong. Not the dependency's health.
  return false;
}

Counting ordinary 4xx responses is the most common misconfiguration. A single caller sending malformed payloads should not open a shared circuit and deny service to every other caller — and it will, because a breaker is usually a process-wide singleton. 401 and 429 are the deliberate exceptions: both mean every request from this client will fail until something changes, which is exactly the condition a breaker exists to detect. The underlying classification is the same one in Retryable vs Non-Retryable Errors.

Step 2 — Scope the breaker to what fails together

Per-operation is the usual right answer. A slow report-generation endpoint should not disable the login endpoint of the same service, and in practice they often fail independently because they touch different datastores.

Step 3 — Implement with jitter on recovery

// breaker.ts — a rolling-window breaker with jittered recovery
export class CircuitBreaker {
  private state: "closed" | "open" | "half_open" = "closed";
  private calls: Array<{ at: number; failed: boolean }> = [];
  private openedUntil = 0;
  private probeSuccesses = 0;
  private probeInFlight = false;

  constructor(private readonly cfg: BreakerConfig = DEFAULTS,
              private readonly now = () => Date.now()) {}

  async run<T>(operation: () => Promise<T>, fallback: () => T): Promise<T> {
    const t = this.now();

    if (this.state === "open") {
      if (t < this.openedUntil) return fallback();          // fail fast
      this.state = "half_open";
      this.probeSuccesses = 0;
    }

    if (this.state === "half_open") {
      if (this.probeInFlight) return fallback();            // one probe at a time
      this.probeInFlight = true;
    }

    try {
      const result = await operation();
      this.record(false, t);
      return result;
    } catch (e) {
      this.record(true, t);
      throw e;
    } finally {
      if (this.state === "half_open") this.probeInFlight = false;
    }
  }

  private record(failed: boolean, t: number) {
    this.calls.push({ at: t, failed });
    this.calls = this.calls.filter(c => c.at > t - this.cfg.windowMs);

    if (this.state === "half_open") {
      if (failed) return this.trip(t);
      if (++this.probeSuccesses >= this.cfg.halfOpenSuccesses) {
        this.state = "closed";
        this.calls = [];
      }
      return;
    }

    if (this.calls.length < this.cfg.minimumThroughput) return;   // not enough evidence
    const rate = this.calls.filter(c => c.failed).length / this.calls.length;
    if (rate >= this.cfg.failureRateThreshold) this.trip(t);
  }

  private trip(t: number) {
    this.state = "open";
    // Jitter prevents a fleet of clients probing in lockstep.
    const jitter = Math.floor(Math.random() * this.cfg.openDurationMs * 0.3);
    this.openedUntil = t + this.cfg.openDurationMs + jitter;
    this.probeSuccesses = 0;
  }
}

Two properties matter beyond the thresholds. Only one probe is allowed in half-open, so a recovering service receives a trickle rather than the full fleet at once. And the open duration carries jitter, so a hundred client instances that tripped together do not all probe at the same millisecond — the thundering-herd failure that turns a brief outage into a repeating one.

Step 4 — Choose thresholds from the dependency’s normal behaviour

Start from the dependency’s observed error rate over a month, not from a blog post’s default. If a service normally errs at 3% with spikes to 15%, a 50% threshold distinguishes an outage cleanly, while a 10% threshold guarantees false trips. Where the normal rate is genuinely high, the breaker is the wrong tool — fix the error rate or use a bulkhead to bound the blast radius instead.

Thresholds are only half the tuning; the other half is what the client does while the circuit is open:

Standard compliance

If your service fails fast because a downstream circuit is open, return 503 with a Retry-After derived from the remaining open duration. Returning 500 tells your callers you are broken, when in fact you are deliberately shedding load for a dependency — a distinction that matters to their breakers.

SDK / codegen downstream effect

  // Generated client, wrapped once with a per-operation breaker
+ const breakers = new Map<string, CircuitBreaker>();
+
+ function guarded<T>(op: string, call: () => Promise<T>, fallback: () => T) {
+   const breaker = breakers.get(op) ?? breakers.set(op, new CircuitBreaker()).get(op)!;
+   return breaker.run(call, fallback);
+ }
+
+ export const invoices = {
+   get: (id: string) =>
+     guarded("invoices.get",
+       () => api.getInvoice({ invoiceId: id }),
+       () => cache.get(`invoice:${id}`) ?? { stale: true }),
+ };

Keying breakers by operationId gives per-endpoint scoping for free — one more reason the naming discipline in Naming Operation IDs for Readable SDK Methods pays off beyond readability.

Common mistakes

FAQ

Related

• Client Fallback Strategies — up-link: the section covering degradation and fallback behaviour
• Error Contracts & Resilience Mapping — up-link: the parent reference for failure contracts
• Retryable vs Non-Retryable Errors — the classification that decides what counts as a failure
• Configuring Exponential Backoff for 5xx Errors — retry behaviour before the breaker opens
• Safe Retries for Non-Idempotent POST — bounding retries on write operations