Distributed patterns: circuit breaker & idempotency

Circuit breaker: stop hammering a dead service

GoF predates the network being the hard part. In distributed systems the patterns that matter are about failure. A circuit breaker tracks failures to a dependency and, once they cross a threshold, opens — rejecting calls instantly instead of waiting on timeouts. After a cooldown it half-opens to test recovery.

// Three states: closed (normal), open (reject fast), half-open (probe).
type State = "closed" | "open" | "half-open";

interface Breaker {
  state: State;
  failures: number;
}

The point is to protect both sides: you stop wasting time on calls that will fail, and you give the struggling dependency room to recover instead of drowning it in retries.

Retries and idempotency go together

Retrying a failed call is the obvious response, but a blind retry can double-charge a customer or send two emails. Retries are only safe when the operation is idempotent — running it twice has the same effect as running it once. The standard tool is an idempotency key: the caller sends a unique key, and the server records the result per key, returning the stored result on repeats.

// Server-side idempotency: same key returns the same stored result.
const seen = new Map<string, unknown>();

function withIdempotency<T>(key: string, run: () => T): T {
  if (seen.has(key)) return seen.get(key) as T;
  const result = run();
  seen.set(key, result);
  return result;
}

Pair retries with exponential backoff and jitter so a fleet of clients does not retry in lockstep and create a thundering herd. These three — breaker, backoff, idempotency — are the table stakes a staff review will assume you know.