Distributed Systems

Circuit Breaker Pattern: Preventing Cascading Failures in Distributed Systems

Named after the electrical device for a reason: when a downstream dependency starts failing, retrying it harder doesn't help — it just spreads the damage. A circuit breaker trips on purpose to stop that spread.

Published July 6, 2026

Picture a checkout service that calls a payment service on every request. The payment service starts timing out under load. Without protection, every checkout request still tries to call it, waits for the full timeout, and ties up a thread doing nothing useful. Under enough concurrent load, the checkout service exhausts its own thread pool waiting on a dependency that isn't coming back soon — and now checkout is down too, even though its own code was never the problem. That's a cascading failure, and it's one of the most common ways a single struggling service takes an entire system down with it.

The three states

A circuit breaker wraps calls to a dependency and tracks recent success/failure outcomes. It moves between three states:

Closed — normal operation. Calls pass through to the dependency, and the breaker counts failures.
Open — after failures cross a threshold, the breaker "trips." Calls fail immediately without even attempting the dependency, for a configured cooldown period. This is the key behavior: it stops piling more load onto something that's already struggling, and it stops your own threads from blocking on a call that's unlikely to succeed.
Half-open — after the cooldown, the breaker lets a small number of test calls through. If they succeed, it closes again and resumes normal traffic. If they fail, it reopens and waits another cooldown period.

A minimal implementation

import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, cooldown_seconds=30):
        self.failure_threshold = failure_threshold
        self.cooldown_seconds = cooldown_seconds
        self.failure_count = 0
        self.state = "closed"
        self.opened_at = None

    def call(self, func, *args, **kwargs):
        if self.state == "open":
            if time.time() - self.opened_at > self.cooldown_seconds:
                self.state = "half_open"
            else:
                raise Exception("circuit open: failing fast")

        try:
            result = func(*args, **kwargs)
        except Exception:
            self.failure_count += 1
            if self.failure_count >= self.failure_threshold:
                self.state = "open"
                self.opened_at = time.time()
            raise
        else:
            self.failure_count = 0
            self.state = "closed"
            return result

This is the shape of the pattern, not a production-ready implementation — real libraries add sliding time windows instead of raw counters (so five failures from an hour ago don't still count against you), configurable exception types worth tripping on versus ignoring, and metrics hooks so you can see state transitions on a dashboard rather than discovering them in logs after the fact.

Why "fail fast" is the actual benefit

The counterintuitive part is that an open circuit is a feature, not a degraded state to be ashamed of. Failing a request in 1ms with a clear error is far better for system health than waiting the full 30-second timeout on every request while a dependency is down — it frees up threads, keeps latency predictable for callers, and gives the failing dependency room to recover instead of getting hammered by retries from every caller simultaneously. The Azure architecture documentation describes this precisely: the goal is "to prevent an application from repeatedly trying to execute an operation that's likely to fail," as detailed in Microsoft's Circuit Breaker pattern reference.

Pairing it with a fallback

A circuit breaker alone just turns a slow failure into a fast one. Combined with a fallback — serve cached data, show a degraded UI, queue the request for later — it turns an outage into a graceful degradation. A product page that can't reach the live inventory service but falls back to "last known stock level, updated 4 minutes ago" is a far better user experience than either a hung request or a blank error page.

Where it doesn't help

Circuit breakers protect callers from a failing dependency; they don't fix the dependency. If the payment service is down, tripping the breaker stops checkout from also going down, but customers still can't pay. It's a resilience pattern for the caller's own stability, meant to be paired with proper retries with backoff for transient blips, timeouts sized to the actual expected latency, and monitoring on the dependency itself — not a substitute for any of those.