When a Service Is Down, Stop Calling It. Waiting in Line Kills Everything Else

When one service calls another that has become slow or unresponsive, each call waits for a timeout before giving up. Requests arriving meanwhile pile up waiting for the same thing, each holding a thread, a connection and memory, until the calling service exhausts its own resources and stops responding to anything at all. Its own callers then queue up behind it. A fault in one unimportant component therefore takes down an entire system, one layer at a time, and the collapse is faster than any human response.

The pattern borrows both the name and the behaviour from electrical protection:

• •Failures against a dependency are counted over a recent window
• •Past a threshold, the switch opens and calls stop being attempted
• •Callers get an immediate failure instead of a long wait
• •After a cooling period, a single trial call is allowed through
• •If it succeeds, normal operation resumes
• •If it fails, the switch opens again and the wait restarts

Refusing to try looks like giving up and is the opposite. An immediate failure returns the caller's resources at once, so the calling service stays healthy and continues serving everything that does not depend on the broken component. It also stops the struggling dependency being hammered by retries at exactly the moment it is trying to recover, which is frequently what prevents recovery in the first place. And it converts an ambiguous slow failure, which is the hardest kind to diagnose, into an explicit and immediate one that monitoring can see.

Every number in the design is a judgement and getting one wrong makes the mechanism worse than nothing. A threshold set too low opens the switch during ordinary transient errors and cuts off a service that was working. Set too high it never opens and provides no protection. A cooling period that is too short hammers a recovering dependency with trial calls, and one that is too long leaves a feature broken after the fault is fixed. Rates work better than raw counts, because ten failures out of ten thousand calls is very different from ten out of twelve.

Opening the switch is only half the design, since something must happen in place of the call, and that is where most of the thought goes. A cached previous answer may be good enough, particularly for data that changes slowly. A default value may be acceptable, such as showing a generic list where a personalised one is unavailable. The feature may be hidden entirely, degrading the page rather than breaking it. Or the request may be queued for later where the operation is not needed immediately. What is never acceptable is a stack trace reaching a user.

Calls to a slow dependency pile up holding resources until the caller collapses and its own callers follow, so counting recent failures and then refusing to try at all keeps the caller healthy. A single trial call after a cooling period tests whether to resume. Refusing also stops retries hammering a dependency trying to recover, and something sensible must be returned instead.