The Code Is Already Shipped. It Is Just Switched Off
By the BrainSnail editorial team. How these articles are written and checked, and how to tell us when one is wrong.
Wrapping new behaviour in a switch that can be flipped without redeploying separates the act of shipping code from the act of turning it on, which changes how teams work.
What the switch does
A condition in the code checks a value held outside the code, usually in a configuration service, and takes one path or the other accordingly. The new behaviour ships to production in a disabled state, doing nothing, and is enabled later by changing the value. Nothing is redeployed and no build is run. Enabling can be limited to particular users, to a percentage of traffic or to a region, and disabling takes effect within seconds, which is far faster than reversing a deployment.
What the separation buys
Splitting release from deployment enables several practices at once:
- •Unfinished work can be merged continuously while hidden
- •Which avoids long-lived branches that become painful to merge
- •A feature can be shown to a selected group for early feedback
- •Two variants can be compared against each other in production
- •A misbehaving feature can be switched off without a deployment
- •Launch timing becomes a business decision rather than a technical one
The debt it creates
Every switch is a branch in the code and their number multiplies the states a system can be in, which is the standing complaint against the technique. Two switches give four combinations, ten give more than a thousand, and almost none of those combinations is ever tested. Old switches left in place after a feature is fully launched are pure clutter that future readers must reason about. The discipline required is to treat each one as temporary, record who owns it and when it should go, and remove it promptly, which is exactly the kind of tidying that gets deferred.
The kinds of switch
Grouping these by purpose makes clear which should be short-lived and which should not. A release switch hides unfinished work and should be removed within weeks of launch. An experiment switch divides traffic between variants and dies when the experiment concludes. An operational switch, sometimes called a kill switch, disables an expensive feature under load and is meant to live permanently. A permission switch controls which customers see which features and is really part of the product rather than a temporary measure. Treating all four the same way is how the clutter accumulates.
The famous failure
A trading firm in 2012 lost more than four hundred million dollars in under an hour because of a misused switch of this kind. An old and unused code path had been left in place behind a flag, new functionality was given the same flag, and one of eight servers was not updated during deployment. That server read the flag, ran the old code, and began issuing erroneous orders at enormous volume. The episode is cited constantly as an argument for removing dead code and old switches rather than leaving them in place as harmless.
The takeaway
A condition checking an externally held value lets new behaviour ship disabled and be turned on later without a deployment, which separates release from deployment and allows continuous merging, staged rollouts and instant disabling. Each switch doubles the possible states and almost none are tested. A 2012 trading disaster followed from reusing a flag attached to old code.