What Is Bayes Theorem? Updating a Belief When New Evidence Arrives

The theorem answers one question: given a belief about how likely something is, and a new piece of evidence, how likely is it now? Its structure has three parts. The prior is what you believed before, expressed as a probability. The likelihood is how probable the evidence would be if the hypothesis were true, compared with how probable it would be otherwise. The posterior is the updated belief, and it is proportional to the prior multiplied by the likelihood. The crucial insight, and the one that defeats intuition, is that the evidence never speaks alone: a strong result applied to an unlikely hypothesis still leaves the hypothesis unlikely, because the prior is doing as much work as the test. Most errors of reasoning about evidence come from dropping the prior, a mistake with the formal name of base rate neglect.

Numbers make it concrete. Suppose a disease affects one person in ten thousand, and a test detects it in 99 percent of people who have it while producing a false positive in 1 percent of people who do not. Imagine a million people tested:

• •100 have the disease, of whom 99 test positive
• •999,900 do not, of whom 1 percent, that is 9,999, test positive anyway
• •So 10,098 people test positive in total, and only 99 of them are ill
• •The chance that a positive result means disease is therefore 99 divided by 10,098, or roughly 1 percent
• •The result is counterintuitive only because the false positives, drawn from an enormous healthy population, vastly outnumber the true positives drawn from a tiny sick one
• •Raising the prior changes everything: if the same test is given to people already showing symptoms, so that one in twenty has the disease, a positive result now means about an 84 percent chance of illness

The same structure appears wherever evidence is weighed. In criminal justice it produces what is called the prosecutor's fallacy, in which the probability of the evidence given innocence is confused with the probability of innocence given the evidence, and it has caused real miscarriages of justice. Sally Clark was convicted in England in 1999 of murdering two of her children after an expert testified that the chance of two cot deaths in one family was one in 73 million, a figure that was wrong in its own terms and, more fundamentally, meaningless without comparison to the equally small probability of a mother murdering two children; the conviction was quashed in 2003 and the Royal Statistical Society issued a public statement about the misuse of statistics. The same reasoning applies to DNA match probabilities, airport screening, spam filters, fraud detection and any alarm system: when the thing being searched for is rare, most alarms are false, no matter how good the detector.

The theorem is uncontroversial as arithmetic and contentious as a philosophy, because it requires a prior, and where the prior comes from is the argument. Frequentist statistics, which dominated the twentieth century, treats probability as the long-run frequency of an event and refuses to assign probabilities to hypotheses, which is why it produces p-values and confidence intervals rather than a probability that a theory is true. Bayesian statistics treats probability as a degree of belief, which permits exactly that, at the cost of requiring the analyst to state what they believed beforehand. The objection is that this injects subjectivity; the reply is that the subjectivity was always there and is now explicit, and that with enough data different reasonable priors converge on the same answer. The argument was bitter for decades, with Ronald Fisher among the fiercest opponents, and it has cooled into a practical division in which most statisticians use whichever framework suits the problem.

The approach was revived by computation. Bayesian methods require integrals that are usually impossible to evaluate by hand, and they became practical only when computers could approximate them by simulation, chiefly through Markov chain Monte Carlo methods developed from the 1950s and widely adopted from the 1990s. The applications are now everywhere: spam filters that update the probability an email is junk word by word, the Bayesian search theory that located a hydrogen bomb lost off Spain in 1966 and the wreckage of Air France 447 in 2011 after earlier searches failed, medical diagnosis systems, genetics, the analysis of clinical trials that update as data arrives, and the whole family of probabilistic models in machine learning. Alan Turing used essentially Bayesian reasoning at Bletchley Park to rank the likelihood of Enigma settings, work that stayed classified for decades. A formula published posthumously in 1763 by a Presbyterian minister who apparently did not think it worth printing turned out to describe how evidence should change a mind.

Bayes theorem updates a prior belief in the light of new evidence, multiplying the prior by how likely that evidence would be if the hypothesis were true. Its most important practical consequence is that a positive result from an accurate test for a rare condition is usually a false positive, because the healthy population is so much larger, and ignoring that base rate causes errors in medicine and in court. It underpins the Bayesian school of statistics, which treats probability as degree of belief, and became practical with computer simulation.