How Does New Information Change the Odds? Probability Given Something Known
By the BrainSnail editorial team. How these articles are written and checked, and how to tell us when one is wrong.
The chance of something changes once you know something else, and the rule for updating it is short and reliably misapplied. The mistakes it produces have decided court cases and medical decisions.
What the idea is
Conditional probability is the chance of one event given that another is known to have occurred, and it is calculated by restricting attention to the cases where the known event happened and asking what fraction of those also have the event of interest. Restricting the field is the whole of the idea, and it is why the answer can differ enormously from the unconditional chance. Two events are independent when conditioning on one makes no difference to the other, which is a strong assumption that is frequently made without being checked and is frequently false. The order matters as well, since the chance of the first given the second and the chance of the second given the first are different quantities that happen to be related by a short formula, and confusing them is the single commonest error in the area.
Where it goes wrong
The recurring errors have names and each has done real damage:
- •Confusing the two directions, so that the chance of the evidence given innocence is treated as the chance of innocence given the evidence
- •Ignoring how common the underlying condition is, which makes a rare condition's test results misleading
- •Assuming independence between events that are actually related, which multiplies small numbers into absurdly small ones
- •Conditioning on a selected sample without noticing the selection
- •Treating a conditional statement as a statement about causation
- •Failing to update at all when new information arrives
The medical test example
The clearest demonstration involves a test for a condition affecting one person in a thousand. Suppose the test correctly identifies ninety nine per cent of those who have it and wrongly flags one per cent of those who do not. A person testing positive naturally assumes they almost certainly have it. Restricting to a population of a hundred thousand makes the arithmetic plain, since about a hundred people have the condition and nearly all test positive, while about ninety nine thousand nine hundred do not and about nine hundred and ninety nine of those test positive anyway. So roughly a thousand and ninety nine people test positive and only a hundred of them have the condition, meaning a positive result carries under a ten per cent chance of being right. Nothing is wrong with the test, and the rarity of the condition dominates everything.
Why intuition fails here
The errors are systematic rather than random, which suggests something about how people reason. Presenting the same problem in terms of natural frequencies, meaning counts of people rather than percentages, improves performance dramatically, which has been demonstrated repeatedly with both students and practising doctors and is the single most effective known intervention. That finding suggests the difficulty is with the representation rather than with the underlying logic, since the same people handle counts correctly and probabilities badly. A second factor is that the vivid specific evidence feels more relevant than the dull general rate, so the rarity of a condition gets discounted even when it is stated explicitly. Teaching the formula alone does not fix either problem, whereas teaching people to lay out the counts in a table does, which is why that approach now appears in medical training.
The courtroom version
The same confusion has produced serious miscarriages of justice. Presenting the chance that evidence would appear if the defendant were innocent, and then treating that number as the chance the defendant is innocent, reverses the conditioning and is known as the prosecutor's fallacy. A related error multiplies the probabilities of several unlikely events as though they were independent when they are not, which produced a notorious case in which the chance of two infant deaths in one family was calculated by squaring a single figure, ignoring that whatever caused the first death might make the second more likely, and produced a wrongful conviction later overturned. Statistical bodies have issued formal guidance on presenting such evidence in court, and the underlying problem is that the correct calculation requires a prior probability that the court is reluctant to state.
The takeaway
Conditioning restricts attention to the cases where the known event occurred and asks what fraction of those contain the event of interest. The chance of A given B and the chance of B given A are different quantities, and confusing them is the commonest error. A very accurate test for a rare condition still produces mostly false positives, because the rarity dominates the arithmetic.