How Scientists Measure Uncertainty

The smallest division on a ruler, thermometer or measuring cylinder limits how finely you can read it. A digital display also rounds values to its available digits. Writing extra decimal places does not create extra information, even though the result may look impressively tidy.

Scientists often record an uncertainty based on the instrument's resolution and the way it is read. For an analogue scale, estimating between marks may be possible. For a digital device, the final displayed digit usually sets a practical limit, although the manufacturer's specification may reveal additional uncertainty.

The measuring process matters too. A ruler may be precise enough, but a fuzzy edge makes the object's endpoint hard to identify. Timing a short event by hand introduces reaction time. Good uncertainty estimates consider the whole method, not only the label printed on the instrument.

Random variation causes repeated readings to scatter. Small changes in conditions, observer judgement or natural behaviour can make one reading slightly higher and another slightly lower. Repeating measurements and calculating a mean can reduce the influence of random variation.

A systematic error shifts results in the same direction. A balance that reads 2 grams too high will bias every measurement unless it is corrected. Repeating the reading does not remove this problem because the same offset remains each time.

Calibration helps identify systematic effects by comparing an instrument with a trusted reference. Scientists also use control measurements, blank samples and alternative methods. Agreement between different methods can strengthen confidence, while disagreement may reveal a hidden assumption or fault.

Results are often reported as a best estimate together with a range or uncertainty. If two measurements have strongly overlapping ranges, a small difference between their central values may not be meaningful. If the separation is large compared with the uncertainties, the evidence for a real difference is stronger. Percentage uncertainty can also help when you compare measurements of very different sizes, because the same absolute uncertainty matters more for a small quantity than for a large one.

Useful habits include:

• •Choose an instrument suited to the scale of the task.
• •Record all justified digits and no invented ones.
• •Repeat measurements when random variation is expected.
• •Check calibration and possible systematic bias.
• •Compare differences with the size of the uncertainty.

Imagine timing one swing of a pendulum with a stopwatch. A single swing might last under two seconds, while your reaction time when starting and stopping the watch could add an uncertainty of a couple of tenths of a second. That is a large fraction of the measurement. A simple improvement is to time twenty swings in one go. The reaction time error stays roughly the same, but it is now shared across twenty swings, so the uncertainty for each swing becomes twenty times smaller.

Uncertainties also travel into calculations. If you use a length and a time to calculate another quantity, the uncertainty in each measurement affects the answer. A useful rule for multiplying or dividing values is that their percentage uncertainties roughly add together. This tells you where to focus effort. If the time measurement has a five percent uncertainty and the length only half a percent, buying a more precise ruler will hardly help. Improving the timing method will make a far bigger difference.

Accuracy and precision are often used as if they meant the same thing, but scientists separate them. Precision describes how closely repeated measurements agree with each other. Accuracy describes how close they are to the true value. Picture darts: a tight cluster far from the bullseye is precise but inaccurate, the sign of a systematic error, while darts scattered evenly around the centre are accurate on average but imprecise. A good method aims for both, and no amount of repetition will fix the first problem.

A famous example came in 2011, when the OPERA experiment in Italy reported that neutrinos sent from CERN appeared to arrive slightly sooner than light would. The measurement was very precise, and the team asked other physicists to help find a mistake rather than announcing a revolution. Months later, the cause was traced largely to a fibre-optic cable that was not properly connected, which shifted the timing. Repeated measurements had agreed beautifully, but every one carried the same hidden offset, a textbook systematic error.

Repeating a measurement reduces random scatter, but more slowly than you might hope. The uncertainty in an average shrinks with the square root of the number of readings. Suppose single readings of a length scatter by about 2 millimetres. Averaging 4 readings cuts the uncertainty in the mean to roughly 1 millimetre, because the square root of 4 is 2. To reach 0.2 millimetres you would need 100 readings, since the square root of 100 is 10. Each further improvement costs far more effort than the last, which is why scientists often switch to a better method rather than simply repeating a poor one many more times.

Particle physicists turn this thinking into a strict standard for discoveries. They describe how surprising a result is in units of standard deviation, written with the Greek letter sigma, which measures how far a result sits from what random chance would usually produce. Their usual threshold for claiming a discovery is five sigma, meaning that if there were no real effect, a fluctuation at least that large would occur by chance only about once in several million tries. When CERN announced the discovery of the Higgs boson in 2012, both of its large experiments had independently reached roughly this level, which made the claim convincing.

Opinion polls show measurement uncertainty in everyday life. A poll asks a sample of people rather than everyone, so its result will differ by chance from the true figure for the whole population. For a well-chosen random sample of about 1000 people, the usual margin of error is around plus or minus 3 percentage points, at a 95 per cent level of confidence. If a poll gives one party 34 per cent and another 31 per cent, the gap of 3 points is within the range that chance alone could produce, so the poll does not show clearly who is ahead.

Quadrupling the sample to 4000 only halves the margin, to about 1.5 points, because of the square root rule. More importantly, the margin of error covers only random sampling variation. It says nothing about systematic problems, such as some groups being harder to reach or less willing to answer, questions worded in a leading way, or people changing their minds before the vote. Pollsters adjust their samples to correct known biases, but hidden offsets can still push every poll in the same direction at once. When many polls agree with each other yet all miss the result, a systematic error is the usual suspect.

Uncertainty describes the limits of a measurement rather than making the result useless. Instrument resolution, random variation and systematic bias all affect confidence. State those limits clearly, and you can compare evidence more honestly, improve the method and avoid treating a neat-looking number as more certain than it really is.