Can You Work Backwards From the Answer? Recovering a Cause From Its Effects
By the BrainSnail editorial team. How these articles are written and checked, and how to tell us when one is wrong.
Predicting what a known cause will produce is usually straightforward, and working out the cause from the observed result frequently is not. That asymmetry underlies medical scanning, seismology and a great deal else.
The two directions
A forward problem takes a known system and known inputs and computes the output, which is what most of applied mathematics does and which is generally well behaved, since a small change in the input produces a small change in the result. An inverse problem takes the observed output and asks what input or what system produced it, and that direction frequently misbehaves in specific ways. A solution may not exist, because the observations are inconsistent with any possible cause given measurement error. It may not be unique, because several different causes produce identical observations. And it may not be stable, meaning that a tiny change in the measurements produces an enormous change in the inferred cause, which is fatal in practice because all measurements contain noise.
Where these problems arise
The pattern recurs across fields that look unrelated:
- •Medical imaging, reconstructing an interior from measurements taken outside the body
- •Seismology, inferring the structure of the earth from vibrations recorded at the surface
- •Astronomy, recovering an image from an instrument that blurs it
- •Machine learning, inferring the rule that generated observed data
- •Geophysical prospecting, inferring what lies underground from measurements above it
- •Reconstructing past climate from proxies that record it indirectly
How instability is handled
The standard remedy is to add information that the data alone does not supply, which is called regularisation and which amounts to expressing a preference among the many solutions consistent with the observations. A preference for smoothness rules out the wildly oscillating solutions that noise generates. A preference for solutions that are mostly zero recovers sparse structures and underlies a substantial body of modern signal processing. Physical constraints including non-negativity, since a density cannot be less than zero, eliminate whole families of mathematical solutions. The cost is that the answer now depends partly on the preference imposed, so the result reflects an assumption as well as the data, and being explicit about which is which is the discipline the field requires and does not always get.
Knowing what cannot be known
A useful part of the discipline is establishing in advance which features of the answer the data can and cannot determine, which prevents a great deal of wasted argument. Resolution analysis asks how fine a structure could be detected given the measurements available, and answers of the form that anything smaller than a certain size is invisible are common and honest. Null space analysis identifies the changes to the solution that produce no change at all in the predicted observations, which are precisely the features the data says nothing about, and any confidence expressed about them comes entirely from the assumptions added. Seismic imaging of the earth's interior routinely reports what is resolved and what is not for exactly this reason, and the equivalent caution is frequently absent where the same mathematics is used elsewhere.
The scanner example
Medical computed tomography is the clearest worked case. The measurement is how much a beam is absorbed along each of many lines through the body, taken from many angles, and the question is what distribution of tissue densities inside produced those absorptions. The mathematics that inverts this was worked out in 1917 for entirely abstract reasons, decades before anyone could apply it, which is a standard illustration of pure mathematics finding an application nobody anticipated. The practical problem is that more measurements mean more radiation dose, so the reconstruction must work from as few as possible, which is precisely where instability bites. Modern reconstruction uses statistical methods incorporating what is known about noise and about anatomy, which allows good images at doses that earlier methods could not have used.
The takeaway
Computing an effect from a cause is usually stable, and inferring the cause from the effect frequently is not, since solutions may not exist, may not be unique and may swing wildly with small measurement errors. The remedy adds a preference, for smoothness or sparsity or physical plausibility, which means the answer reflects an assumption alongside the data and the two should be stated separately.