How Do You Spot a Bad Claim? Signs That Are Suggestive and Not Proof

Philosophers have tried for a century to state a criterion separating science from what imitates it, and no proposal has survived. Requiring that a claim be capable of being proved wrong captures something important and fails as a strict rule, since a theory can be shielded by adjusting assumptions and since legitimate theories have been held despite apparent refutation and later vindicated. Requiring accumulated evidence fails because new fields lack it. Requiring institutional acceptance makes the definition social rather than epistemic and would have excluded several ideas now accepted. The practical position is that no single test works and that a set of signs, weighed together against how the field actually behaves, is the best available guide.

Certain features recur in claims that do not hold up:

• •Evidence consisting mainly of testimonials rather than controlled comparison
• •Explanations that accommodate any result, so no observation could count against them
• •Appeals to suppression, where absence of acceptance is offered as proof of a conspiracy
• •Claims that do not connect to anything else that is known
• •A vocabulary borrowed from physics applied without the mathematics
• •Reliance on an original authority rather than on accumulated work
• •Absence of any research programme, so the claim is unchanged over decades

Every one of those features appears in legitimate science sometimes, which is what makes judgement necessary. Accepted theories have been protected by auxiliary assumptions, and doing so was correct when the assumption turned out to be true, which is how an unexplained wobble in one planet's orbit led to discovering another rather than to abandoning the theory. Genuine findings have been resisted by establishments, and the researchers who identified the bacterial cause of stomach ulcers were dismissed for years. New fields lack connection to existing knowledge by definition. Reliance on an individual's work is normal early on. The signs are therefore probabilistic indicators rather than criteria, and treating them as a checklist produces confident errors in both directions.

Applying the signs badly produces its own failures and they are worth naming. Dismissing an unfamiliar claim because it sounds odd confuses novelty with error, and several well-established findings were absurd when proposed. Treating institutional consensus as the criterion converts the question into an appeal to authority and cannot account for how consensus changes. Using the signs to dismiss an entire field because some practitioners exhibit them is an overreach, since medicine, nutrition and psychology all contain both excellent and poor work. And the signs can be deployed selectively against conclusions somebody dislikes while being ignored for conclusions they favour, which is the commonest failure of all and which is invisible to the person doing it.

The more reliable questions concern behaviour over time rather than features of a single claim. Has the field produced anything new, meaning findings that were not known when it started, or is it defending the same claims with the same evidence decades later. Do its practitioners publish specific predictions in advance that could fail. Does it identify and correct its own errors, and are its practitioners willing to name claims from within the field that turned out to be wrong. Does it engage with the strongest objections or only with weak ones. Is the evidence available for others to examine and reanalyse. Those questions work because they ask what the enterprise does rather than what it asserts, and they distinguish cases the surface features do not.

No single criterion separates science from what imitates it, and every proposal has failed, which is why a weighed set of signs is the practical approach. Testimonial evidence, explanations that accommodate any result and appeals to suppression are suggestive. Each appears occasionally in legitimate work, so asking what a field has produced and corrected over time is more reliable than any checklist.