Detailed guides to painful problems, treatments & more

A common error in alternative medicine

Amateurish speculation about the biological mechanisms of unproven treatments puts the scientific cart before the horse

Paul Ingraham • 2m read

“It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”

Sherlock Holmes (in Doyle’s 1892 A Scandal in Bohemia)

Here’s a very common error that I’ve heard countless times in online arguments and in angry emails:

“We need some science to prove how treatment modality X works!”

No! That is bass-ackwards. It’s a terrible inspiration for doing science, guaranteed to dial confirmation bias up to 11. But this idea is heard all too often in so-called “alternative” medicine. In fact, it’s one of its defining features: enthusiastic but amateurish speculation about the biological mechanisms of unproven treatments.

The premise that treatment X “works” is invariably based only on clinical experience and “educated” guessing, which proves nothing and cannot be trusted.

The fallibility of clinical experience: The pudding does not contain proof

“The proof is in the pudding.” I hear this almost daily from therapists who are convinced by their clinical experience that this or that treatment works. But the only thing in “pudding” is highly fallible clinical impressions, no better than any other form of anecdotal evidence, and possibly even worse. Working therapists in many ways are far more prone to biased misinterpretation and faulty perception than patients — they have ego, reputation, pride, and financial vested interests in being right.

But no: “proof” is only to be had from good clinical trials (and quite a few of them).

Efficacy trials first!

Treatment efficacy must be actually tested and demonstrated rigorously before trying to explain it. We need to find out if an idea actually produces measurable medical benefits and then — if it works, if it actually helps people — only then can we try to illuminate the mechanism.

We cannot study how a treatment works if we haven’t figured out if it works.

“Before we try to explain something, we should be sure it actually happened.”

Ray Hyman

Mechanism masturbation

In lieu of evidence that something actually works, it’s very popular to substitute speculation about how it might work.

“Mechanism masturbation” is wishful and fanciful thinking about why/how treatments might work. Science journalist Jonathan Jarry:

“There is a fascinating phenomenon in the complementary and alternative medicine literature we could call ‘mechanism masturbation’ where the authors, faced with the tiniest of positive signals in a small study, write paragraph after paragraph hypothesizing how, mechanistically, watermelon seeds might cure schizophrenia.”

Musculoskeletal and pain research, alternative as well as more mainstream research, is rotten with “mechanism masturbation” — often because it’s just all there is. There’s no good clinical trial data, so we get wishful thinking and wild speculation instead, even in scientific publications. The field is surprisingly afflicted with with cart-before-horse speculation about how they how they could work, might work, should work, maybe work… when the clinical trials (if they exist at all) tend to show that they don’t actually work, or not very well.

Jonathan’s satirical example really nails the flavour of “research” like this:

“It might interfere with the hypothalamic-pituitary-adrenal axis… One of its compounds does bind to alpha receptors in this cell type… Could play a role in this cascade... Anyway, preliminary results from n = 6. More studies needed!”

Yes, that definitely reads like about a thousand papers I’ve wasted my time reading over the last decade. I’m always looking for the rare scraps of basic science I can actually consider interesting/promising instead of more post-hoc rationalization for someone’s meal ticket. 🙄

About Paul Ingraham

Headshot of Paul Ingraham, short hair, neat beard, suit jacket.

I am a science writer in Vancouver, Canada. I was a Registered Massage Therapist for a decade and the assistant editor of ScienceBasedMedicine.org for several years. I’ve had many injuries as a runner and ultimate player, and I’ve been a chronic pain patient myself since 2015. Full bio. See you on Facebook or Twitter., or subscribe:

Permalinks

https://www.painscience.com/articles/facts-first.php

PainScience.com/facts_first

linking guide

500 words