Detailed guides to painful problems, treatments & more

How much evidence is enough?

 •  • by Paul Ingraham
Get posts in your inbox:
Weekly nuggets of pain science news and insight, usually 100-300 words, with the occasional longer post. The blog is the “director’s commentary” on the core content of a library of major articles and books about common painful problems and popular treatments. See the blog archives or updates for the whole site.

These days most well-trained health care professionals take it for granted that treatment ideas need to be blessed by science to some degree. But to what degree? How blessed exactly?

Back in the good old days there wasn’t evidence of anything one way or another (absence of evidence) and everyone pretty much did whatever they liked as long as it sounded good and the patients were happy. If you could get people to pay for it, that was good enough! Market-based medicine. Experience-based medicine. What could possibly go wrong? Entire modality empires sprang up out of the fertilizer of hunches and pet theories, many of them reasonable but definitely wrong, and many more “not even wrong.”

As standards have gone up and science has (finally!) started to test some of the 20th centuries biggest treatment ideas, the results have shown that in most cases nothing is going on except a creatively induced placebo (evidence of absence of any medical effect)… and placebo isn’t all that powerful and probably shouldn’t be the reason for therapy. In fact, science has become quite the buzzkill, especially for the treatment of pain, and manual therapists of all kinds — physical therapists, chiropractors, massage therapists — have started to wonder if anything actually works, why they read this damn website anyway, and how they can justify what they are selling without more encouraging trials to point to.

Evidence isn’t everything, but clinical experience and patient buy-in are huge

Despite the rise and importance of Evidence-Based Medicine™, evidence produced by good quality trials isn’t everything. It is not and never has been be the sole criterion for choosing health care interventions. There’s much more to it, and there always has been. Specifically, EBM has always formally, explicitly defined itself as the integration of clinical experience and patient values and preferences and expectations with the best available clinical evidence.

As Dr. of Physical Therapy Jason Silvernail argues, “The manual therapy approach is a ‘process’ of care centred on a reasoning model, not a ‘product’ consisting of one or more manipulative techniques,” and that process may be effective even if individual techniques are unimpressive. Good manual therapy is probably more than the sum of its parts.

Patients cannot meaningfully apply their values and preferences until they are informed, but once they are “informed consent” goes a long way. Professionals can legitimately do a lot sketchy stuff if only they speak the magic words: “This is experimental. It may not work. I think it’s worth trying because yada yada yada and the risks are super low. Do you want to proceed?”

Patients really appreciate that approach in my experience.

Absence of evidence is actually not a deal breaker, and it is still very common, even today. For all the progress we’ve made, pain and musculoskeletal medicine research has still only just scratched the surface.

All of this puts evidence in its place … but that is still a place of honour. Testing treatments matters.

That said, just exactly how much scientific evidence is actually needed for a theory or technique to be acceptable?

This is the bare minimum required:

  • Biological plausibility. It has to make sense. If the idea is daft — if it’s at odds with any well-established biology, chemistry, physics — that’s a deal-breaker. Goodbye, therapeutic touch and Reiki.
  • There can’t be evidence-of-absence. If there’s persuasive trial evidence that shows no benefit, or damns a technique with very faint praise (which is extremely common), that’s another deal-breaker. Goodbye, glucosamine.

But the bar gets raised quickly in proportion to the costs and risks, or if there’s no informed consent. Clearly positive good quality and replicated trial evidence becomes necessary then. And support from bad science only is not enough, which actually disqualifies many treatments (homeopathy, for instance).

Surprise! My standards are low! Sort of

I have a reputation for being critical of many (or most?) theories and techniques, so many readers may be surprised by just how low my standards for evidence seem. But I really do think that many unproven theories and techniques are fair game — if they’re fairly safe, cheap, plausible, and haven’t been spanked by good trials yet.

Here’s the “but” though …

The big problem — and the reason I am such a militant skeptic — is that informed consent is usually broken. Most patients are never informed properly, and otherwise justifiable treatments are not presented to patients as experimental. And way too many therapists are overconfident and grossly overestimate value of their clinical experience. And so patients are routinely presented with cocky self-serving claims of efficacy by therapists. That is what keeps me cranky.

For example, I think trigger point therapy, despite its many problems, is still a defensible approach to some kinds of pain as long as the risks and costs are tamed and it’s presented with strong, humble disclaimers. It’s just fine if a therapist puts it to patients like this:

“I do trigger point therapy, even though no one really knows what trigger points are. We have some theories. The science so far is not very encouraging, and there’s a bunch of controversy. Although there are still reasons for optimism, basically no one can really know yet if we can do anything about them. It’s a gamble, and not cheap. But we’ll be gentle and efficient and I won’t recommend a long expensive course of treatment without promising signs. Do you want to proceed?”

But I have a huge problem with this kind of thing (which is, of course, rarely spelled out):

“Trigger point therapy works! My results speak for themselves. I understand this kind of pain and I can treat it. Now enjoy my magic hands [or needles]… which are going to hurt both your body and your wallet, by the way.”

In the absence of good decisive science — which is all too often — it’s really all about the framing and the humility and the doing-no-harm.