Detailed guides to painful problems, treatments & more

Stark statistical errors in much massage research

 •  • by Paul Ingraham
Get posts in your inbox:
Weekly nuggets of pain science news and insight, usually 100-300 words, with the occasional longer post. The blog is the “director’s commentary” on the core content of a library of major articles and books about common painful problems and popular treatments. See the blog archives or updates for the whole site.

I recently wrote that massage therapy is often perceived as a “cute” profession because most massage research is so limited and often amateurish by the standards of medical scientists. Some readers challenged me to substantiate that a little. Fair enough, and it’s easy to do.

An excellent example is the “comparing the wrong things” problem, or more specifically the error of reporting the statistical significance of the wrong comparison (technically, failing to do an analysis of variance, or ANOVA). This was described by Ben Goldacre for The Guardian as

a stark statistical error so widespread it appears in about half of all the published papers surveyed from the academic neuroscience research literature.

Dr. Steven Novella also wrote about it for, adding that

there is no reason to believe that it is unique to neuroscience research or more common in neuroscience than in other areas of research.

And it indeed it is not. Massage research is completely rotten with it. “Rampant” even. Dr. Christopher A. Moyer is a psychologist and a rare example of a real scientist — someone trained and expert in research methodology — who has chosen to focus on massage therapy:

I have been talking about this error for years, and have even published a paper on it. I critiqued a single example of it, and then discussed how the problem was rampant in massage therapy research. Based on the Nieuwenhuis paper, apparently it’s rampant elsewhere as well, and that is really unfortunate. Knowing the difference between a within-group result and a between-groups result is basic stuff.

So this is basic stuff, and yet simply unknown to many well-intentioned researchers, and a serious problem with many of the massage studies you have ever heard of.

There are countless other examples of problems with massage research, but this issue is particularly ubiquitous and pernicious. I tackle a full explanation of significantly irrelevant significant differences in Statistical Significance Abuse. I also put it in the massage context in Does Massage Therapy Work?

PainSci Member Login » Submit your email to unlock member content. If you can’t remember/access your registration email, please contact me. ~ Paul Ingraham, PainSci Publisher