Stark statistical errors in much massage research

I recently wrote that massage therapy is often perceived as a “cute” profession because most massage research is so limited and often amateurish by the standards of medical scientists. Some readers challenged me to substantiate that a little. Fair enough, and it’s easy to do.

An excellent example is the “comparing the wrong things” problem, or more specifically the error of reporting the statistical significance of the wrong comparison (technically, failing to do an analysis of variance, or ANOVA). This was described by Ben Goldacre for The Guardian as

a stark statistical error so widespread it appears in about half of all the published papers surveyed from the academic neuroscience research literature.

Dr. Steven Novella also wrote about it for ScienceBasedMedicine.org, adding that

there is no reason to believe that it is unique to neuroscience research or more common in neuroscience than in other areas of research.

And it indeed it is not. Massage research is completely rotten with it. “Rampant” even. Dr. Christopher A. Moyer is a psychologist and a rare example of a real scientist — someone trained and expert in research methodology — who has chosen to focus on massage therapy:

I have been talking about this error for years, and have even published a paper on it. I critiqued a single example of it, and then discussed how the problem was rampant in massage therapy research. Based on the Nieuwenhuis paper, apparently it’s rampant elsewhere as well, and that is really unfortunate. Knowing the difference between a within-group result and a between-groups result is basic stuff.

So this is basic stuff, and yet simply unknown to many well-intentioned researchers, and a serious problem with many of the massage studies you have ever heard of.

There are countless other examples of problems with massage research, but this issue is particularly ubiquitous and pernicious. I tackle a full explanation of significantly irrelevant significant differences in Statistical Significance Abuse. I also put it in the massage context in Does Massage Therapy Work?