The faulty statistics of complementary alternative medicine (CAM)
Three articles on PainSci cite Pandolfi 2014: 1. The “Impress Me” Test 2. Statistical Significance Abuse 3. Science versus Experience in Musculoskeletal Medicine
original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.
The authors illustrate the difficulties involved in obtaining a valid statistical significance in clinical studies especially when the prior probability of the hypothesis under scrutiny is low. Since the prior probability of a research hypothesis is directly related to its scientific plausibility, the commonly used frequentist statistics, which does not take into account this probability, is particularly unsuitable for studies exploring matters in various degree disconnected from science such as complementary alternative medicine (CAM) interventions. Any statistical significance obtained in this field should be considered with great caution and may be better applied to more plausible hypotheses (like placebo effect) than that examined - which usually is the specific efficacy of the intervention. Since achieving meaningful statistical significance is an essential step in the validation of medical interventions, CAM practices, producing only outcomes inherently resistant to statistical validation, appear not to belong to modern evidence-based medicine.
related content
- “Clinical trials of integrative medicine: testing whether magic works?,” David H. Gorski and Steven P. Novella, Trends in Molecular Medicine, 2014.
- “Recommendations are made in the absence of any good treatments,” David Colquhoun, British Medical Journal, 2017.
- “Scientific method: statistical errors,” Regina Nuzzo, Nature, 2014.
- “A dirty dozen: twelve p-value misconceptions,” Steven Goodman, Semin Hematol, 2008.
This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights:
- Is Neck Posture Subgroup in Late Adolescence a Risk Factor for Persistent Neck Pain in Young Adults? A Prospective Study. Richards 2021 Phys Ther.
- Photobiomodulation therapy is not better than placebo in patients with chronic nonspecific low back pain: a randomised placebo-controlled trial. Guimarães 2021 Pain.
- No effect of creatine monohydrate supplementation on inflammatory and cartilage degradation biomarkers in individuals with knee osteoarthritis. Cornish 2018 Nutr Res.
- The CANBACK trial: a randomised, controlled clinical trial of oral cannabidiol for people presenting to the emergency department with acute low back pain. Bebee 2021 Med J Aust.
- Relationships Between Sleep Quality and Pain-Related Factors for People with Chronic Low Back Pain: Tests of Reciprocal and Time of Day Effects. Gerhart 2017 Ann Behav Med.