A dirty dozen: twelve p-value misconceptions
One article on PainSci cites Goodman 2008: Statistical Significance Abuse
PainSci notes on Goodman 2008:
Goodman’s Dirty Dozen P-value bloopers. This is the most authoritative-yet-manageable paper I’ve found about statistical significance. That doesn’t mean it’s actually easy to read, mind — just easier than most other papers about this.
original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.
The P value is a measure of statistical evidence that appears in virtually all medical research papers. Its interpretation is made extraordinarily difficult because it is not part of any formal system of statistical inference. As a result, the P value's inferential meaning is widely and often wildly misconstrued, a fact that has been pointed out in innumerable papers and books appearing since at least the 1940s. This commentary reviews a dozen of these common misinterpretations and explains why each is wrong. It also reviews the possible consequences of these improper understandings or representations of its meaning. Finally, it contrasts the P value with its Bayesian counterpart, the Bayes' factor, which has virtually all of the desirable properties of an evidential measure that the P value lacks, most notably interpretability. The most serious consequence of this array of P-value misconceptions is the false belief that the probability of a conclusion being in error can be calculated from the data in a single experiment without reference to external evidence or the plausibility of the underlying mechanism.
This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights:
- Association of Lumbar MRI Findings with Current and Future Back Pain in a Population-based Cohort Study. Kasch 2022 Spine (Phila Pa 1976).
- A double-blinded randomised controlled study of the value of sequential intravenous and oral magnesium therapy in patients with chronic low back pain with a neuropathic component. Yousef 2013 Anaesthesia.
- Is Neck Posture Subgroup in Late Adolescence a Risk Factor for Persistent Neck Pain in Young Adults? A Prospective Study. Richards 2021 Phys Ther.
- Sudden amnesia resulting in pain relief: the relationship between memory and pain. Choi 2007 Pain.
- Photobiomodulation therapy is not better than placebo in patients with chronic nonspecific low back pain: a randomised placebo-controlled trial. Guimarães 2021 Pain.