A dirty dozen: twelve p-value misconceptions
PainSci summary of Goodman 2008?This page is one of thousands in the PainScience.com bibliography. It is not a general article: it is focussed on a single scientific paper, and it may provide only just enough context for the summary to make sense. Links to other papers and more general information are provided at the bottom of the page, as often as possible. ★★★★☆?4-star ratings are for bigger/better studies and reviews published in more prestigious journals, with only quibbles. Ratings are a highly subjective opinion, and subject to revision at any time. If you think this paper has been incorrectly rated, please let me know.
Goodman’s Dirty Dozen P-value bloopers. This is the most authoritative-yet-manageable paper I’ve found about statistical significance. That doesn’t mean it’s actually easy to read, mind — just easier than most other papers about this.
The P value is a measure of statistical evidence that appears in virtually all medical research papers. Its interpretation is made extraordinarily difficult because it is not part of any formal system of statistical inference. As a result, the P value's inferential meaning is widely and often wildly misconstrued, a fact that has been pointed out in innumerable papers and books appearing since at least the 1940s. This commentary reviews a dozen of these common misinterpretations and explains why each is wrong. It also reviews the possible consequences of these improper understandings or representations of its meaning. Finally, it contrasts the P value with its Bayesian counterpart, the Bayes' factor, which has virtually all of the desirable properties of an evidential measure that the P value lacks, most notably interpretability. The most serious consequence of this array of P-value misconceptions is the false belief that the probability of a conclusion being in error can be calculated from the data in a single experiment without reference to external evidence or the plausibility of the underlying mechanism.
One article on PainScience.com cites Goodman 2008 as a source:
- PS Statistical Significance Abuse — A lot of research makes scientific evidence seem more “significant” than it is