original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.
A recent article in this journal (Ioannidis JP (2005) Why most published research findings are false. PLoS Med 2: e124) argued that more than half of published research findings in the medical literature are false. In this commentary, we ex- amine the structure of that argument, and show that it has three basic components:
- An assumption that the prior probability of most hypotheses explored in medical research is below 50%.
- Dichotomization of P-values at the 0.05 level and introduction of a “bias” factor (produced by significance-seeking), the combination of which severely weakens the evidence provided by every design.
- Use of Bayes theorem to show that, in the face of weak evidence, hypothe- ses with low prior probabilities cannot have posterior probabilities over 50%.
Thus, the claim is based on a priori assumptions that most tested hypotheses are likely to be false, and then the inferential model used makes it impossible for evidence from any study to overcome this handicap. We focus largely on step (2), explaining how the combination of dichotomization and “bias” dilutes experimental evidence, and showing how this dilution leads inevitably to the stated conclusion. We also demonstrate a fallacy in another important component of the argument –that papers in “hot” fields are more likely to produce false findings.
We agree with the paper’s conclusions and recommendations that many medical research findings are less definitive than readers suspect, that P-values are widely misinterpreted, that bias of various forms is widespread, that multiple approaches are needed to prevent the literature from being systematically biased and the need for more data on the prevalence of false claims. But calculating the unreliability of the medical research literature, in whole or in part, requires more empirical evidence and different inferential models than were used. The claim that “most research findings are false for most research designs and for most fields” must be considered as yet unproven.
- Ioannidis: Making Medical Science Look Bad Since 2005 — A famous and excellent scientific paper … with an alarmingly misleading title
- “Why most published research findings are false: author's reply to Goodman and Greenland,” Ioannidis, PLoS Med, 2007.
- Mini-bio for "Dr. John Ioannidis" on PainScience.com.
- “Are Most Medical Studies Wrong?,” Steven Novella, Neurologica.
Goodman 2007 is about:
- “Why Most Published Research Findings Are False,” Ioannidis, PLoS Medicine, 2005.
This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights:
- Cannabidiol (CBD) products for pain: ineffective, expensive, and with potential harms. Moore 2023 J Pain.
- Inciting events associated with lumbar disc herniation. Suri 2010 Spine J.
- Prediction of an extruded fragment in lumbar disc patients from clinical presentations. Pople 1994 Spine (Phila Pa 1976).
- Characteristics of patients with low back and leg pain seeking treatment in primary care: baseline results from the ATLAS cohort study. Konstantinou 2015 BMC Musculoskelet Disord.
- Effectiveness and cost-effectiveness of universal school-based mindfulness training compared with normal school provision in reducing risk of mental health problems and promoting well-being in adolescence: the MYRIAD cluster randomised controlled trial. Kuyken 2022 Evid Based Ment Health.