Why Most Published Research Findings Are False
PainSci summary of Ioannidis 2005?This page is one of thousands in the PainScience.com bibliography. It is not a general article: it is focused on a single scientific paper, and it may provide only just enough context for the summary to make sense. Links to other papers and more general information are provided at the bottom of the page, as often as possible. ★★★★★?5-star ratings are for sentinel studies, excellent experiments with meaningful results. Ratings are a highly subjective opinion, and subject to revision at any time. If you think this paper has been incorrectly rated, please let me know.
This intensely intellectual paper — it’s completely, hopelessly nerdy — became one of the most downloaded articles in the history of the Public Library of Science and was described by the Boston Globe as an instant cult classic. Despite the title, the paper does not, in fact, say that “science is wrong,” but something much less sinister: that it should take rather a lot of good quality and convergent scientific evidence before we can be reasonably sure of something, and he presents good evidence that a lot of so-called conclusions are premature, not as “ready for prime time” as we would hope. This is not the least bit surprising to good scientists, who never claimed in the first place that their results are infallible or that their conclusions are “true.”
I go into much more detail here: Ioannidis: Making Medical Science Look Bad Since 2005.
original abstract†Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.
There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.
related content
- “Why most published research findings are false: author's reply to Goodman and Greenland,” John P A Ioannidis, PLoS Med, 2007.
Specifically regarding Ioannidis 2005:
- PS Ioannidis: Making Medical Science Look Bad Since 2005 — A famous and excellent scientific paper … with an alarmingly misleading title
- “Assessing the Unreliability of the Medical Literature: A Response to “Why Most Published Research Findings Are False”,” Steven Goodman and Sander Greenland, Johns Hopkins University, Dept. of Biostatistics Working Papers, 2007.
- Mini-bio for "Dr. John Ioannidis" on PainScience.com.
- “Are Most Medical Studies Wrong?,” Steven Novella, Neurologica.
- “Lies, Damned Lies, and Medical Science,” David Freedman, TheAtlantic.com.
These ten articles on PainScience.com cite Ioannidis 2005 as a source:
- PS Save Yourself from IT Band Syndrome! — All your treatment options for Iliotibial Band Syndrome reviewed in great detail, with clear explanations of recent scientific research supporting every key point
- PS Does Arnica Gel Work for Pain? — A detailed review of popular homeopathic (diluted) herbal creams and gels like Traumeel, used for muscle pain, joint pain, sports injuries, bruising, and post-surgical inflammation
- PS Does Hip Strengthening Work for IT Band Syndrome? — The popular “weak hips” theory is itself weak
- PS The “Impress Me” Test — Most controversial therapies are fighting over scraps of “positive” evidence that damn them with faint praise
- PS Statistical Significance Abuse — A lot of research makes scientific evidence seem more “significant” than it is
- PS Massage Does Not Reduce Inflammation — The making of a new massage myth from a high-tech study of muscle samples after intense exercise
- PS Does Fascia Matter? — A detailed critical analysis of the clinical relevance of fascia science and fascia properties
- PS Studying the Studies — Tips and musings about how to understand (and write about) pain and musculoskeletal health science
- PS Why So “Negative”? — Answering accusations of negativity, and my reasons and methods for debunking bad treatment options for pain and injury.
This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights:
- A Bayesian model-averaged meta-analysis of the power pose effect with informed and default priors: the case of felt power. Gronau 2017 Comprehensive Results in Social Psychology.
- The neck and headaches. Bogduk 2014 Neurol Clin.
- Agreement of self-reported items and clinically assessed nerve root involvement (or sciatica) in a primary care setting. Konstantinou 2012 Eur Spine J.
- Effect of NSAIDs on Recovery From Acute Skeletal Muscle Injury: A Systematic Review and Meta-analysis. Morelli 2017 Am J Sports Med.
- Association of Spinal Manipulative Therapy With Clinical Benefit and Harm for Acute Low Back Pain: Systematic Review and Meta-analysis. Paige 2017 JAMA.