The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses
Four pages on PainSci cite Ioannidis 2016: 1. A Deep Dive into Delayed-Onset Muscle Soreness 2. Science versus Experience in Musculoskeletal Medicine 3. Ozone Therapy for Pain 4. New “positive” studies rarely change the bottom line (e.g. massage for exercise soreness)
original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.
POLICY POINTS: Currently, there is massive production of unnecessary, misleading, and conflicted systematic reviews and meta-analyses. Instead of promoting evidence-based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools. Suboptimal systematic reviews and meta-analyses can be harmful given the major prestige and influence these types of studies have acquired. The publication of systematic reviews and meta-analyses should be realigned to remove biases and vested interests and to integrate them better with the primary production of evidence.
CONTEXT: Currently, most systematic reviews and meta-analyses are done retrospectively with fragmented published information. This article aims to explore the growth of published systematic reviews and meta-analyses and to estimate how often they are redundant, misleading, or serving conflicted interests.
METHODS: Data included information from PubMed surveys and from empirical evaluations of meta-analyses.
FINDINGS: Publication of systematic reviews and meta-analyses has increased rapidly. In the period January 1, 1986, to December 4, 2015, PubMed tags 266,782 items as "systematic reviews" and 58,611 as "meta-analyses." Annual publications between 1991 and 2014 increased 2,728% for systematic reviews and 2,635% for meta-analyses versus only 153% for all PubMed-indexed items. Currently, probably more systematic reviews of trials than new randomized trials are published annually. Most topics addressed by meta-analyses of randomized trials have overlapping, redundant meta-analyses; same-topic meta-analyses may exceed 20 sometimes. Some fields produce massive numbers of meta-analyses; for example, 185 meta-analyses of antidepressants for depression were published between 2007 and 2014. These meta-analyses are often produced either by industry employees or by authors with industry ties and results are aligned with sponsor interests. China has rapidly become the most prolific producer of English-language, PubMed-indexed meta-analyses. The most massive presence of Chinese meta-analyses is on genetic associations (63% of global production in 2014), where almost all results are misleading since they combine fragmented information from mostly abandoned era of candidate genes. Furthermore, many contracting companies working on evidence synthesis receive industry contracts to produce meta-analyses, many of which probably remain unpublished. Many other meta-analyses have serious flaws. Of the remaining, most have weak or insufficient evidence to inform decision making. Few systematic reviews and meta-analyses are both non-misleading and useful.
CONCLUSIONS: The production of systematic reviews and meta-analyses has reached epidemic proportions. Possibly, the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, and/or conflicted.
related content
- “Do certain countries produce only positive results? A systematic review of controlled trials,” Vickers et al, Control Clin Trials, 1998.
Specifically regarding Ioannidis 2016:
This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights:
- Restoring trust in menopause management: menopause hormone therapy is not a panacea, and physical activity remains a critical intervention. Tulloh 2025 Br J Sports Med.
- Common interventional procedures for chronic non-cancer spine pain: a systematic review and network meta-analysis of randomised trials. Wang 2025 BMJ.
- Gabapentinoids and Risk of Hip Fracture. Leung 2024 JAMA Netw Open.
- Classical Conditioning Fails to Elicit Allodynia in an Experimental Study with Healthy Humans. Madden 2017 Pain Med.
- Topical glyceryl trinitrate (GTN) and eccentric exercises in the treatment of mid-portion achilles tendinopathy (the NEAT trial): a randomised double-blind placebo-controlled trial. Kirwan 2024 Br J Sports Med.