Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and meta-analysis
Two pages on PainSci cite Gartlehner 2022: 1. Does Arnica Gel Work for Pain? 2. Homeopathy Schmomeopathy
PainSci notes on Gartlehner 2022:
This is some meta-science (science about science) confirming that homeopathy research kinda sucks. Researchers studying homeopathy don’t register their trials often enough, many registered trials are never published (probably because the results were negative), and even when they are registered and published they often change the primary outcomes (moving the goalposts to get a “score”). “This likely affects the validity of the body of evidence of homeopathic literature and may overestimate the true treatment effect of homeopathic remedies.”
original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.
OBJECTIVES: To assess the magnitude of reporting bias in trials assessing homeopathic treatments and its impact on evidence syntheses.
DESIGN: A cross-sectional study and meta-analysis. Two persons independently searched Clinicaltrials.gov, the EU Clinical Trials Register and the International Clinical Trials Registry Platform up to April 2019 to identify registered homeopathy trials. To determine whether registered trials were published and to detect published but unregistered trials, two persons independently searched PubMed, Allied and Complementary Medicine Database, Embase and Google Scholar up to April 2021. For meta-analyses, we used random effects models to determine the impact of unregistered studies on meta-analytic results.
MAIN OUTCOMES AND MEASURES: We report the proportion of registered but unpublished trials and the proportion of published but unregistered trials. We also assessed whether primary outcomes were consistent between registration and publication. For meta-analyses, we used standardised mean differences (SMDs).
RESULTS: Since 2002, almost 38% of registered homeopathy trials have remained unpublished, and 53% of published randomised controlled trials (RCTs) have not been registered. Retrospective registration was more common than prospective registration. Furthermore, 25% of primary outcomes were altered or changed compared with the registry. Although we could detect a statistically significant trend toward an increase of registrations of homeopathy trials (p=0.001), almost 30% of RCTs published during the past 5 years had not been registered.A meta-analysis stratified by registration status of RCTs revealed substantially larger treatment effects of unregistered RCTs (SMD: -0.53, 95%CI -0.87 to -0.20) than registered RCTs (SMD: -0.14, 95%CI -0.35 to 0.07).
CONCLUSIONS: Registration of published trials was infrequent, many registered trials were not published and primary outcomes were often altered or changed. This likely affects the validity of the body of evidence of homeopathic literature and may overestimate the true treatment effect of homeopathic remedies.
related content
Specifically regarding Gartlehner 2022:
This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights:
- Classical Conditioning Fails to Elicit Allodynia in an Experimental Study with Healthy Humans. Madden 2017 Pain Med.
- Topical glyceryl trinitrate (GTN) and eccentric exercises in the treatment of mid-portion achilles tendinopathy (the NEAT trial): a randomised double-blind placebo-controlled trial. Kirwan 2024 Br J Sports Med.
- Placebo analgesia in physical and psychological interventions: Systematic review and meta-analysis of three-armed trials. Hohenschurz-Schmidt 2024 Eur J Pain.
- Recovery trajectories in common musculoskeletal complaints by diagnosis contra prognostic phenotypes. Aasdahl 2021 BMC Musculoskelet Disord.
- Cannabidiol (CBD) products for pain: ineffective, expensive, and with potential harms. Moore 2023 J Pain.