Only poor quality studies of homeopathy have positive results
One page on PainSci cites Shang 2005: Does Arnica Gel Work for Pain?
PainSci commentary on Shang 2005: ?This page is one of thousands in the PainScience.com bibliography. It is not a general article: it is focused on a single scientific paper, and it may provide only just enough context for the summary to make sense. Links to other papers and more general information are provided wherever possible.
Homeopathy is a widely used treatment modality based on highly implausible, pre-scientific ideas. Some studies seem to support its use, but may be the result of researcher bias, which can skew the results of smaller and less rigorous studies in many ways, producing the answers that researchers are hoping for. The purpose of this review was to identify and review homeopathy studies with a lower risk of this. And how they did that was rather clever: they identified 110 randomized, controlled trials of homeopathy, and then compared them to 110 studies of medical treatments for the same conditions.
As expected, lower quality studies of both types tended to produced more “positive” results. In other words, there was probably bias on both sides of the alternative/mainstream fence. But when the analysis of results was restricted to only the higher quality results on each side, something important was revealed: there was only “weak evidence for a specific effect of homoeopathic remedies, but strong evidence for specific effects of conventional interventions.”
In other words, only crappy studies show that homeopathy works. Good quality studies show that it doesn’t … but modern medicine does.
original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.
BACKGROUND: Homeopathy is widely used, but specific effects of homoeopathic remedies seem implausible. Bias in the conduct and reporting of trials is a possible explanation for positive findings of trials of both homeopathy and conventional medicine. We analysed trials of homeopathy and conventional medicine and estimated treatment effects in trials least likely to be affected by bias.
METHODS: Placebo-controlled trials of homeopathy were identified by a comprehensive literature search, which covered 19 electronic databases, reference lists of relevant papers, and contacts with experts. Trials in conventional medicine matched to homeopathy trials for disorder and type of outcome were randomly selected from the Cochrane Controlled Trials Register (issue 1, 2003). Data were extracted in duplicate and outcomes coded so that odds ratios below 1 indicated benefit. Trials described as double-blind, with adequate randomisation, were assumed to be of higher methodological quality. Bias effects were examined in funnel plots and meta-regression models.
FINDINGS: 110 homeopathy trials and 110 matched conventional-medicine trials were analysed. The median study size was 65 participants (range ten to 1573). 21 homeopathy trials (19%) and nine (8%) conventional-medicine trials were of higher quality. In both groups, smaller trials and those of lower quality showed more beneficial treatment effects than larger and higher-quality trials. When the analysis was restricted to large trials of higher quality, the odds ratio was 0.88 (95% CI 0.65-1.19) for homeopathy (eight trials) and 0.58 (0.39-0.85) for conventional medicine (six trials).
INTERPRETATION: Biases are present in placebo-controlled trials of both homeopathy and conventional medicine. When account was taken for these biases in the analysis, there was weak evidence for a specific effect of homoeopathic remedies, but strong evidence for specific effects of conventional interventions. This finding is compatible with the notion that the clinical effects of homeopathy are placebo effects.
This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights:
- Topical glyceryl trinitrate (GTN) and eccentric exercises in the treatment of mid-portion achilles tendinopathy (the NEAT trial): a randomised double-blind placebo-controlled trial. Kirwan 2024 Br J Sports Med.
- Placebo analgesia in physical and psychological interventions: Systematic review and meta-analysis of three-armed trials. Hohenschurz-Schmidt 2024 Eur J Pain.
- Recovery trajectories in common musculoskeletal complaints by diagnosis contra prognostic phenotypes. Aasdahl 2021 BMC Musculoskelet Disord.
- Cannabidiol (CBD) products for pain: ineffective, expensive, and with potential harms. Moore 2023 J Pain.
- Moderators of the effect of therapeutic exercise for knee and hip osteoarthritis: a systematic review and individual participant data meta-analysis. Holden 2023 The Lancet Rheumatology.