Detailed guides to painful problems, treatments & more

Is Homeopathic Arnica Effective for Postoperative Recovery? A Meta-analysis of Placebo-Controlled and Active Comparator Trials

PainSci » bibliography » Gaertner et al 2021
updated
Tags: homeopathy, controversy, debunkery

One article on PainSci cites Gaertner 2021: Does Arnica Gel Work for Pain?

PainSci notes on Gaertner 2021:

This is a rare significant addition to the scientific literature on homeopathic Arnica, a review of the notoriously sloppy body of evidence, written by well-known proponents of homeopathy — and so the slightly positive conclusion is a foregone conclusion.

Homeopathic Arnica has a small effect size over and against placebo in preventing excessive hematoma and other sequelae of surgeries.

Dr. Edzard Ernst points out several flaws beyond the obvious bias, including but not limited to the inclusion of unpublished trials, trials published in junk journals, trials of Arnica in non-homeopathic concentrations, trials that weren’t actually trials, and trials with endpoints that weren’t mentioned in the review.

Remember that small effect sizes extracted from junky little studies are probably all illusory. I doubt there has ever been an example of a small effect size for an implausible treatment in a trial with a high risk of bias that was later reproduced by a high quality trial.

This paper a classic example of garbage-in-garbage-out junk science, and the conclusion is weak sauce even with intense bias-powered p-hacking.


Common issues and characteristics relevant to this paper: ?Scientific papers have many common characteristics, flaws, and limitations, and many of these are rarely or never acknowledged in the paper itself, or even by other reviewers. I have reviewed thousands of papers, and described many of these issues literally hundreds of times. Eventually I got sick of repeating myself, and so now I just refer to a list common characteristics, especially flaws. Not every single one of them applies perfectly to every paper, but if something is listed here, it is relevant in some way. Note that in the case of reviews, the issue may apply to the science being reviewed, and not the review itself.

  1. Garbage in, garbage out — not enough good quality data to meaningfully review/analyze.
  2. Damned with faint praise — technically positive results (at least partially) that don’t actually impress.
  3. A high and/or unacknowledged risk of bias and its consequences (p-hacking, etc).
  4. Any positive results may be attributable to the low validity of statistical significance when testing highly implausible effects (a complex topic, see Pandolfi).

original abstract Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.

Background: Homeopathic Arnica montana is used in surgery as prevention or treatment for the reduction of pain and other sequelae of surgery. Our aim was to perform a metaanalysis of clinical trials to assess efficacy of Arnica montana to reduce the inflammatory response after surgery. Method: We conducted a systematic review and metaanalysis, following a predefined protocol, of all studies on the use of homeopathic Arnica montana in surgery. We included all randomized and nonrandomized studies comparing homeopathic Arnica to a placebo or to another active comparator and calculated two quantitative metaanalyses and appropriate sensitivity analyses. We used "Hegde's g," an effect size estimator which is equivalent to a standardized mean difference corrected for small sample bias. The PROSPERO registration number is CRD42020131300. Results: Twenty-three publications reported on 29 different comparisons. One study had to be excluded because no data could be extracted, leaving 28 comparisons. Eighteen comparisons used placebo, nine comparisons an active control, and in one case Arnica was compared to no treatment. The metaanalysis of the placebo-controlled trials yielded an overall effect size of Hedge's g = 0.18 (95% confidence interval -0.007/0.373; p = 0.059). Active comparator trials yielded a highly heterogeneous significant effect size of g = 0.26. This is mainly due to the large effect size of nonrandomized studies, which converges against zero in the randomized trials. Conclusion: Homeopathic Arnica has a small effect size over and against placebo in preventing excessive hematoma and other sequelae of surgeries. The effect is comparable to that of anti-inflammatory substances.

This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights: