🇨🇦 🇨🇦
February 15 is National Flag of Canada Day. PainScience is a proudly Canadian small business, selling e-books and other educational content about pain and injury since the mid-2000s. Read more about the project. Given Trump’s tariff bullying and threats to annex America’s greatest historical friend, ally, and trading partner, I think I’ll keep this flag up for the rest of the month… or perhaps permanently.
Detailed guides to painful problems, treatments & more

Is Chinese acupuncture research more like propaganda than science?

 •  • by Paul Ingraham
Get posts in your inbox:
Weekly nuggets of pain science news and insight, usually 100-300 words, with the occasional longer post. The blog is the “director’s commentary” on the core content of PainScience.com: a library of major articles and books about common painful problems and popular treatments. See the blog archives or updates for the whole site.

Last year, JAMA Internal Medicine published a new Chinese study of acupuncture for sciatica, along with an editorial praising it for being “methodologically rigorous.” But it can’t be! An un-blinded trial with subjective outcomes cannot prove the efficacy of acupuncture — especially in a hospital in China! Where strong feelings and ideas about acupuncture are going to warp the data in predictable ways.

A critique of the study emphasized that “Chinese studies on acupuncture are essentially never negative.” The implication is that Chinese acupuncture trials are more propaganda than science, because the researchers and journals are too biased to produce or publish anything but what they want to hear. Maybe negative acupuncture results are so politically verboten that they are de facto censored (if not actually censored).

If you’ve ever read anything skeptical about acupuncture, you’ve probably seen that claim, with its nasty insinuations.

Where does it come from? I mean, other than justified general cynicism about China, acupuncture, and alternative medicine?

It comes from Vickers et al, a scientific review published way back in … 1998.

Flag of China.

Vickers et al. reported that China, Japan, and Korea were producing research in the 1990s that was suspiciously “uniformly favorable to acupuncture,” suggesting a huge problem with publication bias. And skeptics have been cheerfully citing that paper ever since — almost every time the topic comes up.

They have not started citing anything more recent to support the point. This is not just a case of “citation needed,” but fresher citation needed!

Using a very stale citation like that is disconcertingly similar to the kind of thing quacks do to justify their quackery: they find one paper that seems to back them up, and then never stop picking that cherry. Is Vickers et al. a bogus citation perpetrated by the people who are supposed be identifying and avoiding them?

More importantly, do the Chinese actually have a publication bias problem with their acupuncture research? Regardless of whether it was true in 1998, is it true now, in 2025?

Short answer: probably. It’s still a big problem, only modestly improved at best. But I need to show my work on that …

Publication bias

Publication bias” is a disease (say that like Mr. Smith), a constant threat to evidence-based medicine. It’s the scientific publishing industry’s preference for publishing the cool studies, the ones with exciting, promising, or surprising results … while ignoring the boring ones, the negative results, the failed replications — which are just as important scientifically, if not more so. All this makes the body of evidence look better than it actually is, and skews the results of reviews and meta-analysis.

Negative results are also neglected by researchers themselves (and corporations funding research). Don’t like the results of a trial? Don’t even give a journal the chance to reject it! Stick it in a “drawer” — which is where we get the “file drawer effect.”

Some areas of science are more badly polluted with publication bias, and the stink of it is all over journals dedicated to alternative medicine (where seldom is heard a discouraging word). It’s a big part of what puts the pseudo in pseudoscience.

It is extremely plausible that China would have a publication bias problem, of course. It would be more surprising if it didn’t! But I’m fact-checking it out of a clear sense of ethical duty. It would not be fair to Chinese researchers or acupuncturists to continue making this accusation without a decent and recent citation.

“Image of Darth Vader from Star Wars standing in a futuristic spaceship interior. He appears to be using the Force to choke a man while another person looks on. The overlaid text humorously reads, ‘I find your lack of fact checking disturbing.’”

Fresher citations for “Chinese acupuncture research is biased”

There hasn’t been another paper with quite the same strong punchline that Vickers et al. handed to skeptics on a silver platter, but there is data, and it’s still telling more or less the same story, which is also the likely story …

  • In 2020, Long et al reported that there have been some improvements, but “major issues remain in the unclear or high risk of bias for allocation concealment and blinding.” Positive results must still be taken with a grain of salt in that context, despite any improvements.
  • Masuyama and Yamashita reported much the same about Japanese research in 2023: the risk of bias remained “high” or “unclear,” although at least some negative trials were published (20% of trials in the 2000s, and 30% in the in 2010s). Japan is not China, but any pro-acupuncture publication problem there is probably less than China’s.
  • A 2023 review by Li et al found that Chinese studies up to 2021 had glaring problems with methodology compared to non-Chinese studies, which predictably produced data making acupuncture seem more effective than data from other countries. This is a weird one too include, because the authors are acupuncture boosters defending Chinese acupuncture research, not critiquing it. And yet their data — if you ignore their spin — sure looks like bad news about publication bias to me! An earlier paper of theirs is competently rebutted by Leigh Jackson, exposing the hypocrisy of their interpretation. (It’s actually a similar story with Vickers: his 1998 paper was pointing out that Chinese acupuncture was not credible enough to support what he believed, namely that acupuncture works. He went on to publish acupuncture trials in more credible Western journals — but the skeptical position is that his positive results were just as bias-powered as anything coming out of China.)
  • Finally, there’s a Chinese study that’s in the works and designed to shine more light on this problem, and it might replace Vickers as the standard citation for this. The premise of that study is that “registration in acupuncture trials is not satisfactory” … which is almost synonymous with publication bias, because trial registration is the main defence against publication bias. No registration? Publication bias is almost inevitable! Even without a huge cultural enthusiasm for certain results. So that work-in-progress aims to confirm that publication bias is happening, which is practically a forgone conclusion if trial registration is poor enough to inspire the question.

I think skeptics are in fact using a stale citation … that happens to be correct

Vickers alone is definitely a stale citation, not up to the job anymore, and it’s a good example of a “bogus citation” being deployed by skeptics. Which means it’s also a good example of how all people — skeptics included — fail to fact check when it suits their case. If a citation seems to tell us what we want to hear … we are all likely to just go with it.

However, what skeptics want to hear in this case is that Chinese acupuncture research has a publication bias problem … and it very likely does. There’s just not one good, modern, slam-dunk citation for it, unfortunately. Instead, as of 2025, it takes a cluster of citations, a meta-citation, rather than one convenient source.

Meanwhile, Tu et al is yet another example of Chinese research that was designed to make acupuncture look better than it is. Dr. Steven Novella:

A non-blinded study with a subjective outcome is not rigorous, it is scientifically worthless. In fact it is worse than worthless, it is actively harmful because it is useful for propaganda and deception. The editors of JAMA Internal Medicine have now played into this propaganda.

One mighty footnote

Most of this blog post has been stuffed into a single huge footnote in my full guide to acupuncture, which I’ve been updating erratically for most of my now long career. There are about 8300 footnotes on PainScience.com, and I’m pretty sure that is now the single largest of them! All in service of answering that pesky question: Is that really true?

PainSci Member Login » Submit your email to unlock member content. If you can’t remember/access your registration email, please contact me. ~ Paul Ingraham, PainSci Publisher