Detailed guides to painful problems, treatments & more

The disaster of scientific publishing

 •  • by Paul Ingraham

Weekly nuggets of pain science news and insight, usually 100-300 words, with the occasional longer post. The blog is the “director’s commentary” on the core content of PainScience.com: a library of major articles and books about common painful problems and popular treatments. See the blog archives or updates for the whole site.

The web comic xkcd, by Randall Munroe, recently satirized scientific publishing (yet again) with a bunch of tediously predictable paper titles:

Adam Meakins, the sweary physio, created this remixed version for the rehab world:

That’s all I was going to share for this post originally — a couple fun comics, poking fun at the overripe target of scientific publishing. But then, oops, I read Benjamin Mazer’s Atlantic piece, “Scientific Publishing is a Joke,” which was inspired by xkcd. So then things got a little more interesting and earnest for this post:

Take a bunch of clever, ambitious people and tell them to get as many papers published as possible while still technically passing muster through peer review … and what do you think is going to happen? Of course the system gets gamed: The results from one experiment get sliced up into a dozen papers, statistics are massaged to produce more interesting results, and conclusions become exaggerated. The most prolific authors have found a way to publish more than one scientific paper a week. Those who can’t keep up might hire a paper mill to do (or fake) the work on their behalf.

The research literature, when stripped of its jargon, is just as susceptible to repetition, triviality, pandering, and pettiness as other forms of communication.

And much more, and much worse. Science itself is fine, mind you, as I always have to point out when griping on this theme. But scientific publishing, never perfect in the first place, has become something of a disaster over the last twenty years. The cliché “unmitigated” would be particularly apt in this case: the disaster really has not been mitigated in any way. Mazer finishes his piece by pointing out that no one really has a clue what to do about any of this:

None of the scientists I talked with could think of a realistic solution. If science has become a punch line, then we haven’t yet figured out how to get rid of the setup.

To be fair, there actually are reform ideas, but none of them seem to have really caught on. And it just keeps getting worse. I went looking for updates on some major industry issues, the one about “gibberish” papers caught my eye, and I found myself far from the original whimsical goal of this post … though perhaps still in the realm of the absurd. In addition to everything that was already wrong with scientific publishing in the 20th Century, here are four examples of serious new (and weird) problems that have emerged so far in this one:

  1. Literal nonsense papers! Computer-generated gibberish, which started as a prankish experiment in 2005, is now apparently a permanent feature of the scientific publishing landscape. Although bizarre, this problem is dwarfed in scale by other industry issues…
  2. Paper mills: “fake-paper factories that churn out sham science.”
  3. Predatory journals are cousins to paper mills, but more dangerously adept at producing science that seems more legitimate, like pseudo-quackery.
  4. Misleading abstracts have always been a problem in science, but it has arguably spun out of control. If you think the papers are bad these days…

And that’s not a comprehensive list, of course — it’s just four obvious examples off the top of my head.

End of post marker

Last post: Now I will hurt and not work right

Next post: A not-so-surprising lack of evidence