Extraordinary Health Claims

“Thinking at Hell’s gate”, by innoxiuss, CC BY 2.0

I once explained to a reader that a claim made by a popular massage therapist author was completely bogus, and she earnestly asked, “Why would he mislead people like that?” She was actually puzzled! Apparently it had never occurred to her that any health care professional might be a crank, or self-serving, or even just too optimistic. It’s hard for me to relate to such innocence. The world is overflowing with misinformation, particularly about health. Indeed, most of it is problematic in some way, even though much of it is produced with good intentions.

Ironically, bad information is the chief hazard of the Information Age. The glut of unregulated information about your health available on the Internet1 constitutes both an incredible opportunity and a real threat to your safety and your pocketbook. (Even good information is hazardous!2) In the absence of any publication standards, how can you tell the difference between the good and the bad? There are three major signs of poor quality in health information:

1. extraordinary claims
2. author hopes to sell you something
3. no evidence or references offered (or only bogus references)

Quackery Warning Sign No. 1: Extraordinary claims (which are special in health care)

A “claim” is any unverified assertion. But not all claims are created equal. Some claims matter more than others, and some are more dubious. In health care and health science, the stakes are high and many claims involve a lot of strikingly self-serving assertions (like “my Awesome Treatment Method™ will cure you”). If a claim could be used as a bullet-point in a sales pitch, it’s more claim-y. If it makes you (or your profession) look better, it’s more claim-y. And the claim-ier the claim, the more it needs to be backed up.

No one ever uses the word “claim” to describe their own beliefs and sales pitches, of course. Talking about claims is vocabulary for critical thinking and skepticism specifically, where it refers to a particular kind of claim — a special sense of the word — spawned by the thorny ethical challenges of selling care to sick, hurt people. Vulnerable people. (For instance, people suffering from aches and pains that are amazingly poorly understood for this late date in history.)

All claims need critical appraisal and independent verification, of course, but it’s much more important when it has more claim-stink and when people’s health depends on it. Sagan’s classic idea that “extraordinary claims require extraordinary evidence”3 is not just about exotic and/or extremely implausible claims like “aliens abducted me!” or “lake monsters are real” or “I have a machine that produces more energy than it requires, defying the laws of thermodynamics!” Really? Prove it! No one has trouble seeing those kinds of claims for what they are.

But the extraordinariness of claims is, in spirit, also about more mundane but profitable claims, such as “this nutritional supplement will reduce your knee pain” or “vibrating your tissues with sound waves will accelerate your tendon healing.” These are much more common hazards than truly exotic and extreme stuff, especially in health care. They are less obvious and yet more ethically problematic than many other common types of claims.

What can be asserted without proof can be dismissed without proof.

Christopher Hitchens

The history of medical science is full of examples of extraordinary claims that turned out to be correct.

• “Cholera is coming from this water pump” was a radical idea in the 1850s, and it was subjected to considerable skepticism, but John Snow really had found the source of the scourge.4
• “Water is full of bugs too small to see” was dismissed as the ravings of a madman — until enough biologists had a chance to a look through a microscope for themselves.
• “Surgeons must wash their hands” nearly resulted in Joseph Lister’s professional excommunication — but he was right, and the evidence was exceptional and ultimately unavoidable.

History is much more full of extraordinary claims that went absolutely nowhere. To this day, several popular pseudosciences and superstitions persist on the fringes of society, and their advocates complain that they are not taken seriously — but they remain marginalized because they do not present extraordinary evidence, or any at all, to support their extraordinary claims.5

The most obvious kind of extraordinary claim in health care is the panacea, a treatment or medicine that is supposed to be good for anything that ails you. Here are some examples:

• Upper cervical chiropractors claim to be able to treat a wide range of pathologies by manipulation of only the top two joints of the spine (one of several classic chiropractic controversies6)
• F. Batmanghelidj, MD, claims that drinking a lot of water (and buying his book, and taking his workshop) can cure “heartburn, arthritis, back pain, angina, migraines, autoimmune diseases, colitis, asthma, high blood pressure, diabetes.” Wow! This is a particularly classic panacea.7
• Reflexologists claim to be able to treat any organ or system in the body by physical manipulation of only the hands, feet and ears. It’s ridiculous.

Panaceas are usually obvious. Unfortunately, health care is also full of extraordinary claims that are often mistaken for ordinary claims. A combination of widespread public biological illiteracy makes it easy for ignorant and/or deceitful people to make bad promises that seem perfectly reasonable to the average person. There is no easy defense against this except a high degree of caution.

Quackery Warning Sign No. 2: Selling (ka-ching, ka-ching!)

Health care information and the profit (or prestige) motive are not usually good bedfellows.

However, it is not wrong in principle to profit from genuinely valuable health care services, products, or information. In fact, it’s necessary for those who have invested in training, product development or publication to promote themselves and earn a financial return — if what they sell is actually beneficial.

Unfortunately, it’s always been easy to make a profit from selling health care that is not beneficial, or only partially so. Public ignorance about science in general and biology in particular is epidemic. Worse still, people who are sick or hurt are eager to believe in anything that might help ease their pain, no matter how ridiculous or even dangerous. Since the only possible way to sell snake oil is to present deceitful and/or irrelevant information, there is an awful lot of that available.

The only solution for this is education — the buyer must be aware and critical. At the least, bogus health care products, services, and advice can waste your time and money. At worst, they may harm you. Never purchase unless you have been convinced by good quality reasoning and/or evidence. Don’t believe anything you are told by someone with a profit (or prestige) motive unless they support their claims responsibly.

Beware in particular of the most insidious threat of all: the salesman who persuades you to buy an ineffective product or service by giving you good information … about something else! Sneaky, eh? It happens all the time. Indeed, this is a favourite tactic of pharmaceutical manufacturers, who promote their reputations and increase sales of their products by providing perfectly good information to consumers and doctors that has nothing to do with the effectiveness or safety of their products. For example, they provide doctors with high quality patient education tools, such as charts — always with a logo on it.

The point is that you must never judge any health care product, service or advice by anything except the evidence for that product, service or advice — and the more someone stands to gain from your belief, the more careful you must be.

It is not just extraordinary claims that require extraordinary evidence — profitable claims require extraordinary evidence!

Quackery Warning Sign No. 3: No evidence, no references = information malpractice

Unreferenced medical information is almost always useless. A health care publication without footnotes is like a doctor without a license to practice. When it comes to information about health, the stakes are simply too high for publishers to ask readers to take their word for anything.

Like references on a resumé it often doesn’t matter if you actually check citations — what matters is that it can be checked, and the more easily the better. Ideally, citations on the web should contain links to abstracts (summaries), or even the full text of the article. Recent publications should generally be preferred over older ones.

A nation’s currency rests on metal8 — there has to be real gold somewhere, or the value of our money is a mass delusion. In a similar sense, citations to scientific literature is the bedrock of medical information. Every medical idea, every recommended treatment, every new idea has to be — sooner or later — based on evidence published in a decent peer-reviewed scientific journal, and, if it can’t be, it had better have a good excuse. The greater the risks, the more extraordinary the claim, the more expensive the service … the greater the need for this trail of bread crumbs to the original research.

Not all health care information has to rise to this standard. Some of the articles I publish don’t require a lot of scholarship, for instance, because they ask nothing of the reader: they don’t have to buy anything, pick sides in a controversy, or make a medical decision based on the information. The need for evidence goes up with the stakes.

Unsupported medical opinions are sometimes accepable — the practice of health care would grind to a halt without them. But opinions are not evidence, must be clearly identified, and opinions that matter more must be more clearly identified. It’s irresponsible for an expert to disguise an opinion as fact, or to carelessly allow opinion to stand as fact in the minds of readers who may take expert opinions as seriously as good evidence. Experts must use cautious language when expressing opinions — such as “in my opinion” or “I believe.”

A document without references or acknowledgements of uncertainty is “information malpractice.”

A final quackery consideration: the source

You will be reasonably safe from bad information if you expect any significant claim to be supported by evidence, and if you raise the bar for extraordinary claims; if you are even more cautious when someone is trying to sell you something; and if you expect to see credible references at the end of any serious medical document. Do these things, and you are officially a “critical reader,” far ahead of the pack.

But there is one more important point, and it’s a good one to finish with.

There is no such thing as an unbiased or objective source of information,9 although some sources are more so than others. I believe that this is a myth perpetuated by journalists and profitable news media who want you to believe that they are a superior source of information. The truth is, everyone has a bias, and media consumers simply need to know what the bias is.

For instance, no one accuses Michael Moore or Rush Limbaugh of being unbiased — but they are good sources of information because we know what their biases are. The worst source of information is an unidentified source, where you do not know and cannot even guess at the biases.

Oddly enough, this is often an advantage of searching for information on the web: the biases of authors are often clear, and it’s often easy to find a wide variety of viewpoints quickly.

Please never be impressed by credentials alone. Credentials can be faked, and they often are. Although credentials may indicate expertise, and give you some idea of what the biases of the author may be, they do not guarantee good information. To be blunt, I believe that credentialled experts are often the worst of the “information offenders.” Unfortunately, it is a strong feature of human nature to rest on one’s laurels, and many professionals don’t seem to think that they need to support their claims.

Happy critical reading to you!

About Paul Ingraham

I am a science writer in Vancouver, Canada. I was a Registered Massage Therapist for a decade and the assistant editor of ScienceBasedMedicine.org for several years. I’ve had many injuries as a runner and ultimate player, and I’ve been a chronic pain patient myself since 2015. Full bio. See you on Facebook or Twitter., or subscribe:

SUBSCRIBE

Get PainSci updates in your inbox, mostly small, 0–3/week depending on coffee dosage. More info.

Appendix: What makes a scientific paper so scientific?

A scientific paper really is a different animal than a normal “article.” Most people have never read a scientific paper. Even though I have done a lot of homework in my field, I have probably only read a few dozen scientific papers at most.

What makes a scientific paper special is that there are strict standards for their publication, and clear general guidelines for all scientists about what constitutes good evidence. Publication in a scientific journal is not a guarantee that information is good — like democracy, it’s an imprefect system, but it’s the best we’ve got.

A good scientific paper has the following qualities:

• Peer-reviewed. Most scientific journals publish only “peer-reviewed” articles: that is, articles that must be approved by a group of scientists with relevant expertise. This means that you can rest assured that no wacky theories will be published in a scientific journal without first impressing several other scientists with the evidence. (This can also be a limitation, of course — good new ideas are sometimes suppressed.)
• Primary research. Scientific journals publish a variety of commentaries and editorials, tutorials and “literature reviews.” These can be useful, but by far the most important kind of scientific article is reporting primary research: direct experiment and study.
• Larger sample sizes. Most medical research involves studying a group of people. Generally speaking, the larger the group, the more meaningful or relevant the results of the research. A small sample is 25, and may be adequate depending on the circumstance. Less than that is rarely useful.
• Randomization. It’s important that the groups of people that medical scientists study be randomized. The problem is that it’s easy to inadvertently skew results by choosing people who are likely to produce certain results. A classic example, and a mistake no good researcher would make, would be to ask how common back pain is by studying a group of massage therapy clients — the results would have no general significance, because massage therapy clients are much more likely to have back pain than most groups of people. So randomization (by any of several methods) is quite important.
• Control. It’s also important that a study group be “controlled” — that is, that it be compared to a neutral group. It is meaningless to study the effect of a surgery for knee injury, for instance, if you don’t compare results to a group of people receiving a fake surgery (a placebo). If you didn’t compare, you could easily make the mistake of thinking that the surgery was effective, when in fact both groups might get better just as quickly. Believe it or not, exactly that research has been done10 — and it is one of the best demonstrations of the importance of control groups in recent history. Unfortunately, a surprising amount of uncontrolled scientific research gets published — so watch out for this one. Research without a control group doesn’t tell us nearly enough!

So, now you are a seriously informed reader: the next time you see a dubious claim, look for supporting evidence published in a “peer-reviewed scientific journal.” Look it up. Is the paper “primary research”? Is so, was there a decent “sample”, and was it “randomized” and “controlled”?

Related Reading

• Pain Science Reading Guide for Skeptics — A tour of PainScience.com for readers who have doubts and concerns about the validity and efficacy of popular treatments for injuries and chronic pain
• Studying the Pain Studies — Tips and musings about how to understand (and write about) the extremely flawed science of pain and musculoskeletal medicine
• 13 Kinds of Bogus Citations — Classic ways to self-servingly screw up references to science, like “the sneaky reach” or “the uncheckable”
• The Not-So-Humble Healer — Cocky theories about the cause of pain are waaaay too common in massage, chiropractic, and physical therapy
• Pseudo-Quackery in Physical Therapy — The large, dangerous grey zone between evidence-based care and overt quackery in rehab and pain treatments
• Therapy Babble — Hyperbolic, messy, pseudoscientific ideas about manual therapy for pain and injury rehab are all too common
• Ioannidis: Making Medical Science Look Bad Since 2005 — A famous and excellent scientific paper … with an alarmingly misleading title
• Statistical Significance Abuse — A lot of research makes scientific evidence seem much more “significant” than it is
• Why “Science”-Based Instead of “Evidence”-Based? — The rationale for making medicine based more on science and not just evidence… which is kinda weird
• Quackery Red Flags — Beware the 3 D’s of quackery: Dubious, Dangerous and Distracting treatments for aches and pains (or anything else)

Books for ramping up your critical thinking skills about health care:

• The demon-haunted world: science as a candle in the dark (book), by Carl Sagan and Ann Druyan. Amazon.com ❐ This treatise on critical thought is one of my all-time favourites, a book that makes ten others unnecessary, the kind of book that will change how you think forever.
• The Skeptics' Guide to the Universe: How to know what's really real in a world increasingly full of fake (book), by Steven Novella and Bob Novella.
• Bad Science (book), by Ben Goldacre. Amazon.com ❐
• Bad pharma: how drug companies mislead doctors and harm patients (book), by Ben Goldacre. Amazon.com ❐ British science journalist Ben Goldacre is fascinating and awesome. Read this book, or at least watch this TED talk about it. [munches popcorn]
• Surgery: The ultimate placebo (book), by Ian Harris (book review). What if a fake surgery had the same effect as a real one, because the “active ingredient” in surgery is just the dramatic ritual? What if surgery delivers a huge placebo effect? This often appears to be case, as shown in a few well-known examples. Most surgeries have still never been subjected to the gold standard of evidence-based medicine, the randomized controlled trial. Instead, they are based mainly on tradition, authority, and the “common sense” of surgeons, who have been slow to embrace the need to subject their methods to trials, citing a list of typical reasons — none of which stand up to scrutiny, and sound more like turf-defending excuses every year. In this superb book, orthopedic surgeon Dr. Ian Harris explores the shameful history of untested surgeries in detail. It’s fascinating, and mostly easy enough reading even for patients. There’s a free excerpt from the book that you can read to get started. Many scientific papers before and since publication of the book have supported Harris’ position.
• Chiropractic Fraud and Abuse: An Insider's Lament (book), by Preston H Long. Amazon.com ❐ SBM’s review by Dr. Hall calls A Chiropractor’s Lament a “valuable addition to the literature on chiropractic, combining Long’s personal story with everything you never wanted to know about chiropractic. It’s fun to read and packed with information. Even if you think you’ve heard it all before, there are revelations here that will be new to you, that will elicit surprise, indignation, and laughter.”
• The Invisible Gorilla: How our intuitions deceive us (book), by Christopher Chabris and Daniel Simons. Christopher Chabris and Daniel Simons created of one of psychology’s most famous experiments, the “invisible gorilla” experiment, which demonstrated a bizarre gap in the fidelity of our perceptions. When we focus on anything, we become amazingly blind to anything else — even a man in a gorilla suit walking right through our field of view. This is known as “inattentional blindness” (which connotes an overall lack of attention, but this is about what happens to us when we *are* paying attention to something). In this book, they describe the weird implications of several more years of related research, using story and surprising science to demonstrate that our minds and senses don’t work the way we think they do. In particular, they argue that we *do not notice* much more than we realize — even the seemingly obvious. A strong theme of the book is that there is a very great deal going on in our noggins that is not intuitively evident, and that consciousness is an incredibly fragmented representation of reality. And they're not just speculating! They draw their conclusions from a parade of bizarre and yet concrete experimental results. It's a bit dizzying.
• Smile or Die: How positive thinking fooled America and the world (book), by Barbara Ehrenreich (book review). A surprisingly dysfunctional cult of optimism sprung up around breast cancer in the 1990s. Barbara Erenreich wrote about it brilliantly for Harper’s way back 2001, eventually leading to her amazing book, Smile or Die (“Bright-sided” in the US: “How How Positive Thinking is Undermining America”).
• Trick or Treatment: The Undeniable Facts About Alternative Medicine (book), by Simon Singh and Edzard Ernst. Amazon.com ❐ A thorough examination and judgement of more than thirty of the most popular “alternative” treatments, such as acupuncture, homeopathy, aromatherapy, reflexology, chiropractic and herbal medicine. The ultimate verdict on alternative medicine is delivered for the first time with clarity, rigour and authority.
• 
  All in My Head: An epic quest to cure an unrelenting, totally unreasonable, and only slightly enlightening headache (book), by Paula Kamen. Amazon.com ❐ Like PainScience.com, this book offers an unusual combination of both humour and information about pain. Kamen is a completely engaging writer, and tells her story with both journalist rigour and personality.

Websites:

• QuackWatch: Your Guide to Quackery, Health Fraud, and Intelligent Decisions (QuackWatch.org). Quackwatch fights health-related frauds, myths, fads, fallacies, and misconduct. Its primary focus is on health consumer information that is difficult or impossible to get elsewhere. Dr. Stephen Barrett, the founder of QuackWatch and author of most of the articles, is one of the great figures of anti-quackery activism. For many years now, I’ve been watching his work and the controversies and legal battles that swirl around QuackWatch, and I have been consistently impressed by Dr. Barrett’s integrity and intelligence — and unimpressed by the tactics and quality of his critics. He is also providing a vital service and a sorely neglected perspective on health care.
• Committee for the Scientific Investigation of Claims of the Paranormal (skepticalinquirer.org). CSICOP encourages the critical investigation of paranormal and fringe-science claims from a responsible, scientific point of view and disseminates factual information about the results of such inquiries to the scientific community and the public. CSICOP.org publishes the venerable magazine The Skeptical Inquirer, the oldest periodical for skeptical minds, and they have archives of full text articles reaching back many years available online.
• Skeptic: Extraordinary claims, revolutionary ideas, and the promotion of science (skeptic.com). The Skeptics Society, headed by Dr. Michael Shermer, is a scientific and educational organization for “anyone curious about controversial ideas, extraordinary claims, revolutionary ideas, and the promotion of science.” In many ways, this is the best of the skeptical websites, and the organization also publishes a pretty good podcast.
• Snopes.com: Urban Legends Reference Pages (snopes.com). Barbara and David P. Mikkelson have been publishing the Urban Legends Reference Pages on Snopes.com since 1995. It is one of the most thorough collections of debunkery available anywhere. Although relatively few items concern health care specifically, it is an essential bookmark for every critical thinker.
• James Randi Educational Foundation: An educational resource on the paranormal, pseudoscientific and the supernatural (web.randi.org). James Randi has an international reputation as a magician and escape artist, but today he is best known as the world’s most tireless investigator and demystifier of paranormal and pseudoscientific claims. The James Randi Educational Foundation is a not-for-profit organization founded in 1996. Its aim is to promote critical thinking by reaching out to the public and media with reliable information about paranormal and supernatural ideas so widespread in our society today. The JREF also sponsors the “One Million Dollar Paranormal Challenge,” which offers a million bucks to anyone who demonstrates a paranormal ability or phenomenon. Since 1964, every single one of about 1000 challengers has either failed the preliminary test, or failed even to agree to acceptable rules for the test.
• Skeptic’s Guide to the Universe (theskepticsguide.org). The SGU is by far the best of the skeptical podcasts, and has attracted an audiences of hundreds of thousands of listeners over the last few years. Led by the almost alarmingly competent and productive Yale neurologist, Dr. Steven Novella (also the Executive Editor of ScienceBasedMedicine.org), every episode is a great mix of news, humour, substantive interviews and entertaining regular features. I’ve listened to every single one.
• Science-Based Medicine: Exploring issues and controversies in the relationship between science and medicine (sciencebasedmedicine.org). Science-Based Medicine is now the single best source of critical thinking about health care available anywhere. Founded by Yale neurologist Dr. Steve Novella of the Skeptic’s Guide to the Universe, it’s written by several physicians who are alarmed by the soaring popularity of dangerous and pointless alternative health care. The title of the post expresses the elegant idea that the value of health care ideas must not only be based on evidence, but must also be reasonably consistent with a well-established body of scientific knowledge of how the human body works.

Notes

1. In 2012, the The Journal of Foot & Ankle Surgery ranked 136 websites about common foot and ankle diagnoses. Expert reviewers gave each a quality score on a scale of 100. The average score? Just below fifty. Fifty! See Smith et al.

   Or see Starman et al. for a review of other kinds of health care information (with nearly identical grades).

   P.S. These references are aging now… but nothing’s really changed!

2. Googling your symptoms is as unavoidable as it is unwise. We almost literally cannot help ourselves. I imagine there is a modern epidemic of reckless, panicky self-treatment based on really bad Internet-powered self-diagnosis — something I am acutely aware of every time I hit my magic “publish” button. (With great power … ) The Flemish government is acutely aware of this problem too, which is why they commissioned a couple of (hilarious) public service announcements about it, one that’s pure sketch comedy, and one about their clever delivery method: the clever monkeys used Google’s own advertising program to target people searching for common symptoms!

3. An interesting debate has gone on … between those who think that all doctrines that smell of pseudoscience should be combated and those who believe that each issue should be judged on its own merits, but that the burden of proof should fall squarely on those who make the proposals. I find myself very much in the latter camp. I believe that the extraordinary should be pursued. But extraordinary claims require extraordinary evidence.

   Broca’s Brain, by Carl Sagan (p62, 1972 hardcover edition)

   Carl Sagan was the Twentieth Century’s champion of public education about science, the scientific method, and critical thinking. His statement about extraordinary claims is perhaps the most famous of all comments ever made about skepticism, and it is the most important piece of wisdom an information warrior needs in the Twenty-First Century. This is why we ask “what would Carl Sagan do?”

4. John Snow was the father of epidemiology, the science of diseases and how they spread. In 1854, Snow mapped cases of cholera and traced the disease to its origin, and proved with a simple test that it was coming from a polluted well: he removed the pump handle, and the cholera outbreak stopped. Naturally, this conclusion was scoffed at by those with competing theories (“bad air”), producing one of my favourite history of medicine stories: to prove that cholera was not in the water, some smartypants biologist drank a glass of the stuff … and got away with it, because he just happened to be immune. It’s a great example of why “experience” is not trustworthy, and why it takes so long to settle even relatively straightforward puzzles in medicine.
5. Consider the example of Harry Houdini, who sincerely wanted a medium to help him talk to his dead mother — but the master of illusion could not find a medium who was not an obvious fraud to him. Or, consider James Randi, a prominent skeptic and debunker, who has offered a million-dollar prize for the last several years to anyone who can convincingly demonstrate a paranormal ability such as ESP, telekinesis or clairvoyant interpretation of Tarot cards. No challenger has yet won the prize.
6. Ingraham. The Chiropractic Controversies: An introduction to chiropractic controversies like aggressive billing, treating kids, and neck manipulation risks. PainScience.com. 9383 words.
7. Ingraham. Water Fever and the Fear of Chronic Dehydration: Do we really need eight glasses of water per day? PainScience.com. 5779 words.
8. Well, that’s how it started anyway! The gold standard was abandoned by all nations. All currencies are now fiat — that is, their value is basically just declared by the government, so modern money is based on confidence in the government (and purchasing power) rather than gold. And yet, critically for my analogy, most countries do maintain substantial gold reserves! Probably not for nothing.
9. Ingraham. Objectivity is Overrated: A response to the common accusation of bias and the mythical virtue of objectivity and journalistic “balance”. PainScience.com. 685 words. Objectivity and balance are highly over-rated as journalistic virtues. They are mostly a pretentious delusion, and we should never trust anyone who claims to be objective. Instead of expecting that, look for someone with a “view from somewhere” (Rosen) from someone who isn’t afraid to disclose and own where they are coming from. The ideal is not to be unbiased, but to be biased with integrity.
10. Moseley JB, O’Malley K, Petersen NJ, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002 Jul 11;347(2):81–8. PubMed 12110735 ❐ PainSci Bibliography 56845 ❐

    This is landmark and fascinating study showed that people with osteoarthritis improved equally well regardless of whether they received a genuine surgical procedure or a sham.

    From the abstract: “In this controlled trial involving patients with osteoarthritis of the knee, the outcomes after arthroscopic lavage or arthroscopic debridement were no better than those after a placebo procedure.”

    Two decades later, this isn’t quite so surprising anymore — many similar studies have been done since — but it was a truly amazing result at that time. It is a particularly striking example of the placebo effect and implies that belief can have an effect even on seemingly “mechanical” problems.

    In 2008, these specific results were fully supported by a Cochrane Collaboration review (Laupattarakasem) which concluded that “there is ‘gold’ level evidence that arthoscopic debridement has no benefit,” and by New England Journal of Medicine (Kirkley) which reported that “surgery for osteoarthritis of the knee provides no additional benefit to optimized physical and medical therapy.”

    Surgeries have always been surprisingly based on tradition, authority, and educated guessing rather than good scientific trials; as they are tested properly, compared to a placebo (a sham surgery), many are failing. Moseley 2002 was the first of many to compare orthopedic (“carpentry”) surgeries to shams. By 2016, at least four more such procedures had been shown to have no benefit (Louw 2016), and that trend has continued since.

    The need for placebo-controlled trials of surgeries (and the damning results) is explored in much greater detail — and very readably — in the excellent 2016 book, Surgery: The ultimate placebo, by Ian Harris.