Claims about subgroups of back pain study subjects have “low credibility”

PainSci » bibliography » Saragiotto et al 2016
updated Apr 13, 2023

Tags: back pain, debunkery, bad news, neat, scientific medicine, bad science, pain problems, spine

Four pages on PainSci cite Saragiotto 2016: 1. The Complete Guide to Low Back Pain 2. The Chiropractic Controversies 3. Does Spinal Manipulation Work? 4. Cognitive Behavioural Therapy for Chronic Pain

PainSci notes on Saragiotto 2016:

“The credibility of subgroup claims in back pain trials is usually low, irrespective of the strength of the authors’ claim.” This interesting conclusion really needs some context …

A subgroup claim is a claim that a treatment works better for a specific group of back pain sufferers — patients with specific signs or symptoms that might indicate the nature of their back pain. Patients for back pain studies are usually quite generic (e.g. people with “chronic non-specific low back pain for more than six months”). There could be many types of back pain going on in that group. It makes perfect sense that some of those study subjects might respond better to a treatment than others, but such “subgroup claims” have become problematic in research, and especially back pain research. Are subgroups a holy grail? Or just a handy excuse for blinkered proponents of unproven treatment methods?

“If only they’d studied the right kind of back pain patients” could be viewed as a legitimate gripe about a study’s limitations, or it could be a great example of special pleading (conveniently disguised as a legitimate gripe). I have often suspected the latter, because subgroup hopes so often seem to be the loudest when expressed by ideologues and true believers, professionals with an obvious financial interest in evidence supporting their pet theories. These are the folks that can be counted on to make subgroup claims.

Back to this study: apparently Saragiotto et al were wondering about the same problem, so they systematically examined the credibility of subgroup claims in back pain trials … and they confirmed their bias (I assume) that the “credibility” of subgroup claims is low, suggesting are indeed often a form of special pleading, baked right into actual scientific studies — never mind biased clinicians commenting on them in arguments online.

original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.

OBJECTIVES: To assess the credibility of subgroup claims in back pain randomized controlled trials.

STUDY DESIGN AND SETTING: A sample of reports of back pain trials from 2000 to 2015 that provided a subgroup claim were included (n=38). Two reviewers independently assessed risk of bias and credibility of subgroup claims as well as the strength of the author's claim. The credibility of subgroup claims was assessed using a 10-criteria tool, and strength of the subgroup claims was assessed based on seven criteria to categorize claims into a reasonably strong claim of a definitive subgroup effect or a more cautious claim of a possible effect.

RESULTS: A total of 91 claims of a subgroup effect were reported in the 38 included trials, of which 28 were considered strong claims of a definitive effect, and 63 were cautious claims of a possible effect. None of the subgroup claims met all 10 credibility criteria, and only 24% (22 claims) satisfied at least five criteria. The only criteria satisfied by more than 50% of the claims were if the subgroup variable was a characteristic measured at baseline, and whether the test of interaction was significant. All other criteria were satisfied by less than 30% of the claims. There was no association between the credibility of subgroup claims and the journal impact factor, risk of bias, sample size, or year of publication.

CONCLUSION: The credibility of subgroup claims in back pain trials is usually low, irrespective of the strength of the authors' claim.

original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.

related content