How much of treatment effect is needed to actually matter?
PainSci summary of Williams 2011?This page is one of thousands in the PainScience.com bibliography. It is not a general article: it is focused on a single scientific paper, and it may provide only just enough context for the summary to make sense. Links to other papers and more general information are provided at the bottom of the page, as often as possible. ★★★☆☆?3-star ratings are for typical studies with no more (or less) than the usual common problems. Ratings are a highly subjective opinion, and subject to revision at any time. If you think this paper has been incorrectly rated, please let me know.
There are various ways of measuring improvement in scientific tests of treatments. As this paper points out, “when an outcome measure improves by, say, five points it is not immediately apparent what this means.” How much improvement matters? It is extremely common for experimenters to confirm real, statistically significant treatment effects that are nevertheless trivial. This paper discusses how much benefit is needed to be taken seriously, and cites the damning example of spinal manipulation. Chiropractors routinely claim that the benefits of spinal adjustment are “proven,” but the authors point out that trials have “shown an effect size for manipulation that is less than the threshold for what is clinically worthwhile.”
I think it’s particularly noteworthy that these authors have no particular axe to grind about chiropractic treatment — this is not a paper about that. They simply needed a good example to make their point, and it’s telling that they picked spinal manipulative therapy.
original abstract†Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.
Patient-based outcome measures have been developed to measure the health status of patients suffering from many conditions found in musculoskeletal medicine. Many types have been developed. Generic measures can be used across a broad spectrum of illness and can compare the health of an affected group with that in the general population.1 Condition-specific instruments on the other hand measure the effect of a single condition on health. Compared to generic measures they have a narrower focus and are more sensitive to small but clinically significant changes in health status over short periods. Examples include the Roland–Morris questionnaire2 for low back pain and Western Ontario McMaster universities arthritis index (WOMAC) for osteoarthritis.
related content
These two articles on PainScience.com cite Williams 2011 as a source:
- PS The Chiropractic Controversies — An introduction to chiropractic controversies like aggressive billing, treating kids, and neck manipulation risks
- PS Statistical Significance Abuse — A lot of research makes scientific evidence seem more “significant” than it is
This page is part of the PainScience BIBLIOGRAPHY, which contains plain language summaries of thousands of scientific papers & others sources. It’s like a highly specialized blog. A few highlights:
- Effectiveness of customised foot orthoses for Achilles tendinopathy: a randomised controlled trial. Munteanu 2015 Br J Sports Med.
- A Bayesian model-averaged meta-analysis of the power pose effect with informed and default priors: the case of felt power. Gronau 2017 Comprehensive Results in Social Psychology.
- The neck and headaches. Bogduk 2014 Neurol Clin.
- Agreement of self-reported items and clinically assessed nerve root involvement (or sciatica) in a primary care setting. Konstantinou 2012 Eur Spine J.
- Effect of NSAIDs on Recovery From Acute Skeletal Muscle Injury: A Systematic Review and Meta-analysis. Morelli 2017 Am J Sports Med.