Statistically insignificant evidence that “more is better” at the gym

PainSci » bibliography » Schoenfeld et al 2016
updated Sep 18, 2018

Tags: exercise, strength, self-treatment, treatment, muscle

One page on PainSci cites Schoenfeld 2016: Strength Training Frequency

PainSci commentary on Schoenfeld 2016: ?This page is one of thousands in the PainScience.com bibliography. It is not a general article: it is focused on a single scientific paper, and it may provide only just enough context for the summary to make sense. Links to other papers and more general information are provided wherever possible.

As of mid-2016, this is the most credible review of the relationship between training volume and hypertrophy performed (replacing Krieger 2010). As written, the authors promote a “more is better” conclusion, but there is a “significant” problem with that: the results that most dramatically support that conclusion were not statistically significant, and this should not be brushed aside. The only results that were statistically significant show an obviously modest effect: more is better, but not by a great deal.

This is undoubtedly why, when summarizing his results in a blog post, Dr. Schoenfeld writes, “Performing less than 5 weekly sets per muscle produced an average hypertrophic gain of 5.4%. Not too shabby. So if you are time-pressed and not concerned about achieving the upper limits of your muscular potential, it should be heartening to know that you can build an impressive physique without spending a lot of time in the gym.”

His major take-home message, however, is that “there is a clear dose-response relationship between volume and hypertrophy” and “10+ sets produced almost twice the gains as performing less than 5 weekly sets per muscle.” This is problematic. It sounds impressive but it was not statistically significant (p = 0.076), and should not be held up as evidence that “more is better” in resistance training. Nor was it correct to assert that “the probability of an effect was nevertheless very high”: that isn’t how P-values work. In fact, it’s common and notorious error (see Statistical Significance Abuse, passage beginning “Above all, a good p-value is not a low chance that the results were a fluke or false alarm … .”.)

The only safe conclusion to draw from this data is the one based on the only statistically significant result: the highest volumes studied were “associated with a 3.9% greater average increase” than the lowest volumes. In other words, more is better, but this evidence does not indicate that it’s much better … or even proportionately better.

~ Paul Ingraham

original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.

The purpose of this paper was to systematically review the current literature and elucidate the effects of total weekly resistance training (RT) volume on changes in measures of muscle mass via meta-regression. The final analysis comprised 34 treatment groups from 15 studies. Outcomes for weekly sets as a continuous variable showed a significant effect of volume on changes in muscle size (P = 0.002). Each additional set was associated with an increase in effect size (ES) of 0.023 corresponding to an increase in the percentage gain by 0.37%. Outcomes for weekly sets categorised as lower or higher within each study showed a significant effect of volume on changes in muscle size (P = 0.03); the ES difference between higher and lower volumes was 0.241, which equated to a percentage gain difference of 3.9%. Outcomes for weekly sets as a three-level categorical variable (<5, 5-9 and 10+ per muscle) showed a trend for an effect of weekly sets (P = 0.074). The findings indicate a graded dose-response relationship whereby increases in RT volume produce greater gains in muscle hypertrophy.

original abstract †Abstracts here may not perfectly match originals, for a variety of technical and practical reasons. Some abstacts are truncated for my purposes here, if they are particularly long-winded and unhelpful. I occasionally add clarifying notes. And I make some minor corrections.

related content