Detailed guides to painful problems, treatments & more

Spectacularly unreliable MRI “results”

 •  • by Paul Ingraham
Get posts in your inbox:
Weekly nuggets of pain science news and insight, usually 100-300 words, with the occasional longer post. The blog is the “director’s commentary” on the core content of PainScience.com: a library of major articles and books about common painful problems and popular treatments. See the blog archives or updates for the whole site.

There have been so many site updates lately, dozens in January alone, that it’s tough to choose one to focus on for the blog. But the results of this late 2016 study (Herzog et al) blew my mind more than anything else I’ve read recently. If I’m judging by WTF-factor, this item wins hands-down.

I already knew MRI is misleading — everyone knows that, if they know anything about medical imaging and back pain. But it may be worse than I thought …

CT scan depicting a vertebrae in cross-section with clearly broken spinal fixations, a screw floating loose in soft tissue.

CT scan of a literal loose screw in my wife’s titanium spinal implants … which the radiologist missed. He also missed the other one.

If you send the same patient to get ten different MRIs, interpreted by ten different radiologists from different facilities, apparently you get ten markedly different explanations for her symptoms. A brave 63-year-old volunteer with sciatica allowed herself to be scanned again and again and again for science. The radiologists — who did not know they were getting the “secret shopper” treatment — cooked up forty-nine distinct “findings.” Sixteen were unique; not one was found in all ten reports, and only one was found in nine of the ten. On average, each radiologist made about a dozen errors, seeing one or two things that weren’t there and missing about ten things that were.

That’s a lot of errors, and not a lot of reliability. The authors clearly believe that some MRI providers are better than others, and that’s probably true, but we also need to ask the question: is any MRI actually reliable?

I’ve seen some startling examples of imaging shenanigans personally, like that one time a radiologist missed literal loose screws — two of them — embedded in my wife’s back. I found them myself. And I’ve heard of many more. I’m not blaming radiologists (I think), because detecting clinically significant things with magic machines is a super hard problem to solve (signal detection theory and all that). But Jebus! For all my cynicism, what Herzog et al found was way worse than I would have predicted. Clearly we need to take all MRI “results” with a brick-sized grain of salt. MRI has now been added to my growing list of examples of unreliable diagnostic methods.