Because the test is wrong 5% of the time, it will find 50 people in a thousand who supposedly have the disease, when in fact there’s only one person in a thousand who really has the disease — 1 in 50, or 2% of the people who tested positive. Taleb concludes, “Think of the number of times you will be given a medication that carries damaging side effects for a given disease you were told you had … ”
Below is the account of a well-known test, and an embarrassing one for the medical profession. The following famous quiz was given to medical doctors (which I borrowed from the excellent Deborah Benett’s Randomness).
A test of a disease presents a rate of 5% false positives. The disease strikes 1/1,000 of the population. People are tested at random, regardless of whether they are suspected of having the disease. A patient’s test is positive. What is the probability of the patient being stricken with the disease? Most doctors answered 95%, simply taking into account the fact that the test has a 95% accuracy rate. The answer is the conditional probability that the patient is sick and the test shows it—close to 2%.