Medical Screening

Some people question whether mass medical screening is a good use of resources. Even if the test is highly accurate most people who test positive will be completely healthy.

"False Positive Screening For Cancer Found To Be Frequent And Costly"

http://www.sciencedaily.com/releases/2004/12/041220002224.htm

"Over 10 years, one third of the women screened had abnormal test results requiring additional evaluation, even though no breast cancer was present. … Physicians should educate women about the risk of a false positive result of a screening test for breast cancer.”

http://content.nejm.org/cgi/content/abstract/338/16/1089

Here is why this happens. A medical test might be 99% accurate in the sense that out of 100 tests on people who have got the disease 99 of them will get a positive result and 1 will get a negative (a false negative rate of 1%) and out of 100 tests on people who do not have the disease 1 of them will get a positive result and 99 will get a negative (a false positive rate of 1%). With this test, when someone is told that they test positive, what are the chances that they have got the disease?

The answer is not 99%. That is the answer to this question, “what are the chances that someone will test positive given that they have got the disease?”. 99% is also the answer to “what are the chances that someone will test negative given that they are healthy?” The question we want an answer to is, “what are the chances that someone has got the disease given that they test positive?”

To answer this we need to know how likely it is that someone has got the disease before they take the test (the prior probability). Say in the general population there is reason to believe that 1 person in 1000 has got the disease without showing any symptoms. Of 1000 people tested during mass screening, the 999 who are healthy will all be subject to the 1% false positive rate so 10 of them will test positive. The one person who does have the disease will also (almost certainly) test positive. So in the mass screening programme 10 out of every 11 people who test positive will not have the disease.

You might be surprised that about 90% of results are wrong from a test that is 99% accurate, but that is not the best way to think about the test. The test modifies prior odds of 1 in 1000 to posterior odds of about 1 in 10. The test on its own does not prove the person has the disease but it might be useful to know that someone has got a 1 in 10 chance of having the disease.

If, instead of mass screening the test was performed only on people who show symptoms then things would be very different. If the symptoms mean that the chance of having the disease is 1 in 10, then out of 1000 such people tested, 100 of them will have the disease (rather than just 1 with mass screening) and 900 will not have the disease. This will result in 99 good positives, 9 false positives and 1 false negative. The same test with the same error rate changes from 1 positive out of 11 having the disease to 99 positives out of 108 having the disease - because the prior probability is different.

From a logical and statistical point of view using fingerprint identification to solve known crimes is like performing a medical test on people with symptoms. Every latent print in a crime scene carries a fair prior probability of having been deposited by the perpetrator of that crime. The prior probability that a latent print was deposited during some unknown crime for which there is no other evidence could be infinitesimally small. Using denied elimination identifications as a means of detecting hidden acts of wrongdoing is like using mass medical screening to detect hidden diseases.

Women who are repeatedly tested for breast cancer have a cumulative risk of getting a false positive over their lifetime. Police officers who are repeatedly in elimination lists have a cumulative risk of misidentification over their careers so I wonder why there have not been more Shirley McKie-type cases. Either fingerprinting is phenomenally safe or something similar has happened in the past but when it did, it was handled differently.

Back to previous document