Shirley McKie case – logical and statistical aspects

I am a computer programmer working in the field of statistics for industry. I think that the original confident belief that Shirley McKie had lied was based on an error of logic. Like the Texas sharpshooter who paints target rings round his bullet holes after they have landed and claims to have hit the bull’s-eye, the suggestion that an improper entry was made to Marion Ross’s house came after the disputed fingerprint identification.

· A denied identification that is not linked to a crime should be treated as having an exceptionally high risk of error, perhaps many orders of magnitude higher than a denied identification that incriminates in a crime that is independently established as fact (other things in the cases being equal).

· It is questionable whether any forensic technique can give conclusive probative evidence that wrongdoing has occurred in the absence of any other facts. Even if the rate of fingerprint error is very low the total number of elimination identifications in all crime investigations is an area of opportunity so large that a disputed identification proves nothing on its own. It is questionable whether there was ever a prima facie case that Shirley McKie committed perjury at David Asbury’s trial.

· If the defence had not secured the services of fingerprint experts who contradicted the identification then jurors in the perjury trial might have made the logical error of thinking that because fingerprint errors are very rare it must be unlikely that McKie was telling the truth (prosecutor’s fallacy)^{1, 2}, or they might have thought that the likelihood that McKie was lying was similar to a normal case where we know that the crime has occurred and a fingerprint identification provides evidence of who did it.

I think that the forensic evidence that gave rise to the Shirley McKie case was not rationally evaluated taking context into account. I think that the correct course was for the Strathclyde police to carry out an investigation to look for other independent evidence of an improper entry to the crime scene. As no such evidence emerged I think that uncertainty should have been the conclusion. Suspicions that McKie had lied should have stopped at that point.

¹ Thompson, W.C. & Schumann, E.L. Interpretation of statistical evidence in criminal trials: The prosecutor's fallacy and the defence attorney's fallacy. 1987, 11, 167-187.

² In a statement about the Sally Clark case the Royal Statistical Society warns of the dangers from frequency estimates being misinterpreted. The assumption that fingerprinting errors are very rare, which will be in the minds of investigators and jurors, is a frequency estimate. RSS statement is here

In Frequency terms:

When a fingerprint identification originates the idea that wrongdoing has occurred the location is set by the forensic evidence. The error rate of fingerprint identification will not be zero so at some place and time an error will occur that cannot incriminate in a known crime. The resulting denial will draw this identification to the attention of the police and they can always propose that the denial is as an attempt to conceal a previously unimagined hypothetical act of wrongdoing. The whole of fingerprinting activity everywhere provides unlimited opportunities for such a thing to occur. Errors may be rare but if the alleged act is equally rare neither is more likely than the other as an explanation for the disputed fingerprint evidence. If there is no other evidence then to be rationally confident that the improper act happened the frequency of such things - specifically when nobody is aware of it, there is no other evidence of it and the perpetrator lies when told of the forensic evidence – must be many times greater than the frequency of misidentification. The more similar the frequencies are, the greater is the danger from not knowing their values precisely³.

When the crime is a fact the criminal act sets the location. We know the frequency of crimes, one per crime scene. The opportunities for forensic error during a single investigation are limited and every latent fingerprint in a crime scene carries a fair chance of having been deposited by the perpetrator. The frequency of errors per crime, therefore, should be very low. When an identification that incriminates in a crime is denied we can usually be confident that the subject is lying, even without knowing the underlying probabilities precisely. But without an independent reason to believe that a crime has occurred, the balance of likelihoods is much less certain and could be weighted towards error even with a very low general error rate.

In Bayesian terms:

The alleged offence was lying about visiting a location in breach of police discipline, and later perjury. The only evidence was an elimination identification that the subject denied depositing. Compared with lying about a crime that we know has happened, the prior probability that someone is lying before considering fingerprint evidence will be very much lower and could be infinitesimal when there is no other evidence that anything improper or unusual has happened (if there is no reason to think that wrongdoing has occurred it probably hasn’t).

With no other evidence there are no local factors to influence the prior probability so we can only generalise. A police officer depositing a fingerprint while disobeying orders in a major crime investigation by entering the crime scene which was secured and guarded and then lying about it when faced with both a verified fingerprint identification and the prospect of a perjury conviction would, we might assume, be very unlikely when there is no evidence to suggest it. A fingerprint elimination error would also be very unlikely in any one investigation. I do not know of any cases similar to the McKie case but I know of some cases of misidentification so perhaps this is the more likely of the two. The prior probability that McKie is lying and the probability of identification error are both very low and uncertain within a broad range of values so the ratio of these two probabilities is highly uncertain³. In these circumstances the posterior probability after considering the fingerprint evidence provides no confidence that McKie’s denial was dishonest. It is questionable whether a juror’s everyday experience would equip them to estimate these probabilities.

³ “Particularly in cases in which there is little other evidence against the suspect, ignorance of the true probability of error creates a disturbing element of uncertainty about the value of the (DNA) evidence.” Thompson, W.C., Taroni, F. & Aitken, C.G.G. (2003). How the probability of a false positive affects the value of DNA evidence. Journal of Forensic Sciences, 48(1), 47-4

This looks at the effects of fingerprint errors, the causes of errors is a separate issue. It considers the effects of errors where the victim is selected at random, such as a failure to individualize leading to a random match (bias is likely to have non-random effects). Whether a competent fingerprint identification has an error rate of zero is another separate matter.

Since 1997 the McKie case has been repeatedly studied and investigated by police officers, a procurator fiscal (prosecutor), judges, forensic experts and, in 2006, by a committee of the Scottish Parliament. In all this time nobody has thought to question the original evaluation of the forensic evidence that gave rise to the case.

I think that the principle described here is important and should be understood not only in relation to the McKie case and fingerprinting but also more generally. I think that forensic results and expert opinion should always be evaluated taking context into account and with the knowledge that errors are possible. In February 2007 the CLPEX forum for fingerprint examiners published “Statistics and Misidentifications” as The Weekly Detail and since then I have participated in discussions in the chat board. The linked pages below are copies of some of my contributions to the discussions.

Sometimes analogies can be useful, The Painted Ping Pong Ball Problem could help give a mental image of how selection affects certainty and when we need to know an error rate precisely.

From a logical point of view using a denied elimination identification as a means of detecting a hidden act of wrongdoing has similarities to using mass medical screening to detect hidden diseases. In mass screening a medical test can be very accurate yet most people who test positive will not have the disease.

There are logical similarities with the Sally Clark case (the Royal Statistical Society used the term “serious error of logic” in connection with that case).

An exercise to see how varying the error rate alters the likelihood of guilt produced some interesting results.

In this type of case the Area of Opportunity could be misunderstood.

This page has thoughts about error rates and prior probabilities when searching databases. This page also looks at similarities between the Mayfield misidentification and the McKie case.

While discussing this in the CLPEX forum it was pointed out that this is a global analysis, taking into account considerations outside the direct sphere of forensic science. While understanding this with this I would say that there is always a place for “whole system” thinking when public safety is involved. I think that nobody should be prosecuted or even assumed to be lying on the basis of a simple frequency estimate (such as the assumption that fingerprint misidentifications are very rare). There must be good reason to believe that the dishonest explanation is much more likely. This is a general point but surely at the very least we should know that a crime or act of wrongdoing has actually happened before someone is subjected to the distresses of not being believed or put through the ordeal of a trial.

Steve Horn.

West Lothian

sz@hornsc.clara.co.uk

Computer programmer working in the field of statistics for industry. Background in engineering (electronics) and quality assurance.