The Prosecutor’s Fallacy

The victim of a street robbery could not describe the robber but could describe a distinctive piece of jewellery that he was wearing. Police inquiries revealed that the jewellery was made locally, only 5 of the type were made and they were all sold recently in the town. Let’s consider 2 situations.

Case A

A police officer had a hunch about one man who he thought might have carried out the crime, based solely on previous criminal activity. The police officer visited the man. He was found to be wearing the exact type of jewellery described by the victim.

Case B

While on patrol a police officer spots a man wearing the type of jewellery described by the victim. Enquiries reveal that this man has a criminal record.

In both cases the suspect was arrested and accused of committing the robbery, which he denied. During the court case the prosecutor told the jury that the town has a population of 250,000. 5 examples of the type of jewellery were made so the odds of someone having one are 1 in 50,000. Chance, therefore, can reasonably be ruled out as an explanation for the jewellery being found with the accused. The only reasonable explanation is that the item of jewellery is the one that the robber was wearing, so the accused must be guilty.

In case B the defence counsel pointed out that 5 of the type were sold so there are another 4 people in the town who could equally have carried out the crime. The odds that his client is innocent, based on the jewellery evidence, are not 1 in 50,000 but 4 in 5.

How can it be that in case A the rarity of the jewellery provides reasonable certainty that the accused is guilty, but in case B any perceived certainty is an illusion?

The answer is that the odds of 1 in 50,000 only apply to people who are chosen AT RANDOM. In case A the suspect was chosen BEFORE anything was known about possession of the article. With respect to the jewellery this is random selection. In case B the suspect was chosen AS A RESULT of possessing the article. This is non-random selection.

The prosecutor was right in case A but in case B he fell foul of the prosecutor’s fallacy. By the jewellery evidence, the suspect changes from being almost certainly guilty to probably innocent just because of the selection method (of course the suspects would be different people).

The prosecutor’s fallacy in a nutshell is: You cannot use the rarity or low likelihood of something as proof of guilt if the suspect was selected because of that thing out of a very large pool. If you do, you have committed the fallacy.

Abnormal Risk in the Shirley McKie Case

Consider 2 outwardly-similar fingerprint cases.

Fingerprint case A

An unidentified police officer is spotted going in to a crime scene and failing to make a note in the attendance log kept at the entrance. The Chief Inspector questions everyone who has been given permission to enter the crime scene. All of them can prove that they were elsewhere at the time of the un-recorded entry. The Chief Inspector is angry to think that an unauthorised officer entered the crime scene and he wants to know who it was. A few days later the fingerprint department announce that a police officer was identified from a fingerprint in the crime scene. This officer did not have permission to enter and vehemently denied it when told of the identification.

Fingerprint case B

A fingerprint department announce that a police officer was identified from a fingerprint in a crime scene. This officer did not have permission to enter and vehemently denied it when told of the identification. Despite a thorough investigation no evidence is found to suggest that anyone had entered the crime scene who did not have the necessary permission.

In both cases the Chief Inspector knew that fingerprint misidentifications are extremely rare. Given that there is no reason at this point to question the competence of the fingerprint department, the odds of a misidentification must be 1 in many tens or hundreds of thousands, maybe even more. A misidentification, he concluded, can reasonably be ruled out. The only reasonable explanation was that the officer deposited the fingerprint, so he will accuse the officer of lying and initiate disciplinary proceedings.

In case B a Quality Assurance (QA) officer in the fingerprint department heard about the case and it made him feel uneasy. He remembered from his QA training that if there was a very low likelihood that a fingerprint was deposited as suggested by the police (given what is known and unknown), there was a heightened chance that the case was based on a misidentification (something to do with “prior probability” he recalled). He called the Chief Inspector and urged caution, pointing out that before the identification nobody had any reason to imagine that any unauthorised person had entered the crime scene. He also pointed out that there is no general perception that crime scenes are frequently visited by police officers who are not authorised to do so, who are also dishonest and reckless enough to lie about it when confronted with a verified fingerprint identification. In fact, in the history of fingerprinting he knows of no such case but knows of a number of cases of misidentification. So although a misidentification is a very rare event, he would consider a police officer entering a crime scene and lying about it to be equally rare. Since there is no independent evidence that this happened, the QA officer thinks that misidentification is the more likely explanation, but he cannot be sure.

How can it be that in case A the rarity of misidentification provides reasonable certainty that the officer is lying, but in case B any perceived certainty is an illusion?

The answer is that the very low likelihood of misidentification only applies to identifications chosen AT RANDOM (or selected out of a small group which was chosen at random). In case A, we KNOW that a police officer who did not have authorisation visited the crime scene. This gives a fair chance before the fingerprint analysis (prior probability), that a fingerprint will have been deposited by the officer. The latent prints in the crime scene became the limited group of potential sources from which the officer might be found, and this was BEFORE the identifications were announced. This is random selection (and this is the way it works in normal crime-led cases). In case B the identification was chosen and the whole case initiated AS A RESULT OF THE DENIAL. With regard to accusing someone of lying this is non-random selection.

The prosecutor’s fallacy in a nutshell is: You cannot use the rarity or low likelihood of something as proof of guilt if the suspect was selected because of that thing out of a very large pool.

The rarity of misidentification is the only proof that the police officer was lying. In case A the identification was selected out of the small group which had a connection with the act of wrongdoing, which we know happened. In case B it was selected out of an unlimited pool of police elimination identifications at all crime scenes, and the act of wrongdoing is hypothetical, having to be inferred from the identification. If misidentifications happen but are very rare, the police officer in case A is almost certainly guilty but the police officer in case B is likely to be innocent.

The limited number of prints and suspects in case A means that if the identification represents a “good match” rather than full individualisation (where only one person in the world could have deposited the print) it would be unlikely to alter the outcome. Engineers would call this a “fault tolerant” situation. A misidentification in case A is extremely unlikely so it can reasonably be ruled out.

Whereas case A was created by a physical act, case B was created because someone could not explain why their fingerprint was found. The vast number of police eliminations throughout the world every day means that to avoid a misidentification creating a case B, every fingerprint department must be fully individualising every elimination identification. So a misidentification is not unlikely if we don’t specify when or where it might happen. It is also feasible for a police officer, somewhere, to enter a crime scene unobserved, deposit a fingerprint and deny it. The balance of these two probabilities does not have the overwhelming odds in favour of the officer depositing the fingerprint that happens in case A. Of course, if the fingerprint department was grossly incompetent or corrupt then the police officer is even more likely to be innocent.

Another situation that is not fault tolerant and requires full individualisation is database searching. However, a random match from a database search would be very unlikely to hit on someone who has a feasible connection with the crime. I am sure that this must help to trap most random misidentifications. Once again, the pre-existing crime provides a degree of safety.

In case B the Chief Inspector agreed with the QA officer that an independent opinion on the identification should be obtained before proceeding. At this point a Texan was spotted outside the crime scene. “Are you a sharpshooter?” asked the Chief Inspector. “Nope, fingerprint expert.” came the reply, so he was invited in to look at the prints. The Texan told the inspector that he excludes the police officer as being the donor of the fingerprint in question. The Chief Inspector and QA officer decided that other identifications should be checked starting with the murder case that was under investigation when all this started - and a lot of trouble was avoided.

Back to previous document