You are here
When Numbers Fail Justice
In many trials, especially in the US, probability and statistics have been used to convict or acquit individuals and institutions accused of murder, theft, espionnage, discrimination, etc. Why then does this use of probability and statistics, although intended to serve justice, often produce the opposite result?
Leila Schneps:1 There are many ways to misuse mathematics, and not only through calculation errors. Very often, it is a matter of failing to understand the meaning of perfectly accurate mathematical results. The charge of gender bias leveled against the University of California, Berkeley is a perfect illustration of what is called “Simpson’s paradox.” On realizing that the institution's acceptance rate was 45% for male applicants compared with only 35% for females—even though they fared better on average—the administrators decided to look into each department's statistics in order to identify the source of the problem. They were quite surprised to find out that all disciplines accepted as many women as men, if not more. The reason for this is very simple: in the “small” departments like art history, women applicants far outnumber men, while large departments like engineering get a lot more applications from male students. Assuming, for example, that out of 100 art history applicants comprising 80 women and 20 men, 16 women and 4 men (or 20% of each group) are admitted, while out of 70 male and 30 female applicants in engineering, 35 men and 15 women are taken in (or 50% of each group), the gender breakdown is perfectly equal. Yet the two groups combined make up a total of 200 applicants, composed of 110 women and 90 men. Of these, the university accepts 39 men (43%) and only 31 women (28%), giving the false impression of a real injustice! This illustrates how accurate figures can lead to erroneous interpretations.
Another striking example is the false accusations of Munchausen Syndrome by proxy…
L. S.: Munchausen syndrome is a mental disorder that induces the patients to hurt themselves in order to get attention and care from medical personnel. By extension, the expression Munchausen syndrome by proxy is used to describe people who, for the same reason, deliberately harm someone who cannot protect or express themselves—often an infant. Even though such cases do exist, this diagnosis at one stage became so “fashionable” in the UK that hundreds of children, perhaps thousands, were taken away from their parents, sometimes to die in foster care, when they were quite simply suffering from a real disease that was not properly diagnosed. Worse, at the height of this “trend,” dozens of mothers whose babies died suddenly were convicted and jailed for murder. This is what happened to Sally Clark, who spent years in prison after being charged with the murder of two of her children. As in most trials of this type, the prosecution’s case was based mainly on the testimony of the doctor who “invented” the famous by-proxy syndrome. He maintained that sudden infant death was a totally random event with a very slim probability, which he estimated at one in 8,000.
Were these probabilities unrealistic?
L. S.: No, they were correct—the problem lay in how they were used. In order to understand it properly, it is necessary to review the methods of calculating probability. Take the case of a couple who want to have two children. If the woman brings her first pregnancy to term, the probability of her giving birth to a boy is 1 in 2. If she has two children in succession, the probability of the couple having two boys is ½ x ½, or ¼. But multiplying the odds like this is valid only if the two events are independent, as is the case for two separate pregnancies. However, if the woman finds out from an ultrasound scan that she is pregnant with identical twins, the two births are no longer independent events, and the probability of her having two sons (or two daughters) is now ½!
Where did the doctor in the Clark case go wrong?
L. S.: Based on the probability of one in 8,000 babies dying of sudden infant death syndrome, the “expert” physician deemed that the odds of the same mother losing two children in this way was therefore 1/8000 multiplied by 1/8000. His apparently logical conclusion was that there was only one “chance” in 64 million for one woman to be struck by two successive cot deaths, and that, if this were the case, the most reasonable interpretation was that it must have been a double murder. Yet this line of reasoning, and the calculation behind it, ignored a well-known fact: the probability of sudden infant death in a given family depends on various factors, including the mother’s age, the presence of smokers in the household, etc. This means that two sudden cot deaths occurring in the same family are not independent events, and therefore the probability of each individual event cannot be simply multiplied to calculate the risk of two successive events. In addition, it was false to assert that there were only two possibilities, that either both children died of sudden infant death syndrome or that their mother had killed them. In reality, there was a third possibility, by far the most probable, namely that one or both children had died of a disease that the doctors had failed to detect, either before death or during the autopsy. And that is exactly what the re-examination of medical records revealed: the second child was suffering from a severe infection at the time of his death.
Are these miscarriages of justice due to deliberate misrepresentation, or rather to ignorance, whether on the part of the accusers, the defenders or the jury?
L. S.: Fortunately, intentional misrepresentation is quite rare. But people involved in trials often use or interpret calculations and figures in a way that supports the conclusion they have already reached, thus falling into the trap of confirmation bias. For this reason, I believe that there should always be a trained, objective observer on hand to monitor the use of mathematics in the courtroom.
In and out of court, there is also such a thing as “wrong mathematical intuition” due to cognitive bias. What are the most common “false intuitions” regarding probability?
L. S.: Let me give you my favorite example, which drives everyone crazy, including mathematicians and even probability experts. Suppose you strike up a conversation with someone who tells you that he has two children, including a son who was born on a Tuesday. The question is: what is the probability of his other child being a girl?
Most people’s immediate response would be 50%. Knowing that one of the children is a boy does not affect the gender probability of the other child in any way, does it? But that’s wrong: the probability of the other being a girl is actually 2/3. With the exception of twins, families with two children can be divided into four types: boy-boy, boy-girl, girl-boy and girl-girl. We already know that this is not a girl-girl family (since there is at least one boy), so it has to be either boy-boy, boy-girl or girl-boy. Since all three types are equally likely, there are two chances out of three that the other child is a girl. And that's not taking into account the fact that the boy was born on a Tuesday. It seems incredible, even absurd, to think that this could change the probability of having a girl, but it does. Making a list of all boy-boy plus weekday possibilities gives 49 possibilities. Doing the same for boy-girl and girl-boy families makes a total of 147 possibilities. After eliminating cases that do not include a boy born on a Tuesday, there are exactly 27 scenarios left, including 13 in which the other child is a boy and 14 in which it is a girl—hence a 14/27 probability that the other child is a girl… Enough to make you pull your hair out!
Is the modern-day reverence for numbers a counter effect of the relative “mathematical illiteracy” among legal experts and the general public?
L. S.: There is no doubt about that. I believe that the feeling of not understanding something triggers all sorts of psychological reactions, ranging from dismissive contempt to near-fearful respect. We are especially susceptible to such feelings when someone in a position of authority silences us with an incomprehensible explanation. If we cannot understand what is being said, let alone counter it with logical arguments, it becomes especially difficult to raise an objection. It is simply easier to accept it, but it’s also normal to feel a degree of resentment at the same time. This is truly regrettable, because the math used in trials is no more complex than the one we learn in school—it is actually within everyone’s grasp.
What do you recommend to remedy the situation?
It’s important to develop a sense of (or dare I say a taste for?) numbers, as well as a critical attitude. To that end, the way children are taught should be changed, starting at a very young age. They should learn to make their own numerical estimates as soon as they can count, and to interpret all mathematical statements from a critical point of view. Math classes should include a wide range of questions that have no “right” or “wrong” answer, such as guessing how many gumdrops there are in a jar, estimating the height of nearby buildings or how many square meters of wallpaper it would take to redecorate the classroom. Pupils could later progress to evaluating the size of a crowd of demonstrators on television, or the actual cost of a large-scale operation. This type of exercise, based on familiar phenomena, can make it possible to spot numerous errors or nonsense in the papers. Numbers are like new words added to our vocabulary for describing the world around us—we should make good use of them!
For more on this subject: Math on Trial, co-authored by Leila Schneps and Coralie Colmez
- 1. Mathématiques et interactions des mathématiques, Institut de mathématiques de Jussieu - Paris Rive Gauche (IMJ-PRG) (CNRS / UMPC / Université Paris Diderot).
Keywords
Share this article
Author
Lydia Ben Ytzhak is an independent scientific journalist. Among other assignments, she produces documentaries, scientific columns, and interviews for France Culture, a French radio station.
See also
Comments
Log in, join the CNRS News community