I've alredy made up my mind. Change my mind! - Confirmation bias in legal setting

Confirmation bias is one of the most widely discussed cognitive limitations that can easily interfere with the reasoned decisions of any judge or arbitrator. It is perilous since the legal decisionmaker's mind is closed (or at least less than fully open) when presented with proofs or arguments of the parties.

The most famous and reported cognitive bias, the' confirmation bias', gained wide recognition when the high-profile case of the Madrid bomber came on the surface and got enormous media coverage. This case earned its place in the history of forensic science as the textbook case of fingerprint misidentification and forensic science errors.

On March 11, 2004, a coordinated series of bombs exploded in four commuter trains in Madrid. The explosions killed 191 people, wounded 1800 others, and set a full-scale international investigation into motion. Based on a fingerprint from a bag containing detonating devices, the FBI identified Brandon Mayfield, an American Muslim, who had been on an FBI watch list. Following standard protocol, several FBI fingerprint examiners independently concluded that the fingerprint was that of Mayfield. After being arrested, Mayfield requested a fingerprint examiner on the defense team examine the prints. That fingerprint examiner concurred with the judgment that the print was Mayfield's. Soon after that, however, the Spanish authorities matched the prints to the real Madrid bomber, an Algerian national named Ouhnane Daoud. Following an internal investigation at the FBI,' confirmation bias' was identified as a contributing factor to the erroneous identification in the case.

Confirmation bias (also known as coherence-based reasoning or focal bias) is the tendency to seek out, interpret, and create new evidence in ways that validate one's preexisting beliefs. Modern psychology confirms the existence of this pervasive psychological phenomenon whereby an individual preferentially favors information that supports an original hypothesis and ignores or dismisses data that may disconfirm the favored hypothesis.

If people can be jaded by existing beliefs, it does not seem a far-fetched idea that this tunnel vision phenomenon has potential consequences in legal settings:

Indeed, several studies have pointed out that this cognitive flaw can easily set into motion and corrupt the judgments of judges, arbitrators, and lawyers. Confirmation bias can interfere with judges and arbitrators when they hear and evaluate evidence brought before them in court.

Specifically, judges might favor evidence confirming their prior hypotheses and

disregarding or discounting evidence

that does not correspond with their previous assumptions.

The Wason selection test is the most widely known demonstration to showcase the confirmation bias in an empirical setting. In a typical version of this experiment, we show people the following four cards:

We inform the participants that each card displays a letter on one side and a color on the other side. We propose the question to the subjects which cards they would need to turn over to test the proposition "if a card has an odd digit on one side, it is colored red on the other side?". The correct answer to this question is cards "3" and the brown card since the red color on the other side of the "3" card or an odd digit on the other side of the brown card would confute the statement. The statement, however, says nothing about what is on the other side of a card indicating an even number, which means that turning over the "8" card would eventuate nothing. Along the same lines, the statement indicates nothing about what is on the other side of a brown card, so turning over the red card is unnecessary.

Interestingly,

people tend to perform pretty poorly on this test:

fewer than 5 percent correctly answer "3" and brown card.

80% of people choose to turn over the "3" card and the red card.

The scientific literature suggests that most people choose to turn over the red card rather than the brown card due to confirmation bias. Finding an odd digit on the back of the red card would constitute confirmatory but not conclusive evidence. Whereas finding an odd digit on the back of the brown card would constitute contradictory evidence which is also conclusive. Therefore, it seems that in this task, people overwhelmingly seek confirmatory evidence rather than contradictory evidence at the cost of non-conclusiveness.

Arbitrators are not an exception to the above cognitive limitation. Helm et al. (2016) tested how commercial arbitrators' responded to the Wason card selection task and concluded that they performed poorly: none of the subjects (0 out of 44 arbitrators) responded correctly. The researchers administered a variation of the Wason card selection task to determine whether arbitrators are also susceptible to the confirmation bias in a more natural setting. This task involved a context that arbitrators might confront when presiding over an arbitration.

The arbitrators were asked to resolve a fictitious dispute in which a woman alleged gender discrimination in the employment context. The employee alleged that "male managers never promote female employees to a software engineer position." The employer in the case withheld the personnel files of four employees with qualifications similar to the claimant's, who had been promoted to the software engineer position. The four withheld personnel files were:

A. The personnel file of an employee whose gender is unknown, whom a male supervisor recently promoted to the position of software engineer;

B. The personnel file of an employee whose gender is unknown, whom a female supervisor recently promoted to the position of software engineer;

C. The personnel file of a male employee who was recently promoted to the position of software engineer by a supervisor whose gender is unknown; or

D. The personnel file of a female employee who was recently promoted to the software engineer position by a supervisor whose gender is unknown.

The researchers asked the arbitrators to identify the file(s) that must be examined to establish if the claimant's allegation that "male managers never promote female employees to a software engineer position." is likely to be true or false. The arbitrators were also warned not to select any more files than were necessary.

Using the logic of the Wason selection test, the correct answer is that files A and D should be reviewed. File A is necessary to falsify the hypothesis if the employee was a female. File D is necessary because it would falsify the hypothesis if the supervisor were male. Files B and C are not necessary because they cannot apply to a situation involving a female employee and a male manager.

The arbitrators reviewing this modified version of the Wason problem performed much better than those who reviewed the original form of the Wason card selection task:

Nineteen percent (8 out of 43) of the subjects responded correctly.

Nevertheless, the arbitrators

tended to insist on too many files.

Of the 35 arbitrators who answered incorrectly, 45 percent (16 arbitrators) chose too many files (ten chose all four and six chose files A, C, and D); 31 percent (11 arbitrators) chose the wrong two files (seven chose files C and D, one chose files B and D, two chose files A and B, and one chose file A and C); and only 23 percent (8) chose too few (five chose only file A, and three chose only file D).

Based on the above results, the most striking observation is that only one arbitrator followed the classic response confirmation bias pattern and chose options A and C. This, after all, suggests that arbitrators were not frequently subject to confirmation bias in the most traditional sense (failing to pick D and unnecessarily picking C). On the flip side, arbitrators displayed a tendency to insist on way more evidence than needed to decide in the case, primarily by picking file C (24 arbitrators picked file C that is a classic confirmation bias mistake), compared to 13 who picked the other unnecessary file (that was file B).

The arbitrators intuitively insisted on more evidence than needed,

suggesting they are likely to be

ineffective discovery and admissibility gatekeepers.

The error among arbitrators seems to be gathering too much information rather than looking at the wrong information. The researchers believe that the driving force behind this is still the classic confirmatory tendency induced by the Wason task structure.

Whatever type of confirmation bias is set in motion during the arbitrators' cognitive tasks, the most apparent practical implication of the arbitrators' susceptibility to this cognitive limitation is that it might increase the cost and duration of arbitration. In addition, research indicates that obtaining what looks like confirmatory evidence can bolster the decisionmaker's confidence even though the additional information might not be relevant. Therefore, the confirmation bias could distort an arbitrators' decision-making.

---

Cheat sheet for busy lawyers

Title

Are Arbitrators Human?

Authors

Year of publication

Theoretical / Conceptual framework

Research question(s) / Hypotheses

Methodology

Analysis / Results