Do androids dream about biased judges? – Self-fulfilling prophecies in litigation prediction by AI

A potentially valuable feature of AI applied in the legal field is identifying insightful patterns of how specific courts and judges operate that are not visible to the human cognition. However, we should never forget that the relationship between humans and algorithms is always reflexive, and we can easily corrupt our quantitative-based prediction algorithms with biases.

It does not break ground to anyone who did not spend the recent years under a rock that technology is rapidly changing the legal professions. We all experience the effect of how technological advancements in AI and Legal Tech enhance the efficiency of the legal practice, particularly the access to adjudication.

One of the most exciting domains of artificial legal intelligence is quantitative legal prediction (let's call it QLP).

Prediction constitutes a fundamental part of the legal practice, and AI and Legal Tech significantly change how the law is predicted. Legal automation has recently shifted towards adopting data-based models and data-driven machine-learning techniques in constructing legal predictions rather than seeking legal conclusions based on logic. The fundamental reason behind that is that the law is difficult to capture via logic-based systems, and it is pretty challenging to mimic how a lawyer is supposed to think. On the other hand, it seems that we can draw meaningful conclusions by using algorithms that lack any semantic understanding of the law. Thus, quantitative-based legal prediction is gaining increasing traction in the legal market.

Quantitative legal prediction employs statistical means to predict various forms of outcome:

- possible costs of the litigation,

- length of the procedure,

- which arguments are likely to lead to success and which are not, and

- ultimately, the outcome of the litigation matter (losing/winning or conviction/acquittal).

The prediction model is based on a dataset of previous cases, which the algorithm uses as an input to learn the correlations between case features and target outcomes. It is noteworthy that correlations are created between specific data available in court cases (such as who the judge is, the words used, which precedents are cited, the legal domain of the dispute, etc.) and the outcome of those cases. Again, we stress that such prediction models largely dismiss legal causality and do not even try to mimic a lawyer's prediction mechanism.

These tools might provide valuable input for significant litigation decisions, such as whether to litigate or not and whether to seek settlement or drop specific claims.

Many scholars envision that if these models' prediction accuracy continues to rise significantly, they may be adopted by courts for solving legal disputes, which has always been the sorcerer’s stone in legal tech.

Even though the application of quantitative legal prediction is manifold, we will disregard the otherwise exciting topic of QLP-based dispute resolution mechanisms in this blog post. Instead, we will shed some light on one specific feature of these systems and particular concerns the adoption of such systems raises.

Hunters of the judges' minds

A potentially valuable feature of the QLP systems is that, based on the data about how the court/judge has ruled in the past, they can identify insightful patterns regarding how specific courts and judges operate that otherwise would be invisible for human cognition:

- which case law a particular judge likes to refer to,

- how the judge tends to decide in non-obvious cases concerning a particular subject matter,

- whether certain lawyers tend to win with a particular judge,

- what are the types of arguments that the judge tends to accept, etc.

This feature enables the litigating parties to conclude what a judge might prefer or not prefer within the adjudication process. This can provide additional transparency and knowledge that enables the litigating parties to make more informed decisions and manage legal processes more efficiently.

In this regard, QLPs operate as super-recognizers of human behavior:

QLP can identify tendencies of a judge that would otherwise be outside of the cognitive abilities of lawyers. QLP thus enables lawyers to overcome some of the cognitive limitations of the human brain to make better strategic decisions and tailor the legal arguments for the particular judge. The QLP might influence the litigation participants’ decisions such as whether to litigate, when to litigate, which motions to submit, which might be the most advantageous venue for the trial.

So let’s think about two aspects of the potential concerns of applying these so-called super-recognizers in the litigation process:

Forum shopping with biases - Is the application of exploiting the biases ethical?

Notably, a growing amount of commercialized predictive justice models seek to identify specific tendencies of courts or particular judges. There is an evident positive effect in such capability – it can identify adjudication patterns otherwise inaccessible to human cognition and thus considerably enhance the transparency of adjudication, which is an essential value of due process.

Simultaneously, there are some concerns that such utilization of QLP raises. One pretty well-reported issue is that such application of QLP might undermine the fairness of the litigation process.

If you are a regular visitor of our blog, you are already aware that rendering a judgment, in its nature, is a cognitive process. Such cognitive process is primarily influenced by the judge’s attitudes, heuristics and biases, and many other extraneous factors constituting the primary reason for the inconsistencies within the adjudication.

In light of the notion that a large set of biases influences the judge’s decision, it is not a too far-fetched idea that QLP tools that recognize unique patterns of judges might, to an extent, enable the litigation parties to exploit these biases. Even though we want to keep the biases as far as possible from influencing the case outcome, whenever a QLP tool connects a strategic decision of the litigation party with the judge's identified bias, it gives an edge that isn’t rooted in the legal rules.

QLP tools are applied to identify specific personal tendencies that a judge display in order to gain an advantage within the adjudication process.

Choosing among several venues of the trial

might also cause a not desirable forum shopping effect,

which is considered unethical and unfair.

In the case that patterns of judges become increasingly visible and the litigating parties exploit such biases, QLP tools might introduce a potential extra element of unfairness.

It is noteworthy that for the above reasons, in 2019, the French legislator introduced a rule that bans the use of QLP for outlining the judges' patterns and predicting case outcomes based on prior behavior of the judge.

Is AI bias-free?

A seemingly evident advantage of AI over the human brain might be that cognitive illusions do not limit it and distort its decisions; thus, it seems to be superior in decision-making. Nevertheless, there is growing concern that AI systems learn human biases and exaggerate them.

One way bias can creep into algorithms is

when AI systems learn to make decisions based on data, including biased human decisions.

In this sense, the often mentioned saying that „An AI system can be as good as the quality of its input data” proves correct. In case the algorithm is based on biased human decisions, then inevitably, the AI will also reflect such biases, and most likely, it will even exaggerate these biases by holding them true for its future decisions our outcome predictions.

From this perspective, the AI is a child who learns its biases from its parents through social learning. If we put it into social development and learning context, we should consider the AI as a super-intelligent but inherently naive agent.

Let’s see an example to make the situation abundantly clear.

Suppose we are the bank's general manager and we have an ongoing litigation matter with one of our retail clients. We reach a point when we have to make a strategic decision to settle or keep fighting. We turn to our super-recognizers for advice and ask them to search for patterns in the judgments of the umpire to whom our case was assigned. Our A.I. concludes that our judge displays a tendency to be client-friendly in his judgments and assigns 57% influence to this variable on the outcome. Let’s assume that based on this prediction, we decided to make a strategic decision, and we dropped our case.

Even though our decision seems reasonable, if we put it in a broader perspective, it is profoundly flawed for the following reasons:

First, let’s assume that our super recognizers’ conclusion was totally on point, and they pinned the judge's bias right: this judge is indeed biased against banks when it comes to bank vs. client disputes. In this case, and considering that ultimately we (as the bank) decided not to fight for our case and indirectly against this bias, our AI has just perpetuated such favor given to clients.

Notably, by dropping our case, we have just put one more brick into the wall of the justification mechanism. When we (or someone else) will find ourselves in a similar decisive situation with the same judge, the biased pattern will be even more visible to our super-recognizers, and they will assign more significant weight to this variable when computing next time. Thus, ultimately, we will find ourselves in a never-ending cycle of our AI reinforcing the given bias.

Another thing that should require our attention is that

the above sequence transforms the prediction of our AI into self-fulfilling prophecies

and

the predictions drive our actions.

When we ultimately decide to rely on opaque predictions, our perception of the law shifts from making legal-critical decisions to a mere risk management process. This approach tends to forget the arguable character of the law: the ultimate purpose of the rule of law is not only about creating absolute foreseeability of what is permitted or not permitted. Instead, the central tenet of law is „letting everything that is arguable be argued” as Neil McCormick (2005) pointed it out.

This arguable character is an essential element of the rule of law: the abstract laws are transformed into rules that govern specific circumstances through the legal debate. By accepting that the law has an arguable character, we perceive individuals as morally capable agents who can fight for their right to participate in how a particular behavior is to be governed by the law.

On the other side of the coin, when we buy ourselves into the idea that our case is already decided by a biased creature, which disregard our legal arguments, we profoundly compromise our image as being an agent who has a right to participate in the process.

If we make our decisions in this spirit, our perception will ultimately be proved to be right, and the prediction of our super-recognizers will become self-fulfilling prophecies.

The AI’s expectation will lead to its own confirmation.

It is also necessary to remind ourselves that even though the algorithms can outline significant patterns specific to judges, such patterns don’t necessarily refer to biases. Even if the correlation in our example is accurate, there is no possibility to determine whether it is just a coincidence or the judge is indeed biased against banks. If we proceed with this idea, we will find ourselves in the same reinforcement cycle of biases outlined above.

When the AI needs a therapist - Treat your robot!

Since AI gains enormous traction, one of the most reported concerns is if humans will lose their jobs. McKinsey has already designed an online test "Will a Robot Take My Job?" that allows you to enter your profession and calculate the chances of being replaced by robots.

One of the professionals highly warned to prepare for disruption is the psychologist, and the notion is that machines can do most of a psychologist’s job. Reading these predictions, we might have this vision of the future:

Nevertheless, if we accept that AI can learn our biases, it is not a too far-fetched idea that

algorithms might also need some sort of cognitive training

to eliminate the embedded human and societal biases (let’s call it debiasing training).

As already discussed above, AI should be treated as a naive child, and in this sense, we have to show some parental responsibility when it comes to cognitive flaws such as biases.

Thus, psychologists might not be in danger of losing their jobs, after all. We don’t know what the future holds. And AI don’t know it, either.

Maybe this:

(Source of the cover picture: www.alamy.com)

---

Cheat sheet for busy lawyers

Title

Quantitative legal prediction and the rule of law

Author(s)

Year of publication

Theoretical/ Conceptual Framework

Research Question(s)/ Hypotheses

Methodology

Analysis/Results