New Scientist – August 17, 2019

8 | New Scientist | 17 August 2019

ARTIFICIAL intelligence is

predicting the outcome of court

cases concerning human rights

violations, presaging the growing

role that AI is expected to take in

the courtroom.

From next year, AI judges are set

to be introduced in small claims

courts in Estonia to moderate

disputes of less than €7000. In the

US, algorithms are already used in

sentencing decisions. Lawyers also

use AIs to analyse texts to answer

legal questions and identify

relevant past court rulings.

Nikolaos Aletras at the

University of Sheffield, UK, and

his colleagues used four AIs to

analyse 11,500 past cases from

the European Court of Human

Rights (ECHR).

The AIs were supplied with

information like the allegations

in cases, relevant domestic law

and the applicant and defendant’s

arguments. They were then

tasked with predicting whether

the court had ruled that a human

rights violation had occurred.

The team measured the AIs’

performance using a statistical

measure called F1, which accounts

for overall accuracy as well as the

number of false positives and

negatives. The best-performing

algorithm scored 82 out of 100

(arxiv. org/ abs/1906.02059).

“Lawyers could use these

models to estimate the likelihood

of winning a case,” says Aletras.

That could help them adjust a legal

strategy or decide whether cases

should be prosecuted in court.

But if an algorithm was used

to make rulings for real in future,

it is possible that case materials

could be written in a way that

games the AI and engineers a

certain outcome, says Aletras.

“We need to make sure our

systems can be trusted and

can be explainable,” he says.

The AIs were also tested on

their ability to predict each case’s

importance and which human

rights had been violated. The

ECHR assigns importance ratings

on a 4-point scale: 4 is relatively

unimportant, 1 is a case that

establishes legal precedent.

The most successful AI’s

predictions of case importance

were about half a point off, on

average. But the AIs were much

less successful at predicting

which, if any, human rights

had been violated. This could be

because some violations didn’t

crop up in their training material.

The team also tested whether

bias arose from the AIs associating

certain case characteristics – for

example, a particular country –

with human rights violations.

To do this, they removed proper

nouns such as location names.

They found the F1 score of the

best-performing AI only dropped

from 82 to 80.

A barrier to using AI judges to

make real court rulings is that they

don’t generate an explanation for

their decisions, says Kevin Ashley

at the University of Pittsburgh,

Pennsylvania. “These methods

are not analysing the cases the

way a lawyer would.”  ❚

€

Value of disputes that could be

decided by AI judges in Estonia

JEA

N-F

RA

NC

OIS

BA

DIA

S/A

FP/

GE

TT

Y^ IM

AG

ES

News

AIs analysed cases from

the European Court of

Human Rights

Health

Chlamydia vaccine

shown to be safe in

first test in people

THE first human trial of a new

vaccine has shown that it is safe

and that it triggers an immune

response against the bacteria

that cause chlamydia.

The team behind the vaccine is

planning a larger trial to find out if

it can protect against the sexually

transmitted infection. If it does, the

vaccine could help prevent some

of the 131 million new chlamydia

infections seen annually worldwide.

Sonya Abraham at Imperial

College London and her team used

a vaccine that has already been

tested in animals including mice,

guinea pigs and primates. The drug,

known as CTH522, is essentially

a human-made version of a protein

found on chlamydia bacteria and

prompts the immune system to

mount a response to the bacteria.

The team combined this with one of

two adjuvants, chemicals that boost

the response of the immune system.

Thirty-five women aged 19 to

45 were given either the vaccine

with one of the adjuvants or a

placebo. Over the course of five

months, each person was given

three injections in their arm, as

well as two doses of nasal spray

in both nostrils.

Every woman who received the

vaccine demonstrated an immune

response, say the researchers

behind the work (The Lancet

Infectious Diseases, doi.org/c9df).

While the volunteers did report

some side effects, these were

nothing more serious than those

associated with existing vaccines,

such as soreness at the site of

injection, a fever or a headache.

It is too soon to tell whether or

not the drug will protect against

chlamydia. The team is currently

preparing for a larger phase 2

trial, which will help determine

which doses might be effective.

The researchers plan to develop a

dosing schedule to match that of the

HPV vaccine given in adolescence.  ❚

Jessica Hamzelou

These spherical

bacteria are

behind the

131 million

annual

chlamydia

infections seen

SP worldwide

L

Automation

Donna Lu

AI judges make good calls on human

rights violations but could be gamed