Science 6.03.2020

SCIENCE sciencemag.org

are also often not evidence-based, being in-
stead informed by politics or the subjective
collective wisdom of professional authorities.
Even when evidence is available from one of
the few dozen legal practice RCTs conducted
in the United States to date, and even when
that evidence fails to support the use of inter-
ventions that are costlier than their alterna-
tives, it has not been used to change practice
( 1 ). When aspects of legal practice are rooted
in philosophical and normative claims about
rights and interests, empirical study may
have little to add. But when interventions aim
to achieve certain instrumental outcomes, as
is often the case, evidence is critical ( 1 , 2 ).

RANDOMIZATION
One commonly articulated objection to le-
gal practice RCTs is that it is unethical to
randomize legal interventions in a way that
temporarily precludes individualized profes-
sional judgment. For example, in objecting
to a proposed A2JL study in which tenants
facing eviction litigation would be random-
ized to participate in a mandatory pretrial
settlement conference or proceed without
one, local legal services providers argued on
the basis of their anecdotal experience that
settlement conferences assist in keeping low-
income tenants in their homes, despite rea-
son to question that assumption. Similarly,
organizations running charitable bail funds
have refused to discuss RCTs, even when they
lack sufficient resources to assist all eligible
arrestees, arguing that because they “already
know” the difference it makes to have bail
posted, it is wrong to randomize “lifesaving”
legal “treatments.”
These examples are indicative, in part, of
the legal profession’s lagging understanding
of foundational principles of research design
and ethics. Similar objections have long been

considered resolved in the realm of clini-

cal research, where one might also be con-

cerned that randomized participants could

be harmed by the denial of some perceived

benefit in a way that violates the physician’s

professional obligation to provide individual-

ized care ( 7 ). However, an RCT can be ethi-

cal—and perhaps ethically mandatory—when

there is sufficient professional uncertainty, or

“equipoise,” regarding the safety or efficacy of

the experimental intervention compared to

the standard of care. In these cases, random-

ization violates no professional obligation be-

cause it will not result in practitioners know-

ingly exposing participants to harm.

In addition to the culture shift needed

in law to adopt this view, it is also essential

to challenge faulty moral intuitions exhib-

ited in the legal profession and elsewhere

that it is permissible to roll out an untested

intervention across an entire population,

but not to randomize population groups to

rigorously examine the intervention’s ef-

fect ( 8 ). Moreover, because resolving extant

uncertainty is essential to the professional’s

fiduciary obligation to provide the best care

or services, it would be logically incoherent

to refuse to subject unproven approaches to

study unless or until evidence is produced to

challenge their effectiveness.

Even when there is relative certainty as

to which intervention to prefer in ideal cir-

cumstances, practical constraints may ren-

der that intervention neither attainable nor

sustainable for some or all of the relevant

population ( 5 , 9 ). In such cases, researchers

should work to resolve “policy equipoise” ( 5 ),

developing evidence needed to inform open

questions, such as how to optimally allocate

the intervention and whether a less resource-

intensive approach might suffice. Doing this

work through RCTs is methodologically ideal

and in many cases ethically permissible.

However, one further consideration particu-

larly relevant to legal practice interventions is

that the ethical permissibility of an RCT that

fails to provide all subjects with the preferred

intervention depends on the absence of legal

or ethical entitlement to that intervention.

Although patients may have ethical entitle-

ments, they have few legal entitlements to

medical services, at least in the United States.

Those affected by legal practice interventions

may have stronger claims.

Consider an offer of representation by a

lawyer. This is a federal constitutional en-

titlement in the United States only for seri-

ous criminal charges and a few other narrow

matters; in these contexts, it would be legally

impermissible to withhold the offer from

some individuals, even for research purposes.

However, there are other circumstances in

which an individual may be expected to

benefit from an offer of legal representation

without being legally entitled to it. In the face

of scarcity, a legal clinic could reasonably

randomize similarly situated individuals to

receive either an offer of traditional repre-

sentation, an offer of self-help resources only,

or no offer at all to research how to maximize

benefits across the population. Individuals

randomized without entitlement have either

not been harmed, because they might not

have received the benefit outside the RCT, or

have not been unreasonably harmed, because

even if they would have received the benefit,

they lack a greater claim to it than others fac-

ing similar need.

VOLUNTARY CONSENT

Other objections to legal practice RCTs are

based on concerns that RCTs may be inher-

ently coercive. Perceived or actual coercive-

ness might stem from the involvement of

QUESTION STUDY DESIGN OBJECTIONS RESPONSE

Does availability of risk
assessment scores improve
judicial bail decisions?

Randomize arrestees as to whether

bail judge receives their risk

assessment score

Too much at stake

(freedom v. incarceration)

Voluntary consent from arrestees

not practicable

High stakes demand empirical study

Equipoise as to utility of risk assessment scores

Either practice (scores or no scores) is permissible

without consent outside research; no entitlement to

either policy

Can legal professionals
effectively allocate scarce
resources for greatest benefit?

Randomize criminal defendants to

have public defender decide whether

to assign a social worker or allocate

services by some other process

Unfair to defendants who would

have been assigned a social worker

but for the study

Scarcity of social workers

Equipoise as to quality of attorney allocation decisions

Either practice (public defender triage or

other approach) permissible outside research;

no entitlement to either a particular triage process

or social work assistance

Does a combative litigation
style produce superior client
results compared to a
facilitative, deal-making style?

Randomize whether legal aid

attorneys in summary eviction

cases litigate aggressively (e.g.,

demand jury trials, file motions,

make ambitious settlement

demands) or nonaggressively

Contrary to responsibility of

attorneys to use their best

professional judgment about ideal

strategy for each individual case

Equipoise as to ideal litigation strategy

Either practice permissible outside research;

both strategies within the “standard of care”

6 MARCH 2020 • VOL 367 ISSUE 6482 1079

Examples of legal practice research questions, RCT designs, objections, and replies

Published by AAAS