Science_-_6_March_2020

(singke) #1

SCIENCE sciencemag.org


are also often not evidence-based, being in-
stead informed by politics or the subjective
collective wisdom of professional authorities.
Even when evidence is available from one of
the few dozen legal practice RCTs conducted
in the United States to date, and even when
that evidence fails to support the use of inter-
ventions that are costlier than their alterna-
tives, it has not been used to change practice
( 1 ). When aspects of legal practice are rooted
in philosophical and normative claims about
rights and interests, empirical study may
have little to add. But when interventions aim
to achieve certain instrumental outcomes, as
is often the case, evidence is critical ( 1 , 2 ).


RANDOMIZATION
One commonly articulated objection to le-
gal practice RCTs is that it is unethical to
randomize legal interventions in a way that
temporarily precludes individualized profes-
sional judgment. For example, in objecting
to a proposed A2JL study in which tenants
facing eviction litigation would be random-
ized to participate in a mandatory pretrial
settlement conference or proceed without
one, local legal services providers argued on
the basis of their anecdotal experience that
settlement conferences assist in keeping low-
income tenants in their homes, despite rea-
son to question that assumption. Similarly,
organizations running charitable bail funds
have refused to discuss RCTs, even when they
lack sufficient resources to assist all eligible
arrestees, arguing that because they “already
know” the difference it makes to have bail
posted, it is wrong to randomize “lifesaving”
legal “treatments.”
These examples are indicative, in part, of
the legal profession’s lagging understanding
of foundational principles of research design
and ethics. Similar objections have long been


considered resolved in the realm of clini-
cal research, where one might also be con-
cerned that randomized participants could
be harmed by the denial of some perceived
benefit in a way that violates the physician’s
professional obligation to provide individual-
ized care ( 7 ). However, an RCT can be ethi-
cal—and perhaps ethically mandatory—when
there is sufficient professional uncertainty, or
“equipoise,” regarding the safety or efficacy of
the experimental intervention compared to
the standard of care. In these cases, random-
ization violates no professional obligation be-
cause it will not result in practitioners know-
ingly exposing participants to harm.
In addition to the culture shift needed
in law to adopt this view, it is also essential
to challenge faulty moral intuitions exhib-
ited in the legal profession and elsewhere
that it is permissible to roll out an untested
intervention across an entire population,
but not to randomize population groups to
rigorously examine the intervention’s ef-
fect ( 8 ). Moreover, because resolving extant
uncertainty is essential to the professional’s
fiduciary obligation to provide the best care
or services, it would be logically incoherent
to refuse to subject unproven approaches to
study unless or until evidence is produced to
challenge their effectiveness.
Even when there is relative certainty as
to which intervention to prefer in ideal cir-
cumstances, practical constraints may ren-
der that intervention neither attainable nor
sustainable for some or all of the relevant
population ( 5 , 9 ). In such cases, researchers
should work to resolve “policy equipoise” ( 5 ),
developing evidence needed to inform open
questions, such as how to optimally allocate
the intervention and whether a less resource-
intensive approach might suffice. Doing this
work through RCTs is methodologically ideal

and in many cases ethically permissible.
However, one further consideration particu-
larly relevant to legal practice interventions is
that the ethical permissibility of an RCT that
fails to provide all subjects with the preferred
intervention depends on the absence of legal
or ethical entitlement to that intervention.
Although patients may have ethical entitle-
ments, they have few legal entitlements to
medical services, at least in the United States.
Those affected by legal practice interventions
may have stronger claims.
Consider an offer of representation by a
lawyer. This is a federal constitutional en-
titlement in the United States only for seri-
ous criminal charges and a few other narrow
matters; in these contexts, it would be legally
impermissible to withhold the offer from
some individuals, even for research purposes.
However, there are other circumstances in
which an individual may be expected to
benefit from an offer of legal representation
without being legally entitled to it. In the face
of scarcity, a legal clinic could reasonably
randomize similarly situated individuals to
receive either an offer of traditional repre-
sentation, an offer of self-help resources only,
or no offer at all to research how to maximize
benefits across the population. Individuals
randomized without entitlement have either
not been harmed, because they might not
have received the benefit outside the RCT, or
have not been unreasonably harmed, because
even if they would have received the benefit,
they lack a greater claim to it than others fac-
ing similar need.

VOLUNTARY CONSENT
Other objections to legal practice RCTs are
based on concerns that RCTs may be inher-
ently coercive. Perceived or actual coercive-
ness might stem from the involvement of

QUESTION STUDY DESIGN OBJECTIONS RESPONSE


Does availability of risk
assessment scores improve
judicial bail decisions?


Randomize arrestees as to whether
bail judge receives their risk
assessment score

Too much at stake
(freedom v. incarceration)
Voluntary consent from arrestees
not practicable

High stakes demand empirical study
Equipoise as to utility of risk assessment scores
Either practice (scores or no scores) is permissible
without consent outside research; no entitlement to
either policy

Can legal professionals
effectively allocate scarce
resources for greatest benefit?


Randomize criminal defendants to
have public defender decide whether
to assign a social worker or allocate
services by some other process

Unfair to defendants who would
have been assigned a social worker
but for the study

Scarcity of social workers
Equipoise as to quality of attorney allocation decisions
Either practice (public defender triage or
other approach) permissible outside research;
no entitlement to either a particular triage process
or social work assistance

Does a combative litigation
style produce superior client
results compared to a
facilitative, deal-making style?


Randomize whether legal aid
attorneys in summary eviction
cases litigate aggressively (e.g.,
demand jury trials, file motions,
make ambitious settlement
demands) or nonaggressively

Contrary to responsibility of
attorneys to use their best
professional judgment about ideal
strategy for each individual case

Equipoise as to ideal litigation strategy
Either practice permissible outside research;
both strategies within the “standard of care”

6 MARCH 2020 • VOL 367 ISSUE 6482 1079

Examples of legal practice research questions, RCT designs, objections, and replies


Published by AAAS
Free download pdf