336 JOURNAL OF LAW AND POLICY
A. Litigation IndependenceThe implementation of these best practices involves
litigation-independent development and testing of any method on
a ground-truth dataset that contains forensically feasible data.^6
The researcher-forensic linguist runs experiments to test how
well a method works outside of any litigation. The results are
simply what they are, not favoring one side or the other of a
legal dispute. Such a testing environment frees the researcher
from confirmation bias because the results are simply what they
are and enable the researcher to design the next set of
experiments, as is usual in normal science.
B. Ground-Truth DataFor the testing to be meaningful, the experiments must be
run on ground-truth data.^7 A ground-truth dataset contains
known, verified examples with features relevant to the
experiments being run.^8 For author identification, a ground-truth
dataset typically contains text samples for which the authorship
is known and verified.^9 For writer identification, a ground-truth
dataset typically contains writing samples for which the hand
writer is known and verified.^10 For linguistic profiling, a ground-
truth dataset typically contains linguistic examples for which the
demographics of each author/speaker are known and verified.
It is impossible to calculate a trustworthy accuracy rate if the
researcher does not use ground-truth data. Determining a
method’s accuracy requires comparing the method’s results to
the correct answers. Correct answers can only arise from
ground-truth data, where the dataset is known and verified. If
(^6) Carole E. Chaski, Author Identification in the Forensic Setting, in THE
OXFORD HANDBOOK OF LANGUAGE AND LAW 494, 494–99 (2011)
[hereinafter Chaski, Author Identification].
(^7) Id.
(^8) Id.
(^9) Id.
(^10) Carole E. Chaski & Mark A. Walch, Validation Testing for FLASH ID
on the Chaski Writer Sample Database, PROC. AM. ACAD. FORENSIC SCI.
ANN. MEETING, 2009.