BEST PRACTICES 339
D. Empirically Established ProtocolIn the research environment, the continual testing of a
method of forensically feasible, ground-truth data empirically
establishes the protocol for using the method in casework. First,
a level of accuracy can be set: for instance, the method won’t be
used forensically until it reaches a certain accuracy level, such
as eighty percent, ninety percent, or ninety-five percent. Second,
the experiments are designed to control for variables such as the
quantity of data, required number of authors, required number
and types of linguistic features, and the required number and
types of individual testing techniques that are combined in the
method.
For the quantity of data, an important issue to resolve is the
minimum number of words, sentences, or texts required for the
method to obtain a certain level of accuracy.^15 For the number of
authors, a method may require a minimum of two, five, or
twenty-five suspects to obtain a certain level of accuracy. As in
other pattern recognition techniques in forensic science, the
number and type of features required for identification or
elimination is established empirically by controlling the variable
in a series of validation tests related to specific accuracy rates.^16
If fifteen linguistic features from syntactic analysis yield eighty-
two percent accuracy, the next experiment will test sixteen,
seventeen and so forth until the desired accuracy level is
achieved. Those experiments empirically establish the number
and type of features required for the method to obtain a specific
accuracy level. Likewise, if measurement or feature selection
techniques can be combined in a method (combining syntax with
(^15) See Chaski, Author Identification, supra note 6, at 503.
(^16) Carole E. Chaski, Empirical Evaluations of Language-Based Author
Identification Techniques, 8 INT’L J. SPEECH LANGUAGE & L. 1 (2001)
[hereinafter Chaski, Empirical Evaluations]; Carole E. Chaski, Who’s at the
Keyboard? Authorship Attribution in Digital Evidence Investigations, INT’L J.
DIGITAL EVIDENCE, Spring 2005, at 1 [hereinafter Chaski, Who’s at the
Keyboard?]; Carole E. Chaski, Presentation at the Eight Biennial Conference
on Forensic Linguistics/Language and Law: Empirically Testing the
Uniqueness of Aggregated Stylemarkers (July 14, 2007) [hereinafter Chaski,
Empirically Testing].