party’s counsel often does not know
what is in its own client’s “cards.”
10
Judicial Blessing
While the technology used by predictive
coding programs is not necessarily new,
5
its use has recently been blessed by some
courts, which gives credence to the
technique and relief to the practitioners
who may want to employ it. Andrew J.
Peck, a federal magistrate judge in the
Southern District of New York, is a high-profile proponent of predictive coding,
author of an often cited pro-predictive
coding article,
6 and the first judge to rule
in favor of the use of predictive coding
in litigation. In Da Silva Moore v. Publicis
Groupe,
7 he stated that wary practitioners
waiting for judicial approval now had it.
8
Da Silva Moore I contains a thorough
review of the predictive coding process
and an analysis of the pros and cons of
its use. In setting the stage for approval of
the use of predictive coding, Peck noted:
In approving the use of predictive
coding software, the court opined:
“Computer-assisted review appears to
be better than the available alternatives,
and thus should be used in appropriate
cases. While this Court recognizes
that computer-assisted review is
not perfect, the Federal Rules of Civil
Procedure do not require perfection.”
11
Peck commented that his decision
to allow computer-assisted review of
documents was “relatively easy—the
parties agreed to its use (although
disagreed about how best to implement
such review).”
12 The plaintiff in Da
Silva Moore I, however, asserted
that she did not agree to the use of
predictive coding and objected to
the magistrate’s rulings, bringing the
matter before the District Court for the
Southern District of New York.
13 The
District Court upheld Peck’s decision,
adopting his rulings because “they
are well reasoned and they consider
the potential advantages and pitfalls
of the predictive coding software.”
14
coding in general, the plaintiff said:
“Instead of having human beings look
at and select documents, the Landow
defendants want a computer program
to make the selections for them.”21
In overruling the objection and allowing
the use of a predictive coding process,
the Loudoun County Circuit Court
made clear that its ruling was “without
prejudice to a receiving party raising with
the court an issue as to completeness
or the contents of the production or the
ongoing use of predictive coding.”
22
The objective of review in ediscovery
is to identify as many relevant
documents as possible, while
reviewing as few non-relevant
documents as possible. Recall is
the fraction of relevant documents
identified during a review; precision
is the fraction of identified documents
that are relevant. Thus, recall is a
measure of completeness, while
precision is a measure of accuracy or
correctness. The goal is for the review
method to result in higher recall and
higher precision than another review
method, at a cost proportionate
to the “value” of the case.
9
Transparent Discovery
With the use of predictive coding come
more communications and disclosures
among counsel. “An important aspect
of cooperation is transparency in
the discovery process.”
23 The party
employing predictive coding must
disclose the keywords and Boolean
connectors used to pull the initial round
of documents, as well as those initial
documents and why they were judged
to be responsive or nonresponsive
(and thus how the software was further
trained to pull responsive documents).
24
In comparing the predictive coding
process to the use of keywords,
which seems to be a more universally
acceptable method to cull massive
ESI productions, the court stated:
A mere three days before the Southern
District affirmed Moore I, the first state
court in the country also approved
the use of predictive coding in
litigation. In Global Aerospace Inc.
v. Landow Aviation, L.P.,
15 a Virginia
trial court overruled a plaintiff’s
objections and ordered the use of
predictive coding for purposes of
processing and producing ESI.
16
The defendant proposed predictive
coding because it had more than 250
gigabytes of ESI and claimed that a
typical attorney review would cost
millions of dollars.
17 The defendant
estimated that predictive coding would
identify at least 75 percent of the relevant
documents and, in conjunction with a
“statistically sound validation protocol,”
would represent the “reasonable inquiry”
required under state discovery rules.
18
This aspect of cooperation was the
subject of dispute in a more recent case
permitting the use of predictive coding.
In National Day Laborer Organizing
Network v. U.S. Immigration and
Customs Enforcement Agency,
25 the
plaintiff sued to compel production of
certain records by various governmental
agencies under the Freedom of
Information Act (FOIA).
26 Among
other issues, the plaintiff asserted
that the agencies did not establish
that the searches they conducted
were adequate to find the documents
responsive to the FOIA request.
27
Keywords have a place in production
of ESI—indeed, the parties here used
keyword searches (with Boolean
connectors) to find documents for
the expanded seed set to train the
predictive coding software. In too
many cases, however, the way lawyers
choose keywords is the equivalent
of the child’s game of “Go Fish.” The
requesting party guesses which
keywords might produce evidence to
support its case without having much,
if any, knowledge of the responding
party’s “cards” (i.e., the terminology
used by the responding party’s
custodians). Indeed, the responding
The plaintiff argued that a 75 percent
rate was unacceptable, because the
defendant has an obligation to produce
“all” responsive documents located upon
reasonable inquiry.
19 Further, the plaintiff
argued that the defendant’s estimates
of 250 gigabytes of information were
artificially inflated.
20 Summing up the
controversy over the use of predictive
The agencies did not disclose the
search terms used to cull responsive
documents and, instead, relied on their
custodians to be “trusted to run effective
searches of their own files, a skill that
most office workers employ on a daily
basis.”
28 District Judge Shira A. Scheindlin
found it impossible to evaluate
the adequacy of the government’s
electronic search for records without
knowing the search terms it used:
29
In earlier times, custodians and
searchers were responsible for
Journal of
Corporate
Renewal
October
2012