Journal of Corporate Renewal

party’s counsel often does not know what is in its own client’s “cards.” 10

Judicial Blessing

While the technology used by predictive coding programs is not necessarily new, 5 its use has recently been blessed by some courts, which gives credence to the technique and relief to the practitioners who may want to employ it. Andrew J.

Peck, a federal magistrate judge in the Southern District of New York, is a high-profile proponent of predictive coding, author of an often cited pro-predictive coding article, 6 and the first judge to rule in favor of the use of predictive coding in litigation. In Da Silva Moore v. Publicis Groupe, 7 he stated that wary practitioners waiting for judicial approval now had it. 8

Da Silva Moore I contains a thorough
review of the predictive coding process
and an analysis of the pros and cons of
its use. In setting the stage for approval of
the use of predictive coding, Peck noted:
In approving the use of predictive
coding software, the court opined:
“Computer-assisted review appears to
be better than the available alternatives,
and thus should be used in appropriate
cases. While this Court recognizes
that computer-assisted review is
not perfect, the Federal Rules of Civil
Procedure do not require perfection.” 11

Peck commented that his decision to allow computer-assisted review of documents was “relatively easy—the parties agreed to its use (although disagreed about how best to implement such review).” 12 The plaintiff in Da Silva Moore I, however, asserted that she did not agree to the use of predictive coding and objected to the magistrate’s rulings, bringing the matter before the District Court for the Southern District of New York. 13 The District Court upheld Peck’s decision, adopting his rulings because “they are well reasoned and they consider the potential advantages and pitfalls of the predictive coding software.” 14

coding in general, the plaintiff said: “Instead of having human beings look at and select documents, the Landow defendants want a computer program to make the selections for them.”21

In overruling the objection and allowing the use of a predictive coding process, the Loudoun County Circuit Court made clear that its ruling was “without prejudice to a receiving party raising with the court an issue as to completeness or the contents of the production or the ongoing use of predictive coding.” 22

The objective of review in ediscovery is to identify as many relevant documents as possible, while reviewing as few non-relevant documents as possible. Recall is the fraction of relevant documents identified during a review; precision is the fraction of identified documents that are relevant. Thus, recall is a measure of completeness, while precision is a measure of accuracy or correctness. The goal is for the review method to result in higher recall and higher precision than another review method, at a cost proportionate to the “value” of the case. 9

Transparent Discovery With the use of predictive coding come more communications and disclosures among counsel. “An important aspect of cooperation is transparency in the discovery process.” 23 The party employing predictive coding must disclose the keywords and Boolean connectors used to pull the initial round of documents, as well as those initial documents and why they were judged to be responsive or nonresponsive (and thus how the software was further trained to pull responsive documents). 24

In comparing the predictive coding
process to the use of keywords,
which seems to be a more universally
acceptable method to cull massive
ESI productions, the court stated:
A mere three days before the Southern
District affirmed Moore I, the first state
court in the country also approved
the use of predictive coding in
litigation. In Global Aerospace Inc.
v. Landow Aviation, L.P., 15 a Virginia
trial court overruled a plaintiff’s
objections and ordered the use of
predictive coding for purposes of
processing and producing ESI. 16

The defendant proposed predictive coding because it had more than 250 gigabytes of ESI and claimed that a typical attorney review would cost millions of dollars. 17 The defendant estimated that predictive coding would identify at least 75 percent of the relevant documents and, in conjunction with a “statistically sound validation protocol,” would represent the “reasonable inquiry” required under state discovery rules. 18

This aspect of cooperation was the subject of dispute in a more recent case permitting the use of predictive coding. In National Day Laborer Organizing Network v. U.S. Immigration and Customs Enforcement Agency, 25 the plaintiff sued to compel production of certain records by various governmental agencies under the Freedom of Information Act (FOIA). 26 Among other issues, the plaintiff asserted that the agencies did not establish that the searches they conducted were adequate to find the documents responsive to the FOIA request. 27

Keywords have a place in production
of ESI—indeed, the parties here used
keyword searches (with Boolean
connectors) to find documents for
the expanded seed set to train the
predictive coding software. In too
many cases, however, the way lawyers
choose keywords is the equivalent
of the child’s game of “Go Fish.” The
requesting party guesses which
keywords might produce evidence to
support its case without having much,
if any, knowledge of the responding
party’s “cards” (i.e., the terminology
used by the responding party’s
custodians). Indeed, the responding
The plaintiff argued that a 75 percent
rate was unacceptable, because the
defendant has an obligation to produce
“all” responsive documents located upon
reasonable inquiry. 19 Further, the plaintiff
argued that the defendant’s estimates
of 250 gigabytes of information were
artificially inflated. 20 Summing up the
controversy over the use of predictive
The agencies did not disclose the
search terms used to cull responsive
documents and, instead, relied on their
custodians to be “trusted to run effective
searches of their own files, a skill that
most office workers employ on a daily
basis.” 28 District Judge Shira A. Scheindlin
found it impossible to evaluate
the adequacy of the government’s
electronic search for records without
knowing the search terms it used: 29