Papageorge: Predictive coding gaining support in courts

January 29, 2014
Back to TopCommentsE-mailPrintBookmark and Share
Indiana Lawyer Focus

By John Papageorge

For attorneys handling larger litigation matters, electronically stored information – or ESI – has changed the landscape on how discovery is conducted. The days of manually reviewing boxes of client documents in cold warehouses have taken a backseat to wading through thousands – and in some cases millions – of emails and other electronic records.

Attorneys now are faced with the monumental task of collecting, reviewing and producing their own client’s electronic documents while also reviewing the opposing side’s electronic documents. This can lead to uncomfortable conversations with clients regarding the significant cost of the process.

Papageorge Papageorge

Supporters of predictive coding argue it makes the electronic discovery process less costly and less complicated. Predictive coding – a type of technology-assisted review or computer-assisted review – uses computers and algorithms to identify relevant and responsive documents in an automated manner. Unlike manual review, where the review is done by the most junior staff, predictive coding involves a more senior attorney or small team who review a “seed set” of documents for responsiveness.

The predictive coding system then applies the algorithms to identify properties of the seed set to automatically code the documents not reviewed by the attorneys. As the attorney team continues to code or identify additional responsive documents, the computer predicts the responsiveness of the universe of documents. Attorneys must review sample sets of documents coded by the computer and ultimately decide they have satisfied the requirements of Rule 26. In the end, the computer can identify thousands or even millions of responsive documents without the need for manual attorney review, saving clients thousands of dollars.

Predictive coding is gaining support in federal courts. In February 2012, United States Magistrate Judge Andrew J. Peck of United States District Court for the Southern District of New York, a leading authority on predictive coding, approved the use of predictive coding in Moore v. Publicis Groupe based on the following reasons: (1) the parties’ agreement, (2) the vast amount of electronically stored information (over three million documents), (3) the superiority of computer-assisted review to the available alternatives (i.e., linear manual review or keyword searches), (4) the need for cost effectiveness and proportionality under Rule 26, and (5) the transparent process proposed by one of the parties.

While Judge Peck cautioned that computer-assisted review is not appropriate in all cases, he did urge the bar to seriously consider using predictive coding in large-data-volume cases where it may save litigants a significant amount of legal fees for document review.

In April 2013, U.S. District Judge Robert L. Miller Jr. of the Northern District of Indiana issued a ruling in In re Biomet related to a discovery dispute involving keyword searches and predictive coding. Biomet produced millions of documents by initially using keyword searches to narrow the field of documents, followed by predictive coding to identify relevant documents to be produced. Biomet spent millions of dollars on electronic discovery. Plaintiffs objected to Biomet’s search method and argued that predictive coding should have been used from the outset. Plaintiffs wanted Biomet to start the discovery process over.

While Judge Miller stated that predictive coding from the outset might have unearthed additional documents, he ultimately rejected plaintiffs’ request to start over because Biomet, through the use of keyword searches and predictive coding, had satisfied the requirements of Rule 26 and the cost to start over outweighed any benefits of starting over. Judge Miller did tell plaintiffs that if they wanted documents produced via predictive coding only, they could pay the additional costs.

In May 2013, in Gordon v. Kaleida Health, an employment matter pending in the Western District of New York, the parties asked the court to resolve a discovery dispute involving approximately a quarter of a million electronic documents. For more than a year, the parties attempted, without success, to agree on how to achieve a cost-effective review of defendants’ voluminous emails using keyword search methodology. The court was dissatisfied with the lack of progress using keyword searches, and it pointed to predictive coding as another option.

After defendants decided to use predictive coding, the parties then fought over plaintiffs’ use of a conflicted consultant. Plaintiffs also took the position that the parties must negotiate a transparent protocol to guide the use of predictive coding software. Defendants, on the other hand, asserted that courts do not order parties in ESI discovery disputes to agree to specific protocols to facilitate computer-assisted review, based on the general rule that ESI production is within the sound discretion of the producing party. Because defendants ultimately agreed to meet and discuss the production using predictive coding, the court did not rule on the protocol dispute.

As caselaw develops on predictive coding, the issue of transparency related to predictive coding will be one to watch.

For larger litigation matters, predictive coding is here to stay, although the process is ever-changing. Many issues need to be resolved related to predictive coding, but cooperation and transparency will certainly take center stage. And as costs continue to escalate, courts will be faced with ongoing disputes over who pays for what.

Even after these disputes are resolved, parties must find the key documents that can be used to win the case. Maybe searching boxes of documents in a cold warehouse was not so bad after all.•


John Papageorge – – is a partner with Taft Stettinius & Hollister LLP, practicing complex civil litigation with significant experience handling issues related to electronic discovery. He serves as the firm’s e-discovery practice contact. The opinions expressed are those of the author.


Post a comment to this story

We reserve the right to remove any post that we feel is obscene, profane, vulgar, racist, sexually explicit, abusive, or hateful.
You are legally responsible for what you post and your anonymity is not guaranteed.
Posts that insult, defame, threaten, harass or abuse other readers or people mentioned in Indiana Lawyer editorial content are also subject to removal. Please respect the privacy of individuals and refrain from posting personal information.
No solicitations, spamming or advertisements are allowed. Readers may post links to other informational websites that are relevant to the topic at hand, but please do not link to objectionable material.
We may remove messages that are unrelated to the topic, encourage illegal activity, use all capital letters or are unreadable.

Messages that are flagged by readers as objectionable will be reviewed and may or may not be removed. Please do not flag a post simply because you disagree with it.

Sponsored by
Subscribe to Indiana Lawyer
  1. Especially I would like to see all the republican voting patriotic good ole boys to stop and understand that the wars they have been volunteering for all along (especially the past decade at least) have not been for God & Jesus etc no far from it unless you think George Washington's face on the US dollar is god (and we know many do). When I saw the movie about Chris Kyle, I thought wow how many Hoosiers are just like this guy, out there taking orders to do the nasty on the designated bad guys, sometimes bleeding and dying, sometimes just serving and coming home to defend a system that really just views them as reliable cannon fodder. Maybe if the Christians of the red states would stop volunteering for the imperial legions and begin collecting welfare instead of working their butts off, there would be a change in attitude from the haughty professorial overlords that tell us when democracy is allowed and when it isn't. To come home from guarding the borders of the sandbox just to hear if they want the government to protect this country's borders then they are racists and bigots. Well maybe the professorial overlords should gird their own loins for war and fight their own battles in the sandbox. We can see what kind of system this really is from lawsuits like this and we can understand who it really serves. NOT US.... I mean what are all you Hoosiers waving the flag for, the right of the president to start wars of aggression to benefit the Saudis, the right of gay marriage, the right for illegal immigrants to invade our country, and the right of the ACLU to sue over displays of Baby Jesus? The right of the 1 percenters to get richer, the right of zombie banks to use taxpayer money to stay out of bankruptcy? The right of Congress to start a pissing match that could end in WWIII in Ukraine? None of that crud benefits us. We should be like the Amish. You don't have to go far from this farcical lawsuit to find the wise ones, they're in the buggies in the streets not far away....

  2. Moreover, we all know that the well heeled ACLU has a litigation strategy of outspending their adversaries. And, with the help of the legal system well trained in secularism, on top of the genuinely and admittedly secular 1st amendment, they have the strategic high ground. Maybe Christians should begin like the Amish to withdraw their services from the state and the public and become themselves a "people who shall dwell alone" and foster their own kind and let the other individuals and money interests fight it out endlessly in court. I mean, if "the people" don't see how little the state serves their interests, putting Mammon first at nearly every turn, then maybe it is time they wake up and smell the coffee. Maybe all the displays of religiosity by American poohbahs on down the decades have been a mask of piety that concealed their own materialistic inclinations. I know a lot of patriotic Christians don't like that notion but I entertain it more and more all the time.

  3. If I were a judge (and I am not just a humble citizen) I would be inclined to make a finding that there was no real controversy and dismiss them. Do we allow a lawsuit every time someone's feelings are hurt now? It's preposterous. The 1st amendment has become a sword in the hands of those who actually want to suppress religious liberty according to their own backers' conception of how it will serve their own private interests. The state has a duty of impartiality to all citizens to spend its judicial resources wisely and flush these idiotic suits over Nativity Scenes down the toilet where they belong... however as Christians we should welcome them as they are the very sort of persecution that separates the sheep from the wolves.

  4. What about the single mothers trying to protect their children from mentally abusive grandparents who hide who they truly are behind mounds and years of medication and have mentally abused their own children to the point of one being in jail and the other was on drugs. What about trying to keep those children from being subjected to the same abuse they were as a child? I can understand in the instance about the parent losing their right and the grandparent having raised the child previously! But not all circumstances grant this being OKAY! some of us parents are trying to protect our children and yes it is our God given right to make those decisions for our children as adults!! This is not just black and white and I will fight every ounce of this to get denied

  5. Mr Smith the theory of Christian persecution in Indiana has been run by the Indiana Supreme Court and soundly rejected there is no such thing according to those who rule over us. it is a thought crime to think otherwise.