ATL reviews a panel at LegalTech (which I hope to attend in the future) about Computer Assisted Document Review:
The panel, which consisted of Wachtell Lipton counsel Maura Grossman, Magistrate Judge Andrew Peck(S.D.N.Y.), and Jackson Lewis partner Ralph Losey began by examining all the problems with human review.
Grossman explained that humans — a.k.a. contract attorneys locked in a room — usually only find about 70 percent of responsive documents. They also turn out a significant number of false positives.
“It makes us uncomfortable,” she said. “It makes me uncomfortable. So we pretend it’s perfect.”
Losey and Judge Peck agree. In Monday’s Quote of the Day, we mentioned their strong opinions about older keyword search technology.
“We are going to price ourselves out of business as a court system if we don’t change,” Judge Peck said.
For those new to the party, Grossman gave a thorough explanation of what computer-assisted review actually means.
Basically, the computer comes up with document review seed sets and gives them to humans, who say yes or no, or tell the computer to systematically alter what it finds. Grossman compared it to visiting the optometrist.
“A or B. B or C.” You continue choosing the better option until they are so similar, she said, that you can no longer tell a difference.
Grossman said it can bring the discovery down to one-tenth the price of what e-discovery often is. And when/if errors happen, the types of errors are different than human errors. Technology makes systematic errors, she said. The problems can be fixed by telling the computer to change its formula for finding documents. Humans make random mistakes, which are harder to consistently eliminate.
During the question-and-answer period at the end of the presentation, someone in the crowd said, “I’m very disappointed in this presentation. We were promised a debate, but we have three for the machines and zero votes for the humans.”
At first it seemed like a joke, and the panelists were waiting for a punchline. The man’s view was very different from what everyone had heard onstage thus far. And he had legitimate concerns.
In a Tuesday morning panel that included in-house counsel from DuPont legal and the Dow Chemical Company, as well as Ari Kaplan and representatives from Dell and FTI technology, the general consensus was that despite all the hype, attorneys are still nervous about the “embryonic nature” of predictive coding. Concerns are less about the cost, and more about the (entirely reasonable) fear of f***ing up a major case. Legal departments are more comfortable with using it for internal investigations.
And it puts lawyers out of a job.