Nate Silver Has Validated The Era Of Big Data–And What It Means For The Law

November 7th, 2012

Perhaps one of the few silver livings of last night–which didn’t exactly turn out as I would have liked–is silver. Nate Silver that is.

For the past several months, I have largely cringed at the constant barrage aimed at Nate Silver and 538. People whom I really respect consistently criticized every little aspect of his model, charging that the samplings were under or over-representative, or that some factors were weighed too heavily, or not heavily enough. I chuckled aloud when pundits, working on nothing more than their “gut” feeling of what would happen, scoffed at Silver and made predictions about how races would turn out.

Silver’s big-data models accurately called all 50 states (with the possible exception of the nail-biter in Flordia). The pundits were almost all wrong. These pundits should not have any credibility–unless their main focus is to bloviate and make stuff up to help their candidate gain momentum, in which case they should take a serious pay cut.

So what does this mean for law?

The era of big data is here. In the past, many attorneys I spoke with cast doubt that computer models could ever predict something so unknowable as a court decision. Instead, attorneys seek to make predictions based on their gut instincts, knowledge about how courts have acted in the past, and personal feelings towards the case. However, the knowledge of these experts is always greatly limited–and far less than they’d care to admit. When they’re right, they’re praised. When they’re wrong, no one remembers.

When presenting this topic, I have framed it in terms of Hayek’s local knowledge problem. Any one attorney, or even a firm of attorneys, can’t possibly know everything. And even assuming there was perfect data (impossibly), it would be impossible for any one attorney to synthesize all of that knowledge. In contrast, algorithmic models serve as clearinghouses for data–information that no one person (or central planner) could ever amass. As one writer for Tech Crunch put it “Silver’s analysis, and statistical models generally, factor in more data points than even the most knowledgeable political insider could possibly juggle in their working memory.”

My early work with FantasySCOTUS is but a crude facsimile of the models Nate Silver has put together. It is simply impossible–and won’t happen–that we will ever have daily polling data about how courts decide (the closest we will probably ever get was data about the Health Care Cases). Even if we have hundreds or thousands of predictions about Supreme Court cases on FantasySCOTUS, there will never been nearly enough interest to offer predictions for the lower courts–where most cases are decided.

In order to advance to the next level, our focus must turn from a crowdsourcing approach–what 538 and Nate Silver does spectacularly–to a pure data crunching mode. This is what I call Assisted Decision Making. Looking at data about how the courts operate, how parties litigate, and how issues are resolved reveals insights into how courts will decide cases in the future.

I have given talks on Assisted Decision Making at LawTechCamp and at Michigan State, and will be presenting this topic on Friday at 1 at Stanford Law School (Sign up here).

Going forward, our predictions about how courts will decide cases will only be limited based on the amount of data we have, and the sophistication of algorithms–two fronts I am tackling in both the academic and entrepreneurial context.

Update: Dick Morris explained that he predicted a Romney landslide to help the cmapaign out:

“I think that there was a period of time when the Romney campaign was falling apart, people were not optimistic, nobody thought there was a chance of victory and I felt that it was my duty at that point to go out and say what I said.”