In The Path of the Law, Oliver Wendell Holmes, Jr. wrote “The object of our study, then, is prediction, the prediction of the incidence of the public force through the instrumentality of the courts.” But how are lawyers supposed to predict how the courts will act? As Yogi Berra famously remarked, “It’s tough to make predictions, especially about the future.”

In recent years, the wisdom of the crowds–collective knowledge that can be pooled together to address problems more efficiently and accurately than decisions from individuals–has been used to create so-called prediction markets. These markets can aggregate the opinions of many individuals, pool them together, and generate a more reliable forecast of what will happen. This method, known as crowdsourcing, have accurately predicted the outcome of political elections, winners of the Academy Awards, and other future events.

In November of 2009, I created FantasySCOTUS.net, a Supreme Court Fantasy League. At first, I thought it would just be a fun game–let law nerds compete and make predictions how the Justices will decide cases. The site went viral, and before I knew it, there were nearly 5,000 Court-watchers making predictions. By analyzing this data, I soon realized that I could use crowdsource a prediction market. During the October 2009 Term, FantasySCOTUS predicted the outcome in more than fifty percent of the cases decided, and the top-ranked predictors forecasted 75% of the cases correctly. We currently offer a prediction tracker to monitor real-time predictions for every case pending before the Court.

I (along with my two co-authors, Adam Aft and Corey Carpenter) have posted an essay to SSRN, titled FantasySCOTUS: Crowdsourcing a Prediction Market for the Supreme Court. This essay explores the wisdom of the crowds in this prediction market and assesses the accuracy of FantasySCOTUS. In this essay we compare our results to the fascinating Supreme Court Forecasting Project, which developed a super cruncher algorithm that could predict Supreme Court cases based on factors, such as circuit of origin and ideology, rather than the merits of the case. Also, we test one of Orin Kerr’s ideas, and show that there is in fact a correlation between the amount of media coverage about a case and accuracy of a prediction. Finally, we consider some of the jurisprudential and practical implications. The abstract is below the fold:

Every year the Supreme Court of the United States captivates the minds and curiosity of millions of Americans—yet the inner-workings of the Court are not fully transparent. The Court, without explanation, decides only the cases it wishes. They deliberate and assign authorship in private. The Justices hear oral arguments, and without notice, issue an opinion months later. They sometimes offer enigmatic clues during oral arguments through their questions. Between arguments and the day the Court issues an opinion, the outcome of a case is essentially a mystery. Sometimes the outcome falls along predictable lines; other times the outcome is a complete surprise.

Court-watchers frequently make predictions about the cases in articles, on blogs, and elsewhere. Individually, some may be right, some may be wrong. Until recently, there was not a way to pool together this collective wisdom, and aggregate ex ante predictions for all cases pending before the United States Supreme Court.

Now there is such a tool. FantasySCOTUS.net from the Harlan Institute is the Internet’s premier Supreme Court Fantasy League, and the first crowdsourced prediction market for jurisprudential speculation. During the October 2009 Supreme Court term, 5,000 members made over 11,000 predictions for all 81 cases decided. Based on this data, FantasySCOTUS predicted the outcome in more than fifty percent of the cases decided, and the top-ranked predictors forecasted 75% of the cases correctly. This essay explores the wisdom of the crowds in this prediction market and assesses the accuracy of FantasySCOTUS.

FantasySCOTUS is only two years old, but the implications, and applications of this information market are intriguing. First, from a jurisprudential perspective, FantasySCOTUS illuminates public perceptions of how the Supreme Court works as an institution. Specifically, it serves as a comprehensive polling device to provide an honest, albeit unscientific, survey of how a large sample size of Court watchers view the Justices, and their legal realist ideological proclivities, particularly in 5-4 cases. If FantasySCOTUS can accurately reduce the Justices to nothing more than a conservative, or liberal vote, that may have broader implications to the rule of law, and objective, detached standards of judging.

From a practical perspective, with more accurate future versions of FantasySCOTUS, attorneys will be able to rely on this program to assist them with litigation decisions involving cases pending before the Supreme Court. As our understanding of judicial behavior improves—perhaps through scanning all filings in PACER—and the program can shift from a pure crowdsourcing technique to a commoditized super cruncher information service, a prediction engine can be created for lower courts, vastly increasing the value for practicing attorneys.