Updated Version of FantasySCOTUS: Crowdsourcing a Prediction Market for the Supreme Court

June 10th, 2011

I have just uploaded a new version of FantasySCOTUS: Crowdsourcing a Prediction Market for the Supreme Court to SSRN. This version is significantly improved from the last version. It is currently under consideration by a few top publications, so we should be able to announce the placement soon. Here is the abstract:

Every year the Supreme Court of the United States captivates the minds and curiosity of millions of Americans—yet the inner-workings of the Court are not fully transparent. The Court, without explanation, decides only the cases it wishes. They deliberate and assign authorship in private. The Justices hear oral arguments, and without notice, issue an opinion months later. They sometimes offer enigmatic clues during oral arguments through their questions. Between arguments and the day the Court issues an opinion, the outcome of a case is essentially a mystery. Sometimes the outcome falls along predictable lines; other times the outcome is a complete surprise.

Court-watchers frequently make predictions about the cases in articles, on blogs, and elsewhere. Individually, some may be right, some may be wrong. Until recently, there was not a way to pool together this collective wisdom, and aggregate ex ante predictions for all cases pending before the United States Supreme Court.

Now there is such a tool. FantasySCOTUS.net from the Harlan Institute is the Internet’s premier Supreme Court Fantasy League, and the first crowdsourced prediction market for jurisprudential speculation. During the October 2009 Supreme Court term, 5,000 members made over 11,000 predictions for all 81 cases decided. Based on this data, FantasySCOTUS predicted the outcome in more than fifty percent of the cases decided, and the top-ranked predictors forecasted 75% of the cases correctly. This essay explores the wisdom of the crowds in this prediction market and assesses the accuracy of FantasySCOTUS.

FantasySCOTUS is only two years old, but the implications, and applications of this information market are intriguing. First, from a jurisprudential perspective, FantasySCOTUS illuminates public perceptions of how the Supreme Court works as an institution. Specifically, it serves as a comprehensive polling device to provide an honest, albeit unscientific, survey of how a large sample size of Court watchers view the Justices, and their legal realist ideological proclivities, particularly in 5-4 cases. If FantasySCOTUS can accurately reduce the Justices to nothing more than a conservative, or liberal vote, that may have broader implications to the rule of law, and objective, detached standards of judging.

From a practical perspective, with more accurate future versions of FantasySCOTUS, attorneys will be able to rely on this program to assist them with litigation decisions involving cases pending before the Supreme Court. As our understanding of judicial behavior improves—perhaps through scanning all filings in PACER—and the program can shift from a pure crowdsourcing technique to a commoditized super cruncher information service, a prediction engine can be created for lower courts, vastly increasing the value for practicing attorneys.