A law firm in Philadelphia is launching a “Supreme Court Project” that “attempts to build on and extend empirical work explaining the United States Supreme Court’s decision making and its review of the thirteen United States Courts of Appeals.” Ultimately they want to predict cases.

But, their methodology is so severely lacking.

Phase II.  While our Phase I work identified variations in circuit court reversal rates, it did not attempt to explain the reasons for variations.  In Phase II we will apply econometric techniques to test various hypotheses that explain these variations.

One hypothesis, for example, is that the political party (or appointing President) of a Justice and the members of the Circuit Court panel may, in part, explain the Justice’s decision to affirm or reverse; a Justice and a judge appointed by the same President or a President from the same party may, all else equal, lead to a lower probability of reversal.  Other hypotheses include:  whether a Justice is less likely to reverse (1) a Court of Appeals over which s/he is the Circuit Justice; (2) a Judge who attended the same law school as the Justice, or (3) a Judge who is the same gender as the Justice.

Clearly, a few caveats are in order.  First, a statistical correlation is only just that and needs to be supplemented by an underlying rationale; a statistical finding that left handed Justices reverse left handed Court of Appeals Judges more often than right handed ones, for example, would likely be spurious.  Nevertheless, data driven inquiries such as these (and others) are important in revealing patterns that actually occur.  We welcome interested readers to e-mail us with suggestions of plausible theories that could be tested with available data.

Second, the tests of these hypotheses should be part of a broader statistical model of Supreme Court Justice decision making.  In that way, we anticipate that this work will build on the pioneering work of, among others, the Washington University Supreme Court Forecasting Project, which applied statistical techniques to predict Supreme Court Justice voting behavior and Court decisions. We hope to extend that work by, among other things, updating their results for the Roberts Court and employing additional explanatory variables (including those relating to the Courts of Appeals.)

Looking at what school the Justice went to? Really?

Also, scoring cases as affirm or reverse is tough. I know that. I have no idea what they mean by the “shadow.”

 For each of these cases, the Supreme Court ruled on a legal issue addressed not just in the case on appeal, but also one or more “shadow decisions” (i.e., court of appeals decisions that have ruled on a legal issue that comprises the circuit split).  Any measure of reversals and affirmances that does not account for these shadow decisions is therefore incomplete and potentially misleading.

We, therefore, have established a “full” measure of the circuit courts’ reversal and affirmance rates, taking into account both decisions on direct appeal and these shadow decisions.  This measure, we believe, provides a more robust and more accurate view of the relationship between the United States Courts of Appeals and the United States Supreme Court.

I’ll keep my eye on this. At least law firms are considering these issues.