Welcome to the eleventh installment of Predictions of the 10th Justice, brought to you by FantasySCOTUS.net. The league has approximately 4,000 members, who have made predictions on all cases currently pending before the Supreme Court.
Is the Supreme Court partisan? Although many perceive the Justices as mere political aids, do the numbers support this assertion? In this installment of 10th Justice, we will be exploring the perception that the Supreme Court Justices make their decisions based on partisan identity. For this purpose, we will be using a standardized majority ratio technique and confidence intervals to analyze Union Pacific Railroad, Salazar, Christian Legal Society, and McDonald.
Epidemiology, the study of factors affecting health and illness of populations, has yielded a standardized mortality ratio (SMR) which is used to make comparisons of mortality rates between different groups. In a slightly morbid twist on the Supreme Court, our FantasySCOTUS SMR is derived from the Epidemiology SMR.
The Epidemiology SMR is calculated by taking a proportion of observed number of deaths within the population of interest (i.e. 100 per 1,000) and dividing that by the expected number of deaths. The expected number of deaths is calculated by multiplying the population of interest by a proportion derived from a larger population (i.e. 10,000 per 100,000). A ratio that equal to 1 (as would be the case with the above numbers) implies that the population of interest is exactly like the larger population. A ratio greater than 1 implies a higher risk of death. A ratio less less than 1 implies lower risk of death. Additionally, SMR can be further refined using confidence intervals to determine if the deviation from 1 is statistically significant.
This SMR provides a method to test whether or not users perceive the Court as dominated by conservative ideology. Using the standardized mortality ratio, we can easily use the predictions to generate the observed number of times each Justice was in the majority. We will calculate the expected times each Justice voted in the majority by multiplying the total number of predictions by the percentage of affirm/reverse. To test the conservative majority, the higher percentage will be used to determine conservative Justices’ expectations, and the lower percentage for the liberal Justices.
Based on the information gained from other posts, Kennedy will be excluded from this process due to his tendency to be the deciding vote. His high majority count would greatly exceed any other Justice’s count. We calculate the confidence intervals with formula involving both the observed and expected values, which are obtained from our predictions.
From an interpretive standpoint, the assumptions concerning the affirm/reverse percentages are still applicable. Using the pure affirm/reverse percentages to determine expected values relies on the assumption that if a ratio is higher than 1 at a statistically significant level, then that particular Justice is less constrained by ideology, and is more likely to join in the majority beyond their ideological camp. However, if the ratio is less than 1 at a statistically significant level, then that Justice is more likely to vote in the minority, and support their own ideological camp.
Results, after the jump.
In Citizens United, 62% of the 908 predictions determined that the Court would reverse the lower courts decision. This yields an expected value of 563 (.62*908) for the expected majority, and conversely 245 (.38 * 908) for the expected minority. The next observed values, the number of times in the majority, are: Roberts (706), Scalia (660), Thomas (667), Alito (650) and Stevens (298), Ginsburg (274), Breyer (270), and Sotomayor (269).
For these cases, the confidence interval will be calculated by mulitplying 1.96 * SQRT(Obvserved)/Expected. 1.96, the standard score taken from a standardized normal distribution, provides for a confidence level of .95. That is, 95% of values on a standard normal curve fall between -1.96 and 1.96,
Citizens United falls close, but not quite along partisan lines. Each Justices’ ratio was different from 1 at a statistically significant level. By having a significant ratio above one, the conservative Justices were considered to be less constricted by ideology, and possibly join in a broader majority. However, the liberal Justices were more likely to refrain from joining the conservatives in the majority with a ratio significantly less than 1.
In this chart, the expected values are listed next to the category of Justices, while the observed values are listed next to each Justice. In this case, the ratios for the conservative Justices contain 1 are well within their confidence interval. In this case, the ideology of the conservative Justices could be interpreted to be a primary factor in predictions about their vote on this case. However, each of the liberal Justices had a ratio significantly above 1, indicating that they had strong possibility of joining the conservative Justices in the majority. Predictions did not rely too heavily on ideology to determine the outcome of this case, and given the unanimous outcome of this case, they were correct in that analysis. High SMRs indicates that the decision is more likely to reach across ideological camps for a large majority. This idea is exemplified by Union Pacific in light of the unanimous opinion the Supreme Court handed down.
This case presents the paradigmatic statistics for showing that ideology is an important factor in how SCOTUS watchers determine how the court will vote. In this case, both conservative and liberal Justices have ratios with 1 well within their confidence intervals (except for Sotomayor and Breyer). Both Breyer and Sotomayor have ratios significantly above 1, but only by .02 and .01 respectively (after subtracting their interval from the ratio to get the bottom limit). While Sotomayor’s deviation can be easily explained by users uncertain of her tendencies as a Justice, Breyer’s deviation could be due to any number of issues such as user unfamiliarity with Breyer’s views on the issue at question or some technicality on which Breyer differs from a pure “liberal” view on the issue.
Our results for Christian Legal Society are interesting because the liberal Justices have more predictions of being in the majority than the conservative Justices do, except for Roberts, who has the most. Before discarding the predictions for a conservative majority for this cases, it is important to note that 1 was well within the confidence interval of all Justice’s ratios. One of the main driving factors behind the wide intervals for the Justices is the number of total predictions. While only 20 predictions are required to use the standardized majority ratio techniques we are working with, more predictions improve the reliability of our results. The implication of sample size for this case is that while the absolute numbers show a possibility, they are not sufficient to make reliable predictions. This also makes it difficult to detect any trends or deviations in those results.
In McDonald v. Chicago, the predictions clearly rely on ideology for determining the behavior of Justices. Most of the conservative Justices have ratios with a confidence interval that includes 1, but Scalia has a statistically significant deviation upwards, indicating that he could possibly join with a broader majority. One possible explanation behind this deviation is that Scalia might balk at using the Privileges or Immunities clause to extend the 2nd Amendment to the states, out of fear of opening Pandora’s Box.
On the liberal side, all Justices are within the expected range. Ginsburg is very close to having a statistically significant deviation downward, indicating that within the liberal ideology, she is even less likely to join in a majority. Overall, the results of this case show that users use ideology to inform their predictions on the outcome of hot-button cases such as McDonald.
These statistics are based on observer predictions, and can only give information about how observers perceive the behavior of Supreme Court Justices. While this data can make strong indications of perception, whether or not the perceptions are correct depends on the actions of the Justice’s themselves, as revealed through their decisions. By having a better understanding of the perceptions however, we can use these statistics as benchmarks to test the outcomes of cases against. This capability furthers our understanding of how Justices make decisions and improve our modeling of that behavior. However, it is always important to remember that models, both quantitative and qualitative, have their limitations as shown by one of the cases we analyzed.
Many thanks to Corey Carpenter for his excellent help with this post.