Adam Liptak reviews a Brennan Center study that shows that contrary to what we may think, the Roberts Court is not pro-free speech. In short, the Court is considering fewer free speech cases each term, and in fewer cases the First Amendment claim wins. Liptak took the (unusual?) step of having Lee Epstein and Jeff Segal review the Brennan Center Study.

Ms. Youn’s study was posted on the blog of the American Constitution Society, a liberal legal group. At the request of The New York Times, two scholars — Lee Epstein, who teaches law and political science at the University of Southern California, and Jeffrey A. Segal, a political scientist at Stony Brook University — examined the data Ms. Youn relied on and confirmed the essence of her empirical conclusions. Professors Epstein and Segal also added several refinements.

What I found interesting was the opposition of some scholars to characterize SCOTUS cases empirically.

Floyd Abrams, the prominent First Amendment lawyer, said he was unimpressed by the new findings. “Statistics cannot tell the story of the willingness of a court to defend free expression,” he said. “Cases do. It is unpopular speech, distasteful speech, that most requires First Amendment protection, and on that score, no prior Supreme Court has been as protective as this.” . . .

David L. Hudson Jr., a scholar at the First Amendment Center at Vanderbilt University, said the studies lacked nuance by, for instance, treating every decision as equally important. His criticism illuminated a gap between the two disciplines used to assess the Supreme Court: political science codes and counts, while law weighs and analyzes.

And the gap between coding and analysis is one I hope to bridge. Teaching computers to understand jurisprudence, rather than simply coding cases, is the key to crunching the law.

Update: Serious pwnage from Mark Savignac. It aint even close.

The study asserts that the difference is statistically significant (as does this more detailed report on the study), and so it might tell us something interesting if the study were comparing comparable things. But it is not; the study’s comparisons are invalid. It can provide no hint of how the Roberts Court would have dealt with the Warren Court’s cases, or vice versa; nor does it show which Court was more pro-speech for its day, relative to the contemporary median American. The problem is that the assertion that the numbers show a “statistically significant” shortfall in support for free speech by the Roberts Court relies on an implicit assumption that the Roberts Court and the other Courts are deciding comparable cases–either the exact same cases under the same conditions (as in standardized testing), or else comparable distributions of cases drawn from the same pool. Yet that assumption is obviously false.

Supreme Courts do not decide a randomly selected distribution of free speech cases, nor do different Courts decide the same or analogous cases or even pick and decide their cases under similar circumstances. The differences are tremendous, and I see no way of controlling for them (though this study didn’t try). The primary factors are the Court’s ability to pick and choose which cases it will hear, the effect of pre-existing precedents on the cases that come before a given Court, and the effect of the lower courts on the cases that end up being decided by the Supreme Court.