Friday, March 4, 2011

Davies, Rust, and Aft on Supreme Court Statistics

Statistics are important, as every baseball fan knows.  So if you want to produce a set of baseball-type cards featuring not sports figures but Supreme Court Justices, you have to have the stats.  The difficulty of tracking down the data is the focus of a new piece:  Justices at Work, or Not: New Supreme Court Statistics and Old Impediments to Making Them Accurate, by Ross E. Davies, George Mason University School of Law, and Editor of The Green Bag, and Craig D. Rust and Adam Aft, both of George Mason University School of Law.  It appears in The Green Bag, of course:  Green Bag 2d, Vol. 14, No. 2, pp. 217-236, Winter 2011.  Here's the abstract:
There is no such thing as an accurate record of the labors of the 112 (so far) Justices of the Supreme Court. And that makes the development and presentation of accurate "Supreme Court Sluggers" statistics a never-ending adventure. Which is not to say that the official records of the Court’s work – the reports of its decisions and opinions (in the U.S. Reports), and the minutes of its proceedings (in the Court’s Journal) – are dangerously unreliable; indeed, for recent years they seem to be very nearly perfect. Rather, it is to say that imperfections do exist and, roughly speaking, the farther back in time you go, the more incomplete and uneven you will find those records to be. It is a tendency that also holds for records of the Justices’ work in chambers, on circuit, and in the lower federal courts, and for records of their work as members of inferior courts before their elevation to the Court. Unofficial reports and other sources, though occasionally useful for filling gaps, are sometimes faulty and often incomplete.

As the Green Bag digs into ever-older records to produce trading cards of past members of the Court, this problem will get worse. Consider two sets of 19th-century examples (fleshed out below), dealing with two basic questions we must answer about each case included in the "Sluggers" statistics: (1) when was it decided – that is, to which term of Court should statistics about that case be assigned – and (2) who participated – that is, to which Justices should credit of some sort be given for this piece of the Court’s work? In a distressingly large and uncertain number of cases, a just-look-it-up-online researcher will find the wrong answer, and even a look-it-up-in-the-U.S.-Reports researcher will get it wrong some of the time.
Image source.