Sunday, November 25, 2007

Leiter on Leiter (and the rankings)

Brian Leiter was kind enough to respond to my post on his latest set of rankings. My posts (here and here) speak for themselves, so I will just respond to a few issues Professor Leiter raises. Leiter defends the rankings by emphasizing his purpose: to inform law students about law schools, and he has rightly suggested that his data may be more illuminating than the rankings supplied by US News and World Report. But of course, once out, his rankings are open to being used by others for various other purposes. This happened right after his listing of the top 10 on each law school faculty, when another blogger (who I will not link to so as not to embarrass him) mentioned himself as among the faculty at his school with the greatest scholarly impact. I suspect many of the top ranked will note their status in annual reports to their Dean about their scholarly progress.

A central concern raised on this blog is about how rankings like this may be used by Deans and by other administrators. Administrators like data. Numbers are great because they purport to be measures of something. If there is a ranking, that can enable a Dean or Provost to gauge how a school or a faculty member is doing based on how high they rank, and whether their rankings go up or down.

Professor Leiter takes issue with my comment that: "Brian Leiter's rankings are not a true measure of 'scholarly impact,'" since after all he was only trying to measure "scholarly impact in legal scholarship." Because Leiter's rankings don't include scholarly impact outside the Westlaw JLR database, there is a risk that incentives will be created by administrators concerned about rankings. My first post on this put it this way:

Once something is seen as "countable," and both law schools and individual faculty are identified based on how their numbers line’ve seen the next step before. Law schools decide they need to move up the rankings. Perhaps they reward faculty based on how far up their numbers are. All of a sudden, that leading, peer-reviewed journal, outside the Westlaw JLR database, is no longer a great prize on your C.V. Your efforts on that were a waste of your law school’s resources.

I pointed out that leading legal historians are not included in the rankings because they do not have full-time appointments in a law school, and I also noted the way the rankings skew in favor of larger sub-fields (like American legal history as compared to medievalists and European or Asian legal history). This point would not matter if Professor Leiter was only ranking schools. Once he turns to rankings within fields, it is important to emphasize the incompleteness of the lists. Appointments committees interested in a major hire in legal history, for example, would miss Christopher Tomlins at the American Bar Association and Henrik Hartog in the History Department at Princeton, and also leading legal historians at law schools in fields other than American legal history. Again, Leiter's lists will be used for purposes other than informing law students, especially rankings of individual faculty members.

Professor Leiter and I appear to differ on the kind of scholarly impact law professors should aspire to. I do think that we all should aspire to be leaders on our campuses not just in our own schools, in the academy not just the legal academy, and in the world not just in the United States.

The comments were not open on his blog, which is why one needs to turn to some unlikely places to start a conversation. Much attention will be paid to Professor Leiter's rankings, so it is important to discuss them. Others have taken up the topic here and here.
Update: Brian Tamanaha weighs in, with a post on the nature of citation practices in U.S. law reviews.
Another update
: The discussion continues here and here. An earlier post I'd missed, raising questions, is here. Building on Leiter's data is a post by Paul Butler on Blackprof, regarding highly cited minority scholars, and Jack Chin.
And more
: Rankings are like crack and we're addicted, says Daniel Solove.