A central concern raised on this blog is about how rankings like this may be used by Deans and by other administrators. Administrators like data. Numbers are great because they purport to be measures of something. If there is a ranking, that can enable a Dean or Provost to gauge how a school or a faculty member is doing based on how high they rank, and whether their rankings go up or down.
Professor Leiter takes issue with my comment that: "Brian Leiter's rankings are not a true measure of 'scholarly impact,'" since after all he was only trying to measure "scholarly impact in legal scholarship." Because Leiter's rankings don't include scholarly impact outside the Westlaw JLR database, there is a risk that incentives will be created by administrators concerned about rankings. My first post on this put it this way:
Once something is seen as "countable," and both law schools and individual faculty are identified based on how their numbers line up...you’ve seen the next step before. Law schools decide they need to move up the rankings. Perhaps they reward faculty based on how far up their numbers are. All of a sudden, that leading, peer-reviewed journal, outside the Westlaw JLR database, is no longer a great prize on your C.V. Your efforts on that were a waste of your law school’s resources.
I pointed out that leading legal historians are not included in the rankings because they do not have full-time appointments in a law school, and I also noted the way the rankings skew in favor of larger sub-fields (like American legal history as compared to medievalists and European or Asian legal history). This point would not matter if Professor Leiter was only ranking schools. Once he turns to rankings within fields, it is important to emphasize the incompleteness of the lists. Appointments committees interested in a major hire in legal history, for example, would miss Christopher Tomlins at the American Bar Association and Henrik Hartog in the History Department at Princeton, and also leading legal historians at law schools in fields other than American legal history. Again, Leiter's lists will be used for purposes other than informing law students, especially rankings of individual faculty members.
Professor Leiter and I appear to differ on the kind of scholarly impact law professors should aspire to. I do think that we all should aspire to be leaders on our campuses not just in our own schools, in the academy not just the legal academy, and in the world not just in the United States.
The comments were not open on his blog, which is why one needs to turn to some unlikely places to start a conversation. Much attention will be paid to Professor Leiter's rankings, so it is important to discuss them. Others have taken up the topic here and here.
Update: Brian Tamanaha weighs in, with a post on the nature of citation practices in U.S. law reviews.
Another update: The discussion continues here and here. An earlier post I'd missed, raising questions, is here. Building on Leiter's data is a post by Paul Butler on Blackprof, regarding highly cited minority scholars, and Jack Chin.
And more: Rankings are like crack and we're addicted, says Daniel Solove.
