With new rankings based on citations making the rounds, it seems a good time to reiterate a point this blog raised earlier:Brian Leiter's rankings are not a true measure of "scholarly impact," especially in a field like legal history. The study is confined to the Westlaw JLR database which only includes legal publications.
What does this miss? Leading scholars will have an impact that ranges beyond their fields and beyond their nations. But the Westlaw database cannot measure impact beyond the legal academy, and the important global reach of many American legal scholars is not measured. All but a very few journals in the database are U.S.-based.
What does this miss? Leading scholars will have an impact that ranges beyond their fields and beyond their nations. But the Westlaw database cannot measure impact beyond the legal academy, and the important global reach of many American legal scholars is not measured. All but a very few journals in the database are U.S.-based.
The impact of interdisciplinary scholars, in particular, will be under-counted. For serious interdisciplinary scholars, especially J.D./Ph.D.s, the true measure of scholarly success is to be seen as leading figure both within the legal academy and within the Ph.D. field. To further one’s scholarship within the Ph.D. field, an interdisciplinary scholar will publish in the field’s leading peer-reviewed journals. If in the humanities and perhaps social sciences, they will publish books.
This leads to two under-counting problems. First, the Westlaw JLR database will miss citations to the scholar’s work in journals other than law reviews -- this includes journals in the Ph.D. field. For American legal historians, this would include citations in the Journal of American History, American Historical Review, and other history journals. Second, legal scholars often confine their research to the same Westlaw database, and so they don’t find and cite to relevant books and articles.
This leads to two under-counting problems. First, the Westlaw JLR database will miss citations to the scholar’s work in journals other than law reviews -- this includes journals in the Ph.D. field. For American legal historians, this would include citations in the Journal of American History, American Historical Review, and other history journals. Second, legal scholars often confine their research to the same Westlaw database, and so they don’t find and cite to relevant books and articles.
The limitations of this sort of study are not ameliorated by separating out a field like legal history. Using the Westlaw database will undercount those scholars who have a stronger impact across scholarly journals (beyond those in the legal database), and who do more publishing in books and peer reviewed history articles.
Even a more comprehensive citation study will skew in favor of scholars in larger sub-fields (e.g. American history as compared to medieval studies).
It is also important to point out that Leiter does not count legal historians with appointments outside of law schools. A number of leaders in the field have such appointments.
More on this, including the problem of unfortunate incentives created by lists like this, is here.
