Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. Giovanni Molteni Tagliabue's avatar
  2. Fabien Muller's avatar
  5. Saul Smilansky's avatar
  6. Dan Dennis's avatar

    Some background: https://www.theguardian.com/education/2026/may/12/thousands-of-university-of-nottingham-staff-told-they-are-at-risk-of-redundancy Not only does Nottingham University have a good academic reputation, the city of Nottingham has a great…

  7. Jacob Barrett's avatar

AI grades law school exams about as well as human law professors

Brian Leiter

Here.

ADDENDUM: I joked with Professor Schwarcz (Minnnesota), one of the co-authors, that soon AI would be writing exam answers, and AI grading them! He replied, sensibly, as follows:

The key difference is that students SHOULD not use AI to craft their exam answers. The purpose of a law school exam is to show their own legal reasoning, which AI assistance undermines. By contrast, the purpose of grading an exam is simply to generate reliable, consistent evaluations.  So if an AI can accomplish this goal well, then it seems to me there is much less of a problem with professors using AI to accomplish that objective.  

At the very least, I think our results suggest that AI can and should be used by law students to help provide feedback to students about the quality of their work and how it is trending over time.  Indeed, our results may even suggest that law schools can meet the new ABA mandate to provide formative feedback to 1Ls simply by relying on AI grading of exams…

,

Leave a Reply

Your email address will not be published. Required fields are marked *

One response to “AI grades law school exams about as well as human law professors”

  1. Pedro Gerson

    This presupposes that we don’t gain anything by grading our exams. But we do. We see what students learned, what clearly came through, what didn’t, what they had a harder time with, etc. And we can capture it in ways that I don’t think AI can or would. Now, of course, grading 100 1L exams is tiring and overkill in terms of what we gain, but if we care about our teaching and improving it then grading is one of the few opportunities to get an honest window into how we’re doing (that is if we want to see through it).

    Reply

Designed with WordPress