Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. LFC's avatar

    You’re right, that’s my mistake. Because the two incidents occurred a couple of months apart, I guess they often get…

  2. EAS's avatar

    This is incorrect. The incident involving Petrov occured on September 26, 1983. Petrov judged that the Oko early warning system’s…

  3. Gwen Bradford's avatar
  4. Anon's avatar
  5. Nathaniel Jezzi's avatar

    Although I didn’t know Dale well, I had the good fortune to meet and interact with him during graduate school…

  6. Abdul Ansari's avatar

    I am shell shocked. Dale was an exemplary and creative moral philosophy, rigorously engaged with the most foundational issues across…

  7. David Wallace's avatar

    This is sharply at variance with my understanding of the situation. The general consensus for some while has been that…

A Case of Class Over Curriculum? (Philosophy and the LSAT)

Christopher Pynes

Philosophers often use LSAT data as a marketing tool to convince students who want to go to law school that they should major in philosophy. But how do we really know studying philosophy is the key to higher scores? Are the high scores really the result of studying philosophy or is there some other cause? In this post, I am going to suggest that privilege and economic class, rather than studying philosophy, better explain philosophy students’ LSAT results. Even if my suspicions are only partly right, it should make philosophers think twice about selling philosophy merely as a means to higher LSAT scores.

Making a causal connection for or against is very difficult (that alone should make us think twice). We don’t have access to either state level LSAT averages or LSAT scores from individual schools. What we have is the average score by majors. I am going to start with 2014 LSAC applicants by major report. Five of the most popular LSAT taking majors are: Political Science, Criminal Justice, Economics, Philosophy, and Sociology. Here are the LSAT results for those five majors:

5 majors LSAT scores

The National Center for Education Statistics uses a specific code to identify each college major: the Classification of Instructional Program (CIP) code. If you are going to look up graduation numbers for majors, you will use these codes. I don’t know what CIP groups each of these LSAT majors include in the LSAC report, but here are the program codes I am using for my argument: Political Science (CIP code: 38.1001), Criminal Justice (CIP code: 43.0103), Economics, (CIP code: 45.0601), Philosophy (CIP code: 38.0101), and Sociology (CIP Code: 45.1101). Moreover, the Illinois Board of Higher Education website has graduation data for Illinois public and private universities since 2010 that anyone can search here: IBHE search.

Before I give the data, let me point out that Illinois is the fifth largest state in the U.S. by population, so there shouldn’t be a sample size objection. There are 10 public universities (well, 9 now) that offer undergraduate degrees in philosophy and 30 private. In this file, Download LSAT Illinois Pub Private 4 Majors Data, the IBHE data is also sorted by race. (The IBHE site will sort it by race and gender if readers are interested.) Here is a chart of the number of degrees awarded by institution for the five majors over a three year span:

5 majors grad pub v private

What do we learn form this Illinois data? We know that philosophy graduates are far more likely to attend a private university than a public one. Of the 30 private colleges and universities offering a philosophy major (more than half of the private school philosophy grads came from 5 schools, check the file). Are the students at private universities academically better than those at state universities? Private school students certainly have access to more philosophy, but I have already written about that here and here.

Economics is the only other degree program that has more private graduates than public graduates. Could this be part of why economics students do so well on the LSAT? It’s not like Economic and Philosophy classes are the same–some philosophers think economics is the new astrology.

I claim a better explanation of philosophy and economic student success on the LSAT is that those students are overwhelmingly more privileged than criminal justice students–remember vastly more students graduate with degrees in economics and philosophy from private than public universities. I don’t know that we can separate the effect of privilege on these scores from the impact of the degree itself.

I have come to believe that it is inappropriate to tell students that a particular major will make them better at the LSAT. What makes students better at the LSAT is practicing the LSAT! I now tell all my students who want to go to law school that they need to dedicate real time (and money) to preparing for the LSAT–that means a course to prep for the exam itself. More affluent students in urban areas can do that; poorer students in less populated rural areas cannot. For example, there are no test prep centers within 80 miles of WIU. That means SAT, ACT, GRE, or LSAT prep courses. Access to test prep services is also an unknown, but important variable in all of this, but it all comes down to a certain kind of economic privilege and the ability to afford private school and private lessons to take the standardized tests. I see this lack of privilege in my students every day.

Unfortunately, the LSAC, programs of study, and the affluent have no interest in digging into the data to determine why some majors have much higher averages than others. I think it’s just another case of class over curriculum.

Leave a Reply to Michael B Cancel reply

Your email address will not be published. Required fields are marked *

25 responses to “A Case of Class Over Curriculum? (Philosophy and the LSAT)”

  1. Craig Callender

    Very interesting. That shows that there may be a "base rate fallacy" component behind the claim. But what about just focusing on public schools? Do philosophy majors at public schools still outperform their public peers? If they do, your claim may still be correct…it might just point to requiring a finer-grained measure of wealth/class than public v private. For even in public schools, it may be that there is an imbalance among rich-public and poor-public students amongst philosophy majors. or it may be that there is a genuine effect, albeit mitigated by the base rate considerations.

    Reply
  2. Perplexed

    Can you provide more details about the data you are discussing. I opened one of the links and was perplexed.

    When are you reporting just those who write the LSAT (by discipline), and when are you reporting those who get into or attend Law School by discipline? I cannot make sense of your claims without this clarification.

    Reply
  3. Christopher Pynes

    Perplexed — let me try to clear it up a bit.

    The first image is a summary of the data in the 2014 LSAC report on undergraduates majors who have a valid LSAT score and APPLIED to law school (the link has the accepted and enrolled percentages for each major as well but I did not include it in the image). That's this link: http://lsac.org/docs/default-source/data-(lsac-resources)-docs/2014-15_applicants-major.pdf

    I really should have sent people to this page with .xlsx files for each year as well as PDFs.
    http://www.lsac.org/lsacresources/data/applicants-by-major

    These files have the majors ordered by number test takers applicants by major So, the numbers in my image are the rankings of those five disciplines. When we see those composite charts of LSAT scores, I believe it comes from this data set. Now, I think you really want to know if the LSAC data is for all test takers or just test takers who applied to law school. It is the latter. I do not believe the that LSAC provides the data for ALL test takers.

    In the second image I am showing the number of philosophy graduate by kind of institution in Illinois (public vs private) as reported to the Illinois Board of Higher Education. I used the CIP code as the way to define those majors across colleges and universities. Each school is responsible for classifying their major appropriately. That file lives on this blog and so read "download LSAT Illinois…" That's the file where I put all the IBHE philosophy graduates data by race. There are lots of tabs and such, but it's a lot of information about race, but I just wanted the total number of graduates.

    The LSAC does, however, report the data on how many test takers there are by state. That data is at the following link: http://www.lsac.org/lsacresources/data/lsat-us-residence But we don't know what the scores per state are.

    Finally, the LSAC people don't make it easy to figure out the data. The have a firm stance that you can't infer form this stuff that one major is better than the other. In fact the direct quote from the report is: "It would be a mistake to infer from them (the data) that any one major is better than another in preparing students for the LSAT, or indeed, for law school itself." I'll agree with the first disjunct about the LSAT, but not the second. I do think philosophy helps students with law school. The LSAT is something else entirely.

    I hope that helps.

    Reply
  4. Anonymous

    At the university where I teach we hired someone who worked at Kaplan test prep for five years before coming to our university, who now works in our academic advising center. He once told me that in his experience teaching the LSAT the single best way he found to improve students' score was for them to take a logic class from a Philosophy department. He said the average improvement was like 12 points. This claim sounds pretty causal to me for what it's worth.

    Reply
  5. Christopher Pynes

    To Anonymous:

    It is certainly a causal claim, but there is no evidence that it is true. Let me give you my experience. I taught for Princeton Review, a rival test prep service to Kaplan, for three years. I have been teaching logic at the university level now for 16 years. There is no way he can make this claim with any credibility! None!! It might make philosophers feel better to hear that their classes improves LSAT scores, but his story just isn't true. A 12 point improvement is massive, and generally requires advanced LSAT test taking skills and strategies — and these are strategies that aren't usually taught in a normal logic class.

    I know because I do special projects with students every semester in my logic classes to help them prep for the test. Why? There is no prep service nearby, and I did the test prep job for so long. I know where and how the two mix, and it's not just 12 free points on the LSAT by taking a logic class. It requires work to connect the logic class with the test to get better. It's never obvious to the students.

    Next time you happen to be in the advising center, ask your advisor how he knows the 12 point improvement claim is true. Then ask him if he quite Kaplan because they were selling students a bill of goods since "the single best way" to get better at the LSAT, "in his experience of course" is by taking a logic class in a philosophy department. He won't have any way to justify that 12 point claim.

    Reply
  6. Michael B

    Just to say: I have long wondered about this issue and what the real causal relation is between studying philosophy as a major and taking these various US tests (I am not from the US or familiar with the tests).

    Certainly, in discussion I've seen, it does not seem plausible to me that the evidence for causation would survive scrutiny in a philosophy class.

    Anyway, thanks for making a post about it.

    Reply
  7. Anonymous

    I agree in thinking about it some more that the 12 point figure is high. But the basic claim at issue is whether taking a logic class or a philosophy class leads to a higher score. And this could be true even if you're right that the results are more modest. Here is a link from US News which says that (see sect. 4) "taking classes in logic, philosophy, or critical writing can prepare you for the test because they require you to analyze complicated theories or texts and present ideas gleaned from those texts in a concise and logical manner, which is similar to what the LSAT demands. Experts note that these classes are far from mandatory….but they can make a difference, even if it's only a few points." There still seems like a causal claim involved here since the classes make a difference. I wouldn't deny that other factors could be involved but that doesn't take away from this point. I have not taken the exam myself and I am merely relying on general information and appreciate your thoughts.

    http://www.usnews.com/education/best-graduate-schools/top-law-schools/lsat-test-prep/articles/2010/05/28/test-prep-7-tips-for-lsat-success

    Reply
  8. Keden T

    As a graduate of an Illinois public University who majored in Economics and minored in Philosophy, as well as an LSAT taker, your article has piqued my interest.

    Here you (correctly) reject a causal inference that certain majors cause higher LSAT scores.
    However, you simply insert your own causal inference in its stead.

    The specifics of the causal inference that you chose to draw are troublesome. If anyone reflects on it for a short while, he will see that "privilege" is quite an elusive concept.

    Is a prodigy who is born in the grips of crushing poverty privileged?
    Is a child with learning disability, born to millionaire parents privileged?

    There are plenty of interesting causal hypotheses that can be drawn from the data. It seems to me that one involving the amorphous concept of "privilege" does not fall into this category.

    Reply
  9. Philosophy Graduate studying for the LSAT

    Interesting post!

    One thing I will say in defence of philosophy with respect to the LSAT is that, in my experience, it gives a very solid foundation for all three sections, if not an inherent ability to score a good grade. What do I mean by this?

    Well, for example, when it came to logic games I really had no clue how to go about solving them when I started. I could maybe get a couple right on a section through mental brute force, but that's it. Without coaching (which ended up being a book in my case – rural UK, so no courses around!) I was about as lost as anyone else. However, when it came to learning the proper technique, there was a lot of fundamentals I was already highly familiar with due to my philosophical training. In particular, conditional reasoning, negating conditionals, etc. Similar stuff held for Reading Comp and Logical Reasoning, although I think in those cases you could perhaps pick up the relevant skills from, say, History or English majors as well.

    So although I would agree that there is perhaps not an isolated causal connection between philosophy and a good score, I think anecdotal evidence will tell you that philosophy certainly gives you a very good leg-up if you do decide to prepare. An interesting further question would be whether this leg-up is unique to philosophy, or if it is a leg-up which can be bridged by non-philosophers (I suspect it can).

    Reply
  10. paolo

    It seems to me that we are, as often, back in Euthyphro. At best, it is possible to claim that "being priviliged and from higher economic class" is a feature of those who score better. But it hardly seems to be (from anything you said) that which makes them score better. In order to show that, one would have to do a bit more work. At least in the case of philosophy we have a correlation of content (i.e., logical puzzle solving in both philosophy and the test) beyond simple correlation of sets (priviliged – score better). In fact, notice that it being priviliged people (whatever that means) is compatible with philosophy still doing it – it might be that people from more affluent backgrounds tend to choose philosophy more often than those from less affluent ones (an unsurprising claim, given philosophy's nature and practicality) but that it is in virtue of them taking philosophy that they end up better poised for the test. It also might be that people who choose philosophy like certain kind of thinking that involves logical puzzles and so on (and liking something often goes hand in hand in being good or at least decent at it – as few people like being frustrated) and that kind of thinking is very much present in the test. And it might be a myriad of other things.

    Reply
  11. Gary Gutting

    To my mind, the basic point is that a mere correlation between taking philosophy courses and doing well on the LSAT is not a good reason to claim a causal relation. If we merely cite the correlation as evidence for causation, we're making a logical mistake in our very effort to show that philosophy improves logical thinking. Are there good randomized controlled studies (or the equivalent) that support the causal claim? If so, we should cite them. If not, we shouldn't make the claim.

    Reply
  12. Ian

    +1 Gary Gutting

    Reply
  13. Ian

    Even if socio-economic privilege is discounted, the problem of self-selection remains. Philosophy is a notably difficult major, certain if we compare it to Criminal Justice. You will find few, it seems to me, undergrads who are weighing the pros and cons of CJ vs. philosophy, which is to say that one has a certain kind of student in CJ–which my father taught for years and in which I am now teaching–and one has a generally other type of student in Philosophy. So why would it be the case that the study of philosophy necessarily leads to higher scores? Couldn't it simply be that a certain type of student (one that likes abstraction and puzzles, one who challenges herself, one that is diligent and booksmart) tends to gravitate toward philosophy and to do well on the LSAT.

    Saying that philosophy undergrads tend to do well on the LSAT is true. Saying that they tend to do well because they are philosophy majors is a highly suspect claim.

    Reply
  14. Horace

    Mystery solved: http://libertyviral.com/wp-content/uploads/2014/06/iq-by-college-major-gender.png

    Reply
  15. David H

    I've always thought that a common cause draws people to philosophy and explains their success on standardized tests. Ian's suggestion about the nature of that cause is plausible. If we could isolate it, we could test whether high school students who have it to the same degree later diverge in their results based on the majors they chose. Another method might be to compare students on either side of a margin – ones who were strongly tempted to choose philosophy but resisted, vs. ones who were strongly tempted to choose a different major but instead chose philosophy. My statistically irrelevant anecdote that I will add to the discussion: surprisingly many successful people I meet tell me "I loved philosophy in college and came very close to majoring in it!"

    Another factor to consider is that different majors "wash out" students at different rates, for differing reasons. This mechanism alone could explain many of the differences of the remaining cohorts.

    Reply
  16. Christopher Pynes

    Horace — I considered not publishing your chart. Here's the thing. If income is correlated with IQ and IQ is heritable, then my case is even stronger.

    Reply
  17. Eddy Nahmias

    How about this as a way to test the causal claims:
    Get a large sample of students and examine the relationship between their pre-college test scores (e.g., using the percentile scores on SAT and/or ACT) and their scores (using percentile) on tests like GRE and LSAT, and see whether the differences correlate with major (or even particular courses like symbolic logic).

    If philosophy majors go up an average of, say, 2%, while most other majors do not go up, I think that would be good evidence that studying philosophy causes improvement on such tests relative to most other majors (perhaps by improving reasoning skills or reading skills). Of course, this method would also allow us to examine whether the improvements, should they exist, show up across different types of colleges, SES, race, gender, etc.

    From the armchair, I'd predict that this method would show that studying philosophy does tend to cause a very small improvement in standardized test scores (pre- to post-college), while most majors do not, except for math and science majors may improve on quantitative tests (SAT vs GRE quant sections).

    I think the relevant data should be out there to be mined, no?

    Reply
  18. Horace

    Christopher Pynes: it supports the claim that philosophy is not causing higher LSAT attainment, but it doesn't support the claim that class or privilege is what is causing higher LSAT scores. Rather, it offers an alternative explanation of the association between philosophy and LSAT scores. The question is whether the best explanation is that philosophers' privilege causes higher test scores or their higher cognitive ability.

    Reply
  19. Ian

    I'd argue that it stands to reason that any major that focuses rigorous and critical reading, as well as on good writing, would likely show improvements. That's not discount the Phil major but to say as the author says above, taking these tests is largely about a skill set and not higher intelligences or actual reasoning ability.

    Reply
  20. Daniel

    One good way of measuring this would be to ask LSAC to release the same data for those who were granted fee waivers. While this wouldn't be a perfect test (because many who may deserve fee waivers either don't qualify according to LSAC's rather stringent criteria, or else because they simply don't know about the existence of such a waiver), it would at least tell you whether philosophy majors using fee waivers fared better than other majors using fee waivers. And I sincerely doubt anyone who could be described as "privileged" qualifies for an LSAC fee waiver.

    Reply
  21. Michael B

    I wonder if other majors *commonly* market themselves as being good for improving scores on these sorts of tests and if philosophy *commonly* markets itself in this way too. If the latter but not the former, do we not now have a possible placebo effect to disentangle? Although I suppose placebos do work (they are not defined as "interventions that don't work") – it is simply the mechanism by which the placebo effect operates is not the mechanism that uninitiated people think. Which kind of leads back to square one, here!

    Reply
  22. Anonymous

    Here is what one test prep place says about whether a logic class will help on the LSAT. While not strictly necessary, the view is that they do help students when they are taken.

    http://nextsteptestprep.com/2011/02/22/will-a-logic-class-help-on-the-lsat/

    Reply
  23. RF

    Suppose we were learn that majoring in philosophy has little causal effect on LSAT scores, career outcomes, and so forth. I contend that even if this were so, there would still be a strong basis for "marketing" philosophy majors' successes; namely, to demonstrate to interested students (and their wary parents) that majoring in philosophy is no impediment to success in law, business, and other non-academic careers. Many students who are keen on philosophy nevertheless have the false impression that a pursuing a philosophy degree will adversely impact their career prospects. Even if we can't show that a philosophy degree uniquely contributes to this or that desirable life outcome, we can perhaps show that it doesn't frustrate such outcomes either.

    Reply
  24. Carrie

    Which selective private LACs or universities even *offer* criminal justice as a major?

    If we look at the IL sample, you won't find criminal justice at UChicago, Northwestern, Wheaton (IL), Knox, or Lake Forest (chose the last three randomly as "somewhat selective LACs").

    If we assume that the selectivity of institutions largely acts in a gatekeeping function for academic talent, or at least "assessments of academic talent" (aka standardized tests, like the SAT and ACT), and we assume that the vast majority of selective (and highly selective and super-selective) institutions do not offer criminal justice majors, it seems sound that criminal justice majors would have average lower scores…no?

    (And yes, I recognize that the gatekeeping assumption is fraught with issues and that many highly talented students may enroll at non-selective institutions for a host of reasons that are systemic, such as cost, access, culture, etc).

    Reply
  25. Ian

    Carrie +1

    —–
    KEYWORDS:
    Primary Blog

    Reply

Designed with WordPress