Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. Fool's avatar
  2. Santa Monica's avatar
  3. Charles Bakker's avatar
  4. Matty Silverstein's avatar
  5. Jason's avatar
  6. Nathan Meyvis's avatar
  7. Stefan Sciaraffa's avatar

    The McMaster Department of Philosophy has now put together the following notice commemorating Barry: Barry Allen: A Philosophical Life Barry…

Inflated Letters of Recommendation

Brian Leiter

We've commented before on letters of recommendation, and the special code they're written in, which is related to the more general probelm of "recommendation inflation" which has become rampant in recent years.  But this report, from a philosopher at a top department, is very worrisome indeed:

I am concerned about a recent trend in departments attempting to place job candidates. After the job candidate assembles his or her dossier, the placement director looks over the letters of the candidate to get a sense of how the candidate will look on the market. Sometimes, after this review, the placement director will contact a letter-writer to ask him or her to consider revising the letter. Officially, this sort of request  is supposed to target unintended infelicities in the letters ("you'll be lucky if you can get him to work for you" or "this research fills a needed gap in the literature," etc.) But it seems to be more common lately for placement directors to ask for changes that alter the letter in order to improve the evaluation of the candidate, usually by asking letter writers to remove salient information such as explicit rankings or comparisons when the candidate is ranked below others.

 Given that we found previously that those who have to read these letters value such comparisons, this seems a very unfortunate trend, if that is what it is.  What do readers think?   Anonymous posts are acceptable on this thread, though please include a valid e-mail address, which will not appear.

Leave a Reply to Anon Job Seeker Cancel reply

Your email address will not be published. Required fields are marked *

47 responses to “Inflated Letters of Recommendation”

  1. Rebecca Kukla

    I agree that this is a problem but I am not sure what to do about it, quite, because having placement directors look over letters and make occasional suggestions also strikes me as valuable. Some faculty just don't know how to write letters, or they sometimes toss off a sloppy one, and they can do real, unfair, and unintended harm to the students they are supporting. I have seen a few this season that could have used such oversight – for example, we've gotten several letters that buy into ethnic or gender stereotypes in totally cringe-worthy ways, and I feel like another set of eyes would likely have nipped the problem in the bud.

    Something, however, simply must be done about letter inflation. We got one letter this year that said, in a boldfaced, centered sentence separate from the rest of the paragraph, "x is, in my opinion, the best philosopher of his generation". Literally. This kind of over the top and completely epistemically unsupportable comment actually undercuts the impact of the letter, in my opinion, by undermining trust in its writer, and it also contributes to the never-ending escalation of the already ridiculously high rhetorical bar that letters must meet to sound positive. I have no idea what should be done as we clearly seem caught in some collective rationality paradox.

    I certainly agree that if people are being asked to revise letters to sound stronger and to have fewer objective comparative claims, this is contributing to the problem rather than to the solution. That's not how placement director oversight has worked in either my current or previous department, but looking at the letters we are receiving it does seem plausible that that's going on.

    Reply
  2. Elizabeth Harman

    I second Rebecca Kukla's comment that there is a real need for placement directors to help letter-writers write better letters. When I was a new professor, I wanted to write good letters (informative letters that were the strongest letters I could write for the candidates I was writing for) but I didn't know how to do it. The placement director at my first job gave me invaluable feedback, to make my letters clearer and to make them more accurately communicate the worth of the candidates.
    Now I have some experience writing letters, but even so I benefit tremendously from having some feedback on my letters. I would be reluctant to burden random colleagues by asking them to help me improve my letters; it's very nice that the placement officer sees it as her/his job to do so.
    Also, I can report that I've never been pressured to make my letters more positive than an accurate reflection of my assessment of the candidates I'm writing about.

    Reply
  3. Alastair Norcross

    I'm not sure that letter inflation has got worse recently. For as long as I've been reading letters, they've seemed inflated. I remember reading one letter during the first search I participated in, eighteen years ago, from an extremely prominent philosopher. The letter declared that the candidate had written the "best dissertation I've ever read, with the possible exception of the Tractatus". I'm pretty sure that letter actually harmed the candidate. We interviewed that person–how could we pass up the opportunity to interview someone who might be the next Wittgenstein?–but the expectations were so impossibly high that the interview was, inevitably, a disappointment. Over the next few years I read many more ridiculous letters from the same letter writer. Now I don't trust anything that person has to say (in recommendation letters). What made the letters even more absurd was that they were all prefaced with a disclaimer along the following lines: "I deplore the recent trend towards recommendation inflation. I propose, therefore, to deliver only the plain unvarnished truth. However, in this case, I find myself embarrassed by the fact that the plain unvarnished truth sounds suspiciously like recommendation inflation".

    Another problem is that letters from UK philosophers are much less likely to be inflated. I've found myself on more than one occasion having to remind myself that a comparatively lukewarm recommendation was by a UK philosopher, and might actually be an extremely strong letter! I remember Jonathan Bennett telling me a probably apocryphal story about a letter written by a prominent Oxford philosopher for another soon-to-be prominent philosopher. The letter supposedly said something like "Mr. X isn't a very good philosopher. But there aren't any good philosophers around nowadays, and Mr. X is one of the least bad philosophers I know".

    As to the subject of comparisons in letters, I have mixed feelings. Many departments, my own included, don't have official departmental rankings of students. There is much disagreement among the faculty about the comparative merits of the graduate students. I don't give explicit comparisons of graduate students with others in the same program, unless I am highly confident that my opinion is shared by at least the vast majority of my colleagues. This occasionally happens, but is certainly not the norm.

    Reply
  4. Job Seeker

    Suppose the following is a norm of letter writing: With respect to the candidate's chances of success, write the strongest, most effective letter possible while remaining honest.

    If that's indeed a norm, then explicit comparisons which might hurt the candidate should be omitted.

    Not sure I agree with that conditional, but I suspect it's worth thinking about.

    Reply
  5. dankaufman!

    i have to admit that i'm a fan of the bolded, separated sentence, especially when it is of the form 'x is at least as good as y (now faculty at fancy-pants u.) and z (now faculty at awesome research u, at blah blah)'.

    however, i think that the letter writers making such claims (and especially those making claims like the one professor kukla mentions) need to consider the potential harm to their students by making such claims. if someone writes that this candidate is the best student that this prestigious department has ever produced, well, that candidate better be transported to their campus interview in a angel-propelled throne made of gold with cushions woven from saffron and the eyelashes of unicorns. and they better give a talk that makes NAMING AND NECESSITY look like the work of a 3rd grader. otherwise, it is very difficult for the candidate to live up to their letters, and that leads unfortunately to disappointment, even when the candidate is really, really good.

    so, if you are thinking of writing something like the sentence professor kukla mentions, maybe you should precede it with something like this: "i know this is going to sound like hyperbole, and it may even sound completely silly. but i firmly believe that…"
    and after the claim, write something like this: "seriously. i mean, i am totally dead serious about what i just said. seriously." seriously.

    xoxo

    Reply
  6. Ryan Tonkens

    From the perspective of a junior philosopher currently on his virgin run at the job market, there does seem to be some benefit to having someone else read your reference letters ahead of time. Since candidates are not supposed to have access to the letters [why, i'll be honest, I am not sure], this is one way to make sure that the people writing the letters have the candidate's best-interests at heart, and that the candidate is not 'mistaken' in her choice of letter writers (which I presume is something that happens, however uncommonly). There is no other honest way that I am aware of that could protect the unknowing candidate from including poor or inaccurate or rushed letters in their application package, especially since hiring committees do not seem to offer any feedback about why the candidate failed, even if they do send out an (often impersonal, blanket) email notification of not making the cut–even then, this would only be retrospective, and thus unhelpful for the current job search. Since it seems that reference letters are quite important in the ranking of job applicants, candidates may sleep a little easier knowing that another trusted faculty member has also given the go ahead.

    [I am glad to hear from some comments on the earlier thread that the names of recognized big-shot philosophers are not the ONLY thing being considered when looking at reference letters, but also the quality of the letter and how realistic it is].

    I believe that the letter of reference is a sort of vouching for the candidate. If the letter is written in such a way that the legitimacy of the writter's vouching is taken into question, then it will certainly hinder the candidate's success. One way to avoid this is to have other people read the letter before it is sent out. Many aspects of academic philosophy are peer-reviewed, in part because it promotes higher quality work. There may be some benefit to making letter writing [and job applications] peer-reviewed as well, with the same hope in mind.

    Reply
  7. Michael Rosen

    Well, colour me cynical, but surely no sane person with a reasonable amount of experience could doubt that recommendation letter-writing nowadays is simply an exercise in (useful American word) bloviation.

    [Note: I say "nowadays". See: http://leiterreports.typepad.com/blog/2005/02/isaiah_berlin_o.html%5D

    Recommendation letters can give a picture of the candidate’s intellectual personality beyond what emerges from the bare facts of a c.v. and – this is the most important, I think – they can explain the originality and importance of her or his work, but anyone who treats letters as a tool for comparative assessment and ranking of candidates deserves whatever they get, in my opinion. (Are these the same people who want to get rid of interviews?)

    Reply
  8. Jeff McMahan

    I have no idea what is done in other departments but as placement director at Rutgers, I can say what I do here. I try – though never wholly successfully – to review all the letters for our candidates. I often find typographical errors and infelicities of word choice or expression that I ask writers, whether from our department or elsewhere, to correct, since carelessness or sloppiness can blunt the effect of an otherwise strong letter. Very occasionally I find a phrase or sentence that I think will be damaging to a candidate when I suspect that the writer doesn't intend it to be. In those cases I alert the writer to what I think the impact will be and suggest that the passage be revised if the writer doesn't intend it to be understood in a way that could have an unfavorable effect. Recently I also pointed out to one letter writer that an unusally short letter can be harmful and suggested ways in which the writer could either explain why the letter was short or add more detail. In a few cases when I have suspected that a letter was weaker than it might be because the writer had read relatively little of the candidate's work, or had read only older work, I have suggested that the writer might read more of the candidate's work, or some of his or her newer work. But I have never requested the deletion of material that would be damaging because it's true, or the writer believes it to be true. I'm fairly confident that in five years as placement director I've never seen a letter that compares one of our candidates favorably or unfavorably with another. I think everyone assumes that any such comparative evaluations will emerge in the differing strengths of the individual letters for the different candidates.

    Reply
  9. Simon Keller

    In my experience, hyperbole in reference letters is usually ignored. In place of straightforward praise, what makes the candidate look good is the letter's explaining the candidate's work and its significance. Often, a letter that's high on praise but short on detail make the letter-writer seem not really to know what the candidate's actually doing. ("The candidate is doing fantastic work in moral philosophy and is the best young philosopher I've ever seen" is ignored. An explanation of what the candidate's doing and why it's important and original is not.)

    On comparisons: here's a kind of comparison that in my experience is often worse than useless. "The candidate is the nth best young philosopher working in this area, rivaled only by A, B, and C."

    I have seen comparisons like this lead people on job committees to think, "Hang on. I am (or my friend/colleague is) a young person working in this area. And Letter-Writer doesn't seem ever to have read my work (or my friend or colleague's work). Letter-Writer is pompous and presumptuous and will be ignored." And I've seen it lead to: "Well, I know A, B, or C's work, and it's not very impressive. Letter-Writer doesn't have a clue. Ignore letter."

    In the end, the best way to make a candidate look good is to say in detail what's good about him or her. Superlatives don't get you anywhere, and comparisons (of whatever kind) cannot do the work on their own.

    Reply
  10. Tom Dougherty

    As an immediate disclaimer, I've never been involved in a search. I'm interested in what more experienced people think about a couple of ideas.

    First, letter-writers specify a group of other students they have advised, who are reasonable points of comparison (e.g. similar areas of research or approach), and then specify whether the candidate is in, say, the top 2/3 of the pack, 1/3 of the pack etc. This group-based comparison strikes me as more helpful than comparing a candidate to a specific individual, as I'd expect the letter-reader's and letter writer's judgments about a group to converge more than their judgments about an individual.

    Second, if letter-writers don't offer this sort of info in their initial letter, then search committees contact letter-writers of candidates on, e.g., the long short-list, and ask for it directly. Maybe search committees follow up like this. But it strikes me that they have the greatest incentive for seeking extra information about candidates, and so their initiatives might be the most reliable way of getting more information into the market.

    I'd be very interested if anyone sees problems with these suggestions, or reasons why other approaches are better.

    Reply
  11. Gabriele Contessa

    Let's not forget that, as Homer Simpson once said to Marge, 'It takes two to lie: one to lie and one to listen'. I think that letter readers are as blameworthy as letter writers for the inflation phenomenon. I have seen people scanning letters of reference through the electron microscope for even the tiniest shadow of a clue that what the letter writer *really* meant to say was that the candidate was not *that* good after all. Since most letter writers are likely to have experienced themselves first-hand many such displays of hermeneutic zeal in action (either because they engage in that practice themselves when they read reference letters or because some of their colleagues do), trying to avoid saying anything that could be interpreted even remotely as not being sufficiently positive is only natural, if one doesn't want to hurt a candidate's chances. But how positive does positive need to be to withstand such an extraordinary amount of scrutiny?

    As far as I'm concerned, I've had enough of trying to second-, third-, or fourth-guess what the letter writer *really* thinks of the candidate. I just take positive reference letters (no matter how positive) to carry virtually no evidential weight.

    Reply
  12. Jon Cogburn

    I think Michael Rosen and Gabriele Contessa get it perfectly right above.

    The only thing letters are useful for is giving members of the hiring committee a better picture of relevant aspects of the candidate's personality. How does an expert see the dissertation and writing sample as fitting into current debates in her field? What evidence can the expert put forward that the candidate will produce good work beyond what can be mined from her dissertation? What evidence can the candidate's committee members put forward that show that the candidate will do her share of service work in the department and otherwise be a good citizen and teacher?

    Really good letters give non-experts who don't know the candidate a more vivid picture of what they are going to get with the person.

    But, like Contessa, I try as much as possible to ignore all praise that involves morally thin (in the virtue theoretic sense of the term) predicates. There are so many heuristic biases at work that it's just noise, and should be regarded as such.

    I also try to set aside any star-struck reactions of the "look what X said about this candidate!" Unfortunately, these kinds of sentiments are distressingly common. Of course X said that. X was the candidate's adviser. I already knew that before I started reading all these letters.

    Reply
  13. Anon Job Seeker

    As a job candidate, I am relatively optimistic about the phenomenon of recommendation inflation, as I believe that most SC members will be attending to the meat of the letter — the description of the candidate's work and its merits and perhaps what the letter writer learned about the subject as a result. Comments about the kind of philosopher the candidate is are also useful in helping departments decide what they want in terms of fit. Some of us are big picture types, some of us meticulous critics, and those of us that are lucky, have both virtues. Statements about strengths and weaknesses should also be addressed to give the committee a full picture of the candidate. Comparisons I don't think are really very useful — they're highly anecdotal and subject to the apples and oranges problem as well as idiosyncratic opinion much more than the other kinds of assessment. I happen to know that my letters are mainly of the meaty kind, not because I have been told the actual content of my letters of course, but because I have been told that those are the kinds of letters my letter writers produce for me, and that they believe that these kinds of letters have the most value. A problem arises I suppose if writers do not take the time to engage with a candidate's work sufficiently to write a meaty letter and if this is pervasive then letters will cease to count for much. A very bad thing indeed. And, of course, there is also the potential problem that SC members will think that candidates without over the top recommendations are not as good as other candidates with more meaty letters, but as I said, I am optimistic, perhaps overly so, about SC members being able to see through this kind of bluster. However, if some of the comments here are right, then maybe my letters that actually contain valuable information about my abilities and work will hurt me. I sure hope not.

    Reply
  14. Mike Otsuka

    A set of letters from a top US PhD program can provide a pretty good sense of how well a given PhD student is generally regarded by the faculty there, as compared with other current and recently graduated PhD students from the same department, all of whom will be reasonably well-known to these faculty members.

    Of course, the candidate’s advisor will generally be biased in favor. But one discounts for that. The faculty will be collectively biased in favor of their own students as compared with students from other departments, so one pays little attention to any cross-departmental comparisons. Some faculty members will tend to be more generous and undiscriminating in their praise than others. But one soon learns who these people are, since one tends to come across multiple letters from faculty in top departments with successful PhD programs. The department will be careful to assemble that set of letters that puts the candidate in the best light, and hence, like the children of Lake Wobegon, every candidate will be above average. But this selection bias works in favor of every candidate to a roughly equal degree. In dog that didn’t bark fashion, it also makes silences meaningful: the suspicion is often justified that a faculty member is not so impressed by a candidate if there’s no letter from him, even though he’d be a natural referee, given the candidate’s dissertation topic.

    Generally speaking, faculty members at top departments are good philosophers who are good at judging philosophical talent. Often a top department is as good as it is because their members are good at spotting and hiring talented philosophers. So it’s useful to have a sense of whatever collective ordinal ranking of the PhD students in their department emerges from their different sets of letters. For that reason, I wouldn’t be so dismissive of references, however inflated in absolute terms they may be.

    Reply
  15. Gabriele Contessa

    With all due respect to Mike Otsuka whose philosophical work I really appreciate (and leaving aside his use of the male pronouns), the sentence "the suspicion is often justified that a faculty member is not so impressed by a candidate if there’s no letter from him, even though he’d be a natural referee, given the candidate’s dissertation topic" brings the sort of heremneutic zeal I was talking about to a whole new level (a level which unfortunately I once witnessed first-hand). May I remind conscientious letter readers that there are all sorts of legitimate reasons why a candidate might not want to have anything to do with a faculty member who from the outside might look like a natural referee for the candidate?

    Also, two more points: (1) in order not to contribute to job-season panic, I should add that I suspect that the sort of hermeneutic zeal I was talking about in my comment above is (regrettably) often used to bring down candidates one already doesn't like.

    (2) to complement something Rebecca said earlier, it is disappointing to see that some very well-regarded figures in the profession (which typically happen to be what some would call 'old white men') seem to have a hard time not to adopt a condescending tone when writing about (at least some) female or "ethnic" candidates. (Although, luckily, I haven't seen this very often, I think it is still regrettable to see otherwise intelligent people failing to act intelligently.)

    Reply
  16. Mike Otsuka

    I agree that one shouldn’t assume that the non-letter-writer thinks badly of the candidate. But is there something wrong with wondering why there isn’t a letter from an obvious person and for someone in the department to follow this up by asking that person what she thinks of the candidate? At the very least, you’ll get the opinion of someone who knows what she’s talking about, when it comes to the area in which the candidate is working, and who’s also probably also at least somewhat familiar with the candidate. And often, you’ll end up hearing something useful and less varnished than what’s in the letters.

    Reply
  17. another anon job candidate

    """But is there something wrong with wondering why there isn’t a letter from an obvious person and for someone in the department to follow this up by asking that person what she thinks of the candidate? At the very least, you’ll get the opinion of someone who knows what she’s talking about, when it comes to the area in which the candidate is working, and who’s also probably also at least somewhat familiar with the candidate. And often, you’ll end up hearing something useful and less varnished than what’s in the letters."""

    Less varnished, sure, but how can you be sure that it's useful? One has heard of conflicts between potential (even natural-given-subject-matter) letter writers and job candidates such that there are personal or political reasons why the potential letter writer, on the whole, is not a good source of information about the candidate, whether formal or informal. It may not be easy, or possible, to separate one's assessment of a candidate's philosophical abilities from the facts of the relationship more generally.

    It's possible that such a person, if contacted, would have the self-awareness to decline to comment, but even that could and probably would be taken as an indictment of the candidate's abilities, rather than a reflection of the relationship between the candidate and the potential letter writer. And it's also possible that such a person, if contacted, would take the opportunity to badmouth the candidate. (Perhaps thinking of this as simply giving an unvarnished opinion.)

    I don't see that the person on the other end of this conversation would be particularly well placed to be able to tell whether the information thus obtained was actually useful.

    Reply
  18. Rebecca Kukla

    I think that willing oneself to ignore positive evaluative language in letters can be a good technique for not getting swept up in false positives, but that doesn't mean that the inflated rhetorical standards do no harm or can be rationally ignored when reading letters. Even if you decide to ignore positive evaluative language, if you get a letter that says "Bob is among the top half of our graduate students, and is a solid philosopher who deserves a job in the field. He has written a competent dissertation that shows a good grasp of the literature and makes a small but useful move of his own in the contemporary debates", how should you read this? Given that we all *know* about the inflated standards, we would be crazy not to read this letter as a total pan. It would be the kiss of death. But of course, this is a perfectly literal description of many an excellent job candidate deserving of a good job – the very type of job candidate that may well land a good job, given that he will actually be described as a spectacular philosopher who is sure to transform his field and is indeed already doing so, one who makes everyone who has the honor of teaching him giddy with excitement. This means that people are stuck using the crazy language even if not one is taking it as actually saying anything positive, because otherwise they know they will produce false negatives.

    Reply
  19. Cora Diamond

    On hermeneutic zeal, in relation to the issue mentioned by Alastair Norcross, — of the different conventions governing letter-writing in the US compared with the UK. For candidates applying to jobs in both the UK and the US, I used to write two letters, one in UK-reference-style for the UK jobs, and one in US-style for the candidate's US applications. Then I realized that the hermeneutic zeal of people in the UK who were familiar with American letters might make a UK-style letter from someone in the US appear much more negative than I intended. Unfortunately one can't write with less inflation than one might be expected to go in for. If 'outstanding' is going to be read as 'merely outstanding, not superb', what can one do?

    Reply
  20. Hilde45

    Whether praise is escalating or not, I don't know. But I would agree with previous comments discounting the evidential value of praise. What has helped me as a search committee member are (in no particular order): (a) concise summaries of the candidate's research that quickly give me the gist, (b) specific comments which help me see the candidate as a person (e.g., their sense of humor, their ability to push past obstacles–specific anecdotes are great), and (c) a nuanced appreciation by the writer to explain why the candidate would fit well with *my* department. Too often, I think, job seekers come from high powered research institutions with high powered letter writers who don't really take a moment to think about what a department without a Ph.D. really needs in a colleague. My advice would be that writers consider drafting two versions: one for other Ph.D. programs and another for departments more oriented toward teaching undergraduates.

    Reply
  21. Untenured placement director

    On whether it is reasonable for search committees to contact faculty they assume would be working in a job candidate's area but who didn't write letters: it seems to me that this practice should be strongly discouraged. We all know that there are many reasons a particular student X would not want to work with a faculty member Y. Sometimes faculty member Y is a sexist, amoral jerk. Sometimes faculty member Y can't stand having his or her work criticized. But more often, there's just a clash of personalities. This happens all the time, and shouldn't be held against a job market candidate. Refusing to ask for a letter from faculty member Y is just a common reflection that maybe they didn't get on, and that X simply exercised his or her right not to engage with that faculty member. For search committees to dig around in that sort of stuff strikes me as unlikely to yield good information, and furthermore to open up a job candidate to an unfair assessment.

    But there's more evidence than just this speculation. I'm a current placement director in a department with a certain faculty member who has developed acrimonious relationships with other faculty members in the department (call this person "P"). P, though, works in an area in which some of our job market candidates work, and consequently wrote letters on behalf of some of those candidates. But these letters were laced with vitriolic remarks about other faculty members and the department as a whole, and said, quite often, very little about the candidates. So these letters were pulled from the relevant files. Now if a search committee noticed that this person didn't write a letter and contacted P for an assessment, they would very likely get the same sort of vitriol, which would certainly damage the candidate in question.

    Ultimately, the overly positive rhetoric in letters of recommendation is leading to a circumstance in which very few of those letters are actually trusted to give an account of the candidate. And instead the burden is borne by the candidate's writing, CV, teaching file, face-to-face interaction with search committee members, and ability to give an engaging talk or teaching demonstration. Hard to see why this could possibly be a bad thing.

    Reply
  22. Anonymous Job Candidate

    When the top two criteria for job placement are pedigree and letters, we are but one small step from the olden days in which departments would call up the chair at, say, Harvard or Princeton and ask who the top job candidate is. That person would then get hired without an interview. Thankfully, we at least have interviews these days, but the fact that some folks inflate letters and others don't makes our current system in some ways even worse than this "good ole boy" club practice. One's entire career can depend on whether one's letter writers are willing to write inflated letters.

    Reply
  23. Appalled_Grad_Student

    As one soon to go on the job market, I am utterly horrified by Jeff McMahan's statement above:

    "I have never requested the deletion of material that would be damaging because it's true, or the writer believes it to be true."

    If I ask whether a professor can write me a strong recommendation, don't I have a reasonable expectation that if the writer says "yes, I can" then they will not write anything "damaging" to my chances of getting hired (whether true or false)?

    And if a department invests its placement director with powers of review over recommendations by faculty, isn't it a breach of trust (and professional ethics) for that person to allow the faculty to write "damaging" material (whether true or not)?

    I would think the practice of recommendation writing/reviewing ought to come with some set of ethical norms that protect the student.

    BL COMMENT: This raises a difficult issue, one closely connected to the original topic of this thread. Surely the ethical obligation that comes with agreeing to write a letter of recommendation can't include falsely praising the work of the student when the recommender believes otherwise. On the other hand, a letter writer who is lukewarm or worse on the student's work should decline to write a letter. Thoughts from readers on these issues?

    Reply
  24. Fritz Warfield

    Appalled_Grad-Student seems to misunderstand the power of at least most placement directors when writing this:

    "And if a department invests its placement director with powers of review over recommendations by faculty, isn't it a breach of trust (and professional ethics) for that person to allow the faculty to write "damaging" material (whether true or not)?"

    That a department gives a placement director the power to "review" letters of recommendation does not typically mean that the placement director is in a position to forbid faculty from writing what we want in letters. Review power is not strong editorial power.

    Reply
  25. Anonymous Job Seeker

    It might be a good time to remind search committees of the wise comments by Professor DeRose, Professor Stanley, and others, here.

    http://leiterreports.typepad.com/blog/2004/12/the_thread_on_t.html

    In my view, they give compelling reasons why pedigree and high-profile letters should matter a lot less, and why publications and writing samples should matter a lot more.

    Reply
  26. Appalled_Grad_Student

    Fritz…fair enough. I wasn't thinking of a "line item veto" power being had by the reviewer. But what I was thinking was that, at the very least, one would think that the reviewer would and (as far as I can see should) be able to inform the student (i) that professor x's recommendation is more likely to hurt them than help them and (ii) that they should seriously consider other sources for the recommendation. It just seems unethical to me that a reviewer would passively allow a bad or "damaging" recommendation to go out in the student's dossier without at the very least asking the student to seek another recommendation, assuming of course that point of reviewing is to help the student perform as well as possible on the job market.

    If I ask professor x for a strong recommendation, and x agrees to write one for me; I think it is a reasonable expectation that x will not write a recommendation that they know will damage my chances for employment. And I think if professor y, acting as a reviewer of the dossier, tells me that everything in my dossier is in order, then I have a reasonable expectation that nothing in the dossier is likely to cost me a chance at employment. If these are not the norms, then, with all due respect, what is the point of asking faculty for recommendations or asking them to review the dossier in the first place?

    Reply
  27. Zena

    On the original topic: it is my understanding that if one uses rankings in letters–and I find them useful, and sometimes use them myself–one ought use them only if it helps the candidate.

    That does not mean that one can conclude from the lack of a ranking in a letter that a candidate is poorly thought of, because it is obvious to me that many letter-writers hold Simon's view that such rankings are useless. There are other ways of recommending.

    However, I have read letters where a ranking was included that sabotaged the candidate. I can see no reason for including such a ranking except that the letter-writer thinks we all may benefit from his or her instruction on the relative merits of various philosophers. To sacrifice one's students for this sort of motivation strikes me as reprehensible. If I were a placement director, I would use any legitimate means in my power to get the letter-writer to remove such a ranking from the letter.

    Reply
  28. anon

    a nuanced appreciation by the writer to explain why the candidate would fit well with *my* department.

    Something like this has come up on many threads about hiring- a person in some department wants a real account about why the candidate would be a good fit for his department. This is understandable, but I'd like to think that anyone who thinks about it for a minute will see why it's also unreasonable. Many job applicants are applying for 50-100 jobs. Even for applicants applying for, say, 25 jobs, there is likely to be a good deal of variety among the jobs. The applicant, at least, might be able to say some small things in the cover letters, but it seems to me that asking letter writers to write specific letters for particular departments, or even types of departments (how many? 2? 3? 4?- there are at least that many "types" of jobs that a normal candidate might apply for) seems deeply unreasonable. Surely it's better to write a single detailed letter that mentions, say, research, teaching, and the type of colleague one would be, and then for the school to focus on what it thinks is important. We'd all like this to be about _us_, special person or department that we are, but I'm sure a moment's reflection will lead us to see that that's not really possible, and that demanding it is unfair and unreasonable.

    Reply
  29. Jeff McMahan

    The appalled graduate student appears to assume that letters of reference have only one purpose: to benefit those who are applying for positions. But they have a second function, which must originally have been their sole or at least primary function, which is to enable departments to make sound hiring decisions. If all letters of reference uniformly expressed only the highest forms of praise, they would completely fail to fulfill either function. The usual way in which letters helpfully discriminate among candidates is through the delicate calibration of the degree of praise. Thus, as everyone knows, letters can be damning to varying degrees through the faintness of their praise. But a letter that is positive overall, or even very strong, can still include explicit qualifications or reservations. This is sometimes necessary if the letter is to fulfill the second of the two functions they must serve. So letters can indicate reservations both by what they don’t say and by what they do say. And just as it is not the placement director’s prerogative to ask a writer to add praise that the writer believes is unmerited, so it’s not the director’s prerogative to ask a writer to remove a qualification or reservation that the writer believes is merited. Of course, if a letter is so negative that it will torpedo the candidate’s applications, the placement director should intervene. Brian Leiter is right that in such a case, the writer ought never to have agreed to write on the candidate’s behalf. Hence cases of this sort are rare. I have seen only one instance of a clearly fatal letter. The writer had actually gone sour on the candidate’s field rather than on the candidate, but the letter was lethal nonetheless. I consulted with the candidate’s supervisor who persuaded the writer to allow the candidate to use an earlier letter written before the writer had developed a jaundiced view of the field. In general in such cases, I think the placement director ought to ask the writer to withdraw the letter and notify the candidate to that effect. Failing this, the director ought to alert the candidate that there is a badly damaging letter in the file and request the candidate’s permission not to distribute it. Having never faced this problem, I haven’t thought it through thoroughly, but I’m inclined to think that the director ought to notify the writer that the letter will not be sent but ought not to reveal the writer’s identity to the candidate without the writer’s consent.

    Reply
  30. Anon. Junior Faculty

    To appalled grad student, in particular, but this may be relevant for more:
    I found out after I was rejected from a school that one of my letters (it was not specified which) contained a single line that gave a hiring committee concern, even though I was otherwise under consideration at that school for an APA interview. It was not a terrible line, but it did express a limitation. (For the record, I only know of the one school that this bad line effected me at (but I imagine there was probably more).)

    I figured out which letter writer it was (or I have a well-educated guess…see below). Now, because 1) it was not terribly bad, 2) it was what the writer believed to be true (given the writer's honest disposition), and 3) the rest of the letter was good and detailed, my advisor did not ask the writer to change it.

    I want to emphasize this: going in, I did ask each writer if they would be comfortable writing me a good letter. The writer in question said, yes, but that it would be honest. He was not comfortable elaborating on this. I took this to mean that it would not contain any gross exaggerations of my strengths and that it may identify a limitation or two. As I worked very closely with this letter writer, it was clear that he was, indeed, aware of my weaknesses (or at least how I may not match up against "the cream of the crop").
    I share this anecdote to emphasize that the reality for many letter-writers may be quite similar. They may think: this is a good candidate, who deserves a job and will do a good one, but I'm not recommending him or her as if s/he is in the top bracket, because that is not the case.
    So, appalled grad student, If I'm right that this may be common, then when a letter writer says to the candidate something like the above, should you just automatically go and find another writer (i.e., one who will not express a single limitation)? I don't think so, and I clearly didn't here, and that's for many reasons (at least because this letter writer was very familiar with my work (indeed second only to my advisor) and his areas were closer to my areas than anyone else's). There could be many good reasons to still use the less-than-perfect letter, and thus many reasons for a placement director, like McMahan, to not request changes.

    So I didn't get that job, and got a different, non-TT job later that year. Before I got my current TT job, I did stop using that letter writer as a reference (which worked out nicely since I would now need a letter from the Chair of the Dept. where I was then employed). (For the record, I'm still friends with that letter writer.)

    But here's the rub: it's quite possible that the system worked here. I didn't get a job that maybe I wasn't (at that point at least) cut out or ready for. It's very difficult for me to tell, but I'm just pointing out my letter writer's honesty here (and my advisor's letting it pass) may have functioned as it's supposed to: it kept me out of the programs that I wasn't ready for at that time, and directed me towards programs that were more appropriate for me (which is maybe the case where I'm at now). Not certain, of course, of this. But it's quite plausible.

    Reply
  31. YA job candidate

    Where does the inflation come from, anyway? Surely in large part it has to do with the same thing that leads undergrads and high school students to think that any grade under an A- is the kiss of death for their ambitions, if they're ambitious: the extraordinary competition for places and the zeal with which the best, as opposed to the good, must be pursued?

    Rebecca Kukla's hypothetical letter above is a case in point: there's surely nothing wrong with a grad student whose dissertation makes a "small but useful move" in some area, and such a candidate, while merely "solid", is nevertheless solid. Perhaps those who were around in the halcyon days of the 60s can say whether letters were more like that then; it seems plausible, anyway, and I would guess that if the numbers of jobs and candidates were more even, letters with that level of candour would be commoner. When there are gobs of applicants for every job, there will be some pressure on people writing letters (who I'm sure want to do right by the people they're writing letters for, however much they may also want to help the hiring departments make good decisions) to pump up the praise to help their students stand out. The same thing will take place in graduate study generally; it will be (or has become) not just the time when the student writes her masterpiece and gains admittance to the ancient and honourable community/guild of scholars, and focuses on doing that; it will also be incumbent on dissertators to publish, which was formerly what they were expected to start doing on becoming faculty, because they'll be competing against hordes of graduate students, and also against hordes of VAPs, adjuncts, and Fellows who've had more time to burnish their c.v.s (not that publishing becomes any easier when so many are doing so because it's necessary just to get placed). It's a buyer's market, so the buyers can demand more and more, even if they'd likely be satisfied with whatever came down the pike.

    None of this embodies a suggestion for how to read such letters, or what to replace them with, or what to do about them, or anything like that (and obviously philosophers can't summon more jobs from the vasty deep just like that), but these problems of colelctive rationality have an origin, which it is likely worthwhile to look at in itself. (I don't know if the previous thread on this topic does so.) I think this partly because I think this kind of meritocratic pressure is bad for philosophy as a whole, but it also seems like a potentially useful perspective.

    Reply
  32. David Hildebrand

    anon takes exception to my desire to see "a nuanced appreciation by the writer to explain why the candidate would fit well with *my* department."

    I appreciate the scale of many job candidates' applicaton process. I was on the market myself for 7 years. What I'm basically suggesting is that letter writers, if they can, try to avoid writing letters pitched at the wrong level. That does not seem deeply unreasonable to me. (I asked my letters to do this and not one balked.) Again, I'm not asking for the moon, merely that letter writers consider writing two (just two) versions of their letter. Sure there are more types of school than this, but the basic division of teaching/research is a sensible way to make this work. Anon's suggestion (just one letter) does sound a lot easier for the writers, but I'm speaking from the hiring side: if a letter can be pitched more carefully, it's worth doing.

    P.S. I'd add that I see more than a couple letters that actually take care to say something specific on behalf of their candidates. It definitely strikes a chord that someone is trying to help find a match for both us and the candidate.

    Reply
  33. Anon Graduate Student

    Can I ask about what happens in the case if you attend a non PGR-school where there is no placement director? If there isn't as much oversight as other programs with more resources, then certainly that might be why it is so common. But, there are other more probable reasons. I assume there are many competing graduate programs that sate the niche of producing teacher-scholars and not researchers. The above debate seems to presuppose the SC work of a research position. Moreover, if there is prima facie evidence to the claim of letter inflation, then doesn't that have to do with the desire of faculty to place their students well both for reputation of the PhD program producing the student and based on the fact that there is so much competition already for jobs that there will always be the "noise" of letter inflation?

    Reply
  34. Alan Nelson

    Looking over online discussions of philosophy hiring one finds asserted: 1) Interviews provide unreliable information, 2) pedigree provides unreliable information, 3) journal pedigree provides unreliable information, and now 4) letters provide unreliable information. It is usually allowed that writing samples do provide reliable information (though that could be questioned). But no hiring committee can study hundreds of writing samples.
    Perhaps few are asserting all of 1)-4), but it does seem that some would advocate that jobs be assigned by lottery.

    Reply
  35. Tom Hurka

    A strong application is an organic unity, whose different parts reinforce each other.

    If you find that the writing sample has certain specific merits and the reference letters praise the candidate for just those merits, then that 1) helps confirm your judgement of the sample, 2) is an indication that the letters are probably reliable, and 3) suggests that the same merits may be found in the candidate's work more generally, since the letters are based on reading much more of the candidate's work than you have.

    For this kind of reinforcement you need letters, but they have to be letters that say specific things rather than just heap general praise. General praise is indeed useless, but many letters do more than that.

    Reply
  36. Anon Job Candidate

    I think that Alan Nelson's lottery comment was made in jest or out of frustration, but I must confess to having had the same thoughts before. I've met many people who are job candidates, from both "good" and "not so good" departments, and I must say that I have been left with the impression that the job market is more or less in a Hobbesian state of equality.

    Which leaves us with an empirical question, no? Why not throw out all the obviously unqualified applicants (no Ph.D., Ph.D. in the wrong field, materials missing from the dossier, etc.), and then run a lottery? Or run a lottery for which candidates to interview? I wouldn't be at all surprised if it produced results every bit as good as a regular search. I'm not advocating folks do this, but I'd be very interested in the results if a few departments tried it.

    Reply
  37. ben w

    "it does seem that some would advocate that jobs be assigned by lottery."

    A friend of mine advocates more or less exactly that. (He doesn't advocate a lottery explicitly, but he does think that the current setup gets good-for-the-most-part outcomes because any setup that filters out the obvious clunkers would get good-for-the-most-part outcomes, because the candidate pool is, for the most part, good.)

    Just think of the psychological benefits to candidates a lottery would have!

    Reply
  38. Ken

    There's a fundamental fact about letters of recommendation that, in a highly competitive job market, means that letter inflation is almost inevitable. The fundamental fact is that they are letters of recommendation. They are not neutral, third party letters of evaluation, of the sort we seek in tenure cases. I have noticed no inflation whatsoever in the many tenure letters I have read or the lesser number of tenure letters I have written.

    Though letters of recommendation are partly (and only partly) intended to serve hiring committees as sources of information, they are primarily intended to serve the interests of the candidates on the market. If I agree to write a student a letter of recommendation, I am not agreeing simply to offer a third party, neutral evaluation of the candidate so that the hiring committee is maximally informed about all comers and can make the wisest choice. This is in pretty stark contrast to tenure letters. As long as that is the case — and I think it will always be the case — it is almost a theorem, I would think, that the more intense the competition for jobs is, the more likely letters are to be inflated.

    I am not saying this is a good thing or a bad thing. It's just there. We who write for students are trying to get them placed — as well placed as they reasonably can be expected to be placed. Of course, there are two countervailing pressure to not over-inflate letters. First, I think most of us do think about where a student can reasonably expect to be a plausible candidate and that to some extent shapes how we present the student. Plus our letter writing career typically spans many years. And you don't really want to get a reputation for over-inflating letters, because that lessens the value to your students of your letters.

    But I don' t think the pressures to over-inflate and the countervailing pressures not to are equal. If just a few overinflated letters get your students well placed, even if they don't turn out to be god's gift, that alone ups the value of your future letters.

    No practical solution here, I know. But perhaps food for thought.

    Maybe in a morally perfect world, we would replace letters of recommendation by neutral third party evaluations of the sort we seek in tenure cases. British letters used to be more like that in the late mid to late 80's when the first real flood of Brits starting applying for American jobs. But as far as I can tell many many Brits have mastered the art of writing inflated letters for the American market.

    Anyway, I can't really our culture of letter writing for entry level jobs changing to match our culture of letter writing for tenure. Again, not saying it's a bad thing or a good thing. Just a thing.

    Reply
  39. Michael Kremer

    (1) Related to much of the above: when one reads letters of reference that are over the top in their praise of a candidate's research, and then finds on reading the writing samples that in fact the work constitutes what Rebecca Kukla's mythical letter describes — competent, makes a small original move contributing to the debate, etc — there is an inevitable sense of disappointment that can work against the candidate's prospects. That is a bad consequence of inflated letters, for the candidates themselves. (I hasten to add that I agree with others that students who produce the kind of work described in Kukla's hypothetical letter can indeed be entirely deserving of consideration for a position.)

    (2) More tangentially related: the inflation of letters is part of a more general application arms race that seems to me to be running out of control, perhaps exacerbated by the relative cost-free nature of online application. Recently I have read an application running over 300 pages in total length; an application with 10 letters of recommendation; an application with 6 writing samples. Surely there is some point of diminishing marginal return here — the additional information does not strengthen the application in any real way since search committees have limited time, and can in fact start to look like padding. I have no idea what to do about this (though individual institutions could write ads that said things like "no more than 5 letters of reference" and "no more than two writing samples, withe one marked clearly as the primary writing sample," etc).

    Reply
  40. Eric Schliesser

    Dear Michael,
    The UofC online system actually accommodates the submission of three publications and two writing samples, as well as an addition writing document. I can easily see a panicked job-applicant infer the wrong message from this.
    Eric

    Reply
  41. Michael Kremer

    Eric: Good point. In fact I think the egg has landed squarely on my own face. I now realize that all the materials included in the application with the six pieces of writing are really our own fault. Our ad specifies that optional elements in the application can include up to one additional writing sample and up to four publications. In my view an ad like that can only serve to cause candidates to think they should submit as many pieces of writing as possible.

    I hasten to add (especially given this fact) that I have not held it against candidates that they have submitted this additional material. But I would like to see some way to keep the size of applications manageable.

    Reply
  42. Brian O'Connor

    Two points, if I may.

    (1) We are really complaining here that inflated letters of recommendation form an unwelcome part of the negotiation process: we are forced to try to separate what is true from what is not whilst remaining fair to the candidate. All we want, though, is a philosophical account of the candidate’s contribution to philosophy (or evidenced capacities). With some effort those letters can be (have to be) negotiated.

    I have, however, found teaching recommendations significantly harder to decipher. Forgive me for saying, but it appears rare for a U.S. based candidate — uniformly a "terrific pedagogue" from TA level upwards — not to have won some University teaching prize.

    (2) There is also the phenomenon of the well-timed lobbyist's letter, arriving after the short-listing process. I have received two to this effect: "Congratulations to you and your SC for your wisdom in selecting X for interview. If I may add to the enthusiastic testimonials he/she has no doubt already received from those who know his/her work best…" These letters prompt an unattractive image of the candidate as standing in the middle of a network that inequitably roars into action whenever one of "its own" stands to gain.

    Reply
  43. Tim

    Ken is absolutely correct. I'll state the obvious here: there are too many students on the market and not enough jobs. It's not clear if it's a failing of philosophy departments to expand faculty sizes or be more influential in a general university environment, but there are a substantial number of highly qualified PhDs who are unable to find work. I do think it's cynical to blame philosophy students and refereeing faculty for their desire to place hard-working students in positions that never existed.

    It's a natural course for a recommender who has put in 6-10 years of cultivation to want to see their student obtain some level of stability by his or her early 30s. It's a guarantee that even if a majority of the field agreed to amend letters to be more accurate, realistic, or what have you, that other writers would still inflate letters, leading to an unfair disparity in recommendation content.

    The best way to approach this question is to commence a strategy for how the field may increase its visibility among undergraduates, leading to higher enrollment, and thus more positions.

    Reply
  44. Ken

    A test for letter readers. Suppose among the batch of inflated letters, there occurred an outlier set of letters for one particular candidate such that the letters in this set were completely honest and straightforward in their collective evaluation of candidate x. Suppose, to use something like the example cited above, they were of the candidate x, who is a smart and competent philosopher and an able teacher, makes a small but interesting contribution to debate D. Would you or would you not take such letters, in the current climate, as a kiss of death? Would you or would you not, upon reading work so describe, and finding it to be accurately described, set it that file aside and move on to others?

    On the other hand, suppose the same file, more hyperbolically described, but by letter writers with a track record of placing strong students. How would your reaction to the very same work vary as a function of the way it is described by the letters?

    Reply
  45. Eddy Nahmias

    To emphasize what some others have said, I find it helpful to see that letter writers can explain clearly and concisely what the candidate's philosophical projects are about and what their significance is. If my own reading of his or her writing sample(s), research statements, etc. then jibes with that of the letter writers, that is good evidence that they have worked closely with the candidate and their assessment of him/her is likely accurate and useful, and it reinforces my own assessments. I find specific comparisons with others very helpful, especially from reliable letter writers who make such comparisons regularly. I also find it useful when reliable writers provide specific information about the teaching qualities and personal/professional qualities of candidates–this information is as much as one can hope for to glean from letter writers information about how well a candidate will fit in one's department.

    Reply
  46. Anonymous Job Candidate

    Given the myriad problems discussed here with letters, why aren't more people here seriously discussing the possibility of dispensing with letters altogether in the application process? Sure, there are problems with every element in the dossier, but to say there are problems with each of these is a very long way from saying there are equal problems with each of these. It seems painfully obvious to me that the types of problems caused by over-reliance on letters–which include the most egregious forms of cronyism, elitism, and favoritism–far surpass the types of problems associated with reliance on other dossier elements, such as teaching evaluations, publications, and writing samples. Moreover, many in this discussion thread seem to think there is no way of getting rid of the problems caused by inflation, bias, and the like.

    Do we really need letters at all? If so, why do they seem to play such a major role in interviewing and hiring decisions?

    BL COMMENT: Perhaps this discussion is giving the wrong impression, but I think letters, for all their problems, can be very helpful–and this is, in part, because there are notorious offenders when it comes to letter inflation, and there are folks who are extremely reliable. Plus one gets experienced at deciphering the code in which letters are written, and some of them really do perform the valuable tasks mentioned above, such as explaining to a non-specialist how the candidate's work makes a contribution to an existing literature or problem. No hiring committee can read every writing sample submitted, so it's unrealistic to think there won't be screening devices used, and letters are among the better ones. Or so it seems to me!

    Reply
  47. Junior Professor on the Market

    If the best that can be said in favor of letters is that they are written in a code that experienced faculty can decipher, then the situation is pretty dire. What about those writers who are unskilled in using this code, or readers who are bad at deciphering it? I am admittedly naive here, but if I were on a hiring committee I would be inclined to pay no attention to letters at all. Is it really that hard to look at a candidate's CV and research statement, then read the first few pages of a writing sample in order to see if it is worth spending more time on their dossier (which ideally would include one or two other writing samples that could be read to get a better sense of things)? Would this be any more time-consuming than trying to decode reference letters?

    BL COMMENT: I assume msot search committee members do lok at the CV. Readings the "first few pages" of a writing sample is going to be worthless, unless the candidate is illiterate, or makes some howling mistake early on. Often, after 10 or 15 pages, one can draw some conclusions about the quality of the work, but there is no way any committee has the time to do that for all applicants. If there weren't letters, then committees would rely even more on PhD program as a proxy for quality than they do already.

    Reply

Designed with WordPress