Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. Fool's avatar
  2. Santa Monica's avatar
  3. Charles Bakker's avatar
  4. Matty Silverstein's avatar
  5. Jason's avatar
  6. Nathan Meyvis's avatar
  7. Stefan Sciaraffa's avatar

    The McMaster Department of Philosophy has now put together the following notice commemorating Barry: Barry Allen: A Philosophical Life Barry…

What Kind of Obligation do Philosophers Have to Referee Papers?

A philosopher writes:

I've recently been talking to Journal editors in various fields, who all lament the difficulty of securing quality referee reports.  Certainly, authors can often be heard voicing the same complaint, and one wonders if this is implicated in the dearth, alleged in the recent Chronicle article, of Journal submissions by senior academics (who often have "invited" placement options).  My impression is that the amount of refereeing people do varies wildly, and it occurs to me that it might be useful to attempt articulating some disciplinary norms in philosophy, to help people determine what would count as "pulling one's weight."  To take an arbitrary example, an appropriate workload might be 3 a year for untenured people, and 6 a year for tenured people, with standard administrative workloads. But for all I know the number should be lower, or higher (perhaps the latter is more likely, at least for tenured people). 

 

So: How much refereeing do people *actually* do, and how much do they think people *should* do in a typical year?

Because I have edited a journal since 2000 (Legal Theory until recently, Oxford Studies in the Philosophy of Law currently), I probably do less refereeing than some, since I already have a lot of referee work connected to my editorial duties.  Still, I probably do 2-3 referee jobs for other journals each year.  I have no idea what the norm is.  Comments are open; signed comments preferred, but you must at least use a real e-mail address (which will not appear)–if you don't want to sign the name, at least indicate something about your professional position (e.g., 'full professor,' 'assistant professor,' that kind of thing).

Leave a Reply

Your email address will not be published. Required fields are marked *

45 responses to “What Kind of Obligation do Philosophers Have to Referee Papers?”

  1. This academic year I have so far refereed:
    -10 journal articles
    -one book, one book proposal, and one book series proposal
    -one NSF grant
    -53 conference papers (most as a member of an APA program committee)
    -and two outside tenure cases.

    That's a bit more than usual for me, but — except for the conference papers — not an atypical year.

  2. Professor Gross's comment casts the net more widely in terms of refereeing than the original question, which concerned only paper refereeing, but perhaps these other kinds of refereeing should be included as well.

  3. In my view, one should be willing to referee at least twice as many papers one submits for publication, since each paper one submits will typically be reviewed by at least two reviewers. I typically do a bit more than this.

  4. Peter Carruthers

    Last calendar year I refereed 4 books or book proposals and 18 journal articles. That's about my normal.
    Just as important as the duty to referee at all, however, is the duty to do one's refereeing promptly.

  5. Over the past five years or so my refereeing averages are as follows:

    around 15 journal articles a year;
    1-2 book manuscripts/year;
    and 2-3 tenure cases/year.

    There has been little variation from year to year during this time.

  6. I strongly suspect the "refereeing superstars" are posting so far, so don't let the admirable service of these folks discourage others who undertake more mdoest loads from posting.

  7. I'd like to echo Edouard Machery's point; because people earliest in their careers cannot be in a position to referee nearly to the extent that they place a burden on refereeing, those of us who are in a position to referee need to referee, on average, significantly more than the burden that we place on refereeing. Anyone who is in a position to referee and does not take on at least close to twice as many assignments as papers that they send out is not even covering for themselves, let alone "pulling their weight" in the overall burden that is placed on the field.

    I've personally been out of graduate school a little under five years, four as an assistant professor and one as an associate with tenure. Counting resubmissions, which in some cases are less work, I've refereed 60 journal articles during that time, not counting the PPQ, for which I've handled another 40 cases (sometimes only to refer them to another referee). That workload has increased significantly over that time.

  8. I'm not really sure how much refereeing I do, because I don't keep track it and it comes in ebbs and flows. I'd guess I've agreed to at least 6 jobs in the past 7-8 months. But I wouldn't be surprised if I've inflated that number in my mind a bit.

    My basic rule of thumb is that I accept the gig if I can knock it out within two weeks. This rule is supplemented by some sense of a balance between what I'm putting in and what I'm getting out. If I've done a lot of refereeing and not much submitting, I'm more likely to say no. If I've done a lot of submitting, I'm more likely to say yes.

    Edouard's principle seems about right, although I'd push for something like a 3:1 ratio.

    And, I'll second Peter's point about promptness. I think it is really insane that people take as long as they do to referee things. My colleagues in other fields, especially the sciences, are invariably stunned to hear how long we take to get things refereed. It is not like waiting months to look at the paper somehow makes the read go faster.

  9. Mark van Roojen

    I do about one journal paper per month on average, and a book manuscript each year. Recently I've also been doing tenure and/or promotion cases at an average of one per year. There's also been some conference refereeing here and there, but that's too sporadic to say what the amount is on average.

    Trying to write the sorts of reports I would like to get makes these a significant use of my time resources, but I'm not a fast reader of philosophy.

  10. I refereed 7 papers, three book proposals, and two grant proposals in 2007-08. Now, like Brian, I'm editing more and, perforce, refereeing less. (I'm also now on the AHRC Peer Review College in the UK, so about to be refereeing a lot more grant proposals.)

  11. Christopher Hitchcock

    Just guessing, and without counting, I would say that I average 8 – 10 papers/year, 2 – 3 book proposals, 1 – 2 grant proposals, 1 – 2 tenure cases, and a book MS about every other year.

    My sense is that how much I referee is not really a function of the goodness or badness of my heart. I almost always agree to referee something if I am a very natural choice for the paper — I can give good feedback and make a sound judgment. Usually this means that I referee papers on causation. I almost always decline to referee papers that are outside my intellectual center of gravity, even if I might be qualified to do so. In particular, I co-wrote one paper on the doomsday paradox 10 years ago, and still get asked to referee papers on the topic. Most of these make the same three mistakes over and over again, and so I have simply given up.

    I make a strong effort not to turn down requests for tenure evaluations, since sometimes refusals to provide such evaluations are taken as negative indicators. I once declined to write a letter for someone who was being with tenure at another school when I was ill. Even then I made a point of telling the dept. chair that my refusal was based on purely external factors, and not out of lack of respect for the candidate's work.

  12. “it might be useful to attempt articulating some disciplinary norms in philosophy…”

    To that end I would highlight another feature of a norm that is distinct from the quantity of papers one referees, and that concerns the *quality of referee* one thinks they would make for a particular paper.

    So I myself adopt the following norm (which I did not consciously formulate until I thought about in response to this question): “if I am asked to referee a paper that addresses a topic on something I am working on and/or know very well (e.g. a debate I published on in the past, etc.) I almost always agree to referee the paper”.

    The reasons for this are threefold: (1) my input as a referee is usually more valuable to the author, editor and readership of the journal (2) the cost to myself of refereeing the paper is usually low (because I know the literature well and, at a minimum, I usually learn something new and interesting when refereeing such papers) and (3) given the, shall we say, “oddity” of some of my research interests I can appreciate that there might not be a large pool of competent referees for such a paper so I find it hard to say “no” to an editor when asked to referee such papers.

    Bringing this all together we get something like the following “rule of thumb”: “the greater the fit between a paper and your expertise, all else being equal, the greater the duty to agree to referee that paper”. But I would add two provisos.

    (1) It does not follow that one should never agree to referee a paper that addresses issues more peripheral to their main research interests. I often do referee such papers if I have time (but if I am overcommitted I sometimes "pass" on such papers and feel less bad about it because I assume others are more suitably positioned to referee it than I). Getting feedback from someone a bit removed from a debate can be very valuable. So our norm for refereeing should not play the “epistemic competence” trumps all concerns of "epistemic diversity" card. If one has time, they should agree to referee papers outside of their main areas of research (provided of course they feel they do meet the minimum competence to referee the paper).

    (2) Depending on how broad and popular one’s areas of expertise are, many people will not be able to agree to referee everything they get requests for in their main area of expertise (in other words, the “all else being equal” does not always hold). In this case one needs another rule of thumb of the sort mentioned by others above (e.g. a limit on the number of papers you agree to referee).

    My point is just that a defensible norm for refereeing should track other things besides the sheer number of papers one referees.

    Cheers,
    Colin

  13. Can I also put in a plug for referees to attend to our refereeing quickly? I know that PPR and Nous now ask referees for a turnaround time of 2 weeks. One might think that this asking alot — until you remember Manuel's point ("It is not like waiting months to look at the paper somehow makes the read go faster"). In addition, in many cases it would only be a slight stretch to say that the careers of younger philosophers hangs in the balance: they won't be taken seriously as job candidates until they have a few publications under their name, and they won't get tenure until they have even more. I really think all of us, especially those post-tenure, should feel a strong sense of professional responsibility to do our refereeing duties promptly, and often. (Sorry for sounding so much the scold.)

  14. István Aranyosi

    I have refereed 7 papers for 4 journals in 2007-2008; none so far this year. I refused 3 request – they involved papers whose topics I didn't feel competent with (I wondered why they were sent to me in the first place), and I suggested alternative referees. The deadline for submitting the report was invariably one month (which I always kept, with one exception, when I was on holiday when receiving the request, I let the editor know this, and I sent the report in 6 weeks rather than 4) but it rarely happens that what I submit gets a first editorial opinion within 2 months (the longest was 12 months for a rejection). So, if my case is not atypical, the long average turnaround times are probably not completely explained by referee slowness.

  15. I've done 3 this year so far. Last year I did 2 or 3, and 4 the year before that. I graduated in 2007.

  16. Gualtiero Piccinini

    Besides quantity and speed, there is quality. When I write a referee report – regardless of whether I recommend acceptance, revise and resubmit, or rejection – I make a sincere effort to give the author useful information and detailed suggestions on how to develop and improve their work.

    Not all those who refereed my papers did the same for me.

  17. Let's see, over the last three AYs I've refereed four books, about 30 articles, half a dozen conference submissions, and a few book proposals.

    When calculating the appropriate ratio of journal system load to contribution, it's worth bearing in mind that grad students do submit papers to journals but do not referee.

  18. I'm an associate prof, and I typically review 3 articles a year and 1-2 book manuscripts. I try to return book mss within a month, and article reviews within 2 weeks.

    Undergraduate essays and tests, on the other hand, seem to take longer for me to return. Weird.

  19. I do roughly 25-30 journal submissions (in addition to administering manuscripts submitted to the journals that I edit), 2-4 outside tenure cases, usually a book proposal p/a, anywhere between 1-5 grant application reviews.

  20. While I've recently been asked to referee several papers for journals each year (6 or 7 – and I've tended to be able to agree to these requests), for several years I was only asked to referee for 2 or 3 submissions. I'm a small fish, with a few decent publications, but frankly I'm not as well-published (or well-known) as many of those who have posted comments so far. But during those years where I was only receiving a few requests to referee, I would have been happy to do more.
    So, a related question: should editors be approaching lesser known philosophers to serve as referees more than they do? Is there a worry that if a person is not publishing a great deal that they may not have the competence to adequately referee?
    I suspect that there is a significant pool of capable and willing potential referees who are simply never approached by editors, and that instead well-known, well-published individuals receive more requests than they can manage.

  21. As an early career person (graduated last year; in the UK but currently something like a VAP) I have a number of concerns. Some of this may be ill-informed, because I don't know the procedures that well, but this is my perspective and I'd be happy to be put right.

    Last year, drawing on my dissertation, I think I sent around ten papers out (including resubmissions). If each of those required two reports, then that means accruing quite a karmic debt, as it were. As someone says above, those later in their career need to do more to cover for those earlier on (who necessarily run up a debt it seems).

    Moreover, I assume that there are lots of graduate students and VAPs who will never make it to tenured posts, but who are sending stuff out to review. So those in tenure now not only need to cover their own debt, but that of others weeded out before they could repay their own. This sounds like quite a burden on you senior people (I feel like things are tough now, but not sure they get any easier).

    As far as my contribution goes, I reviewed the one and only paper I've been asked to. It was a long one (c15k words) and I did it in 2-3 weeks, despite it being Christmas, and tried my hardest to write a helpful report, even though I was told a 'casting vote' would be enough. I did, during grad school, referee papers for our grad conference though, and to an extent this is something I want more experience of – if only because I think it will help me write papers for blind review myself.

    In a way then I'd like to echo small fish's comment about there being an untapped body of junior people out there, who don't get the requests. What is the best thing to do? Volunteer my services to lower-ranked journals?

    Also, I'm curious if all refereeing is equal. I have heard stories that suggest (some) journals will send papers they consider likely rejects (e.g. from grad students) to inexperienced reviewers, thinking it doesn't matter much, while papers from big name profs get sent to big name profs. Is this true of (some) journals?

  22. During 2008, I refereed:
    -a book manuscript for OUP
    -a book chapter for CUP
    -4 articles for journals

    During 2007, I refereed:
    -a book manuscript for OUP
    -a book manuscript and a proposal for Broadview
    -7 articles for journals

    I have probably said no to about 2 journal articles during each of those years as well. I also serve as a section editor for the Internet Encyclopedia of Philosophy.

  23. Associate professor in political science, but in political theory and therefore referee a fair amount of philosophy (across all categories– article manuscripts, book proposals, tenure, grants, etc.)

    Including assignments on my desk to be completed, over the past twelve months I've refereed two grant/ fellowship competitions (with 20+ proposals per), 11 journal manuscripts not including revisions, a book manuscript and a book proposal. Roughly an average year.

  24. Looking back over my records for the last 3-4 years, it looks like I'm at about 6-8 journal papers per year with about half as many declinations. As an assistant prof, it was more like 2-3 papers per year with almost not declinations. I also edit a book series for MIT Press (Life and Mind), and recently became an associate editor for the Australasian Journal of Philosophy, which coincides with a spike in activity but not the deluge of referee requests I had feared. Also looks like about 10 conference papers per year, a promotion or tenure case per year (and the same declined), an external thesis examination per year, 1-2 book manuscript reviews for other publishers, and a handful of grant applications each year. The most noticeable change in the last few years has been a growing number of invitations to review that I feel like I have to say "no" to.

    In addition to the criteria that others have mentioned–felt competence, something constructive to say, and likely speedy turn around (nearly always within a month, often weeks)–I also use two other criteria in deciding what to take on: past relationship with the journal or publisher, and my sense of the "journal uptake" to advice given. These amount to my sense of allegiance to the journal, and often to the folks there I'm interacting with. If a journal or editor has screwed me around a lot, either in my own submissions or as a referee, or just pretty much ignored my advice as a referee or sat on the paper I reviewed in 2 weeks for another 6 months (it happens), I'm very unlikely to accept a refereeing request from them again, at least in the near future. Correspondingly, where I have had very good experiences with journals on these fronts–they give and keep firm deadlines, keep referees in the loop, and at least show signs of listening to advice that you've just spent up to a shitload of time delivering–I am very unlikely to decline requests. I operate here both in terms of my own experience and on what I consider to be reliable hearsay. In a small way, I take this reinforce positive trends in journals doing the right thing–Philosophers' Imprint and Nous are two stars here, as others have noted elsewhere–and I guess there's a kind of punitiveness in reinforcing not so good trends in journals that can be, frankly, a pain in the arse to deal with. Since we'd all be professionally better off if there were fewer such journals, I'm not sure that this ultimately helps the profession, except perhaps through some analogue to natural selection.

  25. As a graduate student, I have been encouraged by some to submit a paper only if it has been recommended by one of my professors who works in the area of the subject of the paper. This would reduce the refereeing load of journals. Is this good advice? I don't think that all graduate students receive this advice or take it.

  26. For the last few years, I have done 1-2 refs per year, and for some excellent journals, which is great for a "small fish" (as posted above) like me. One job that required a ref for a top-tier journal took more time than I put into my own published article for same–but being challenged by the work of a good philosopher who took me to the limit of criticism of deciding whether it made the cut or not made me a better philosopher for it. It is a responsibility I do not take lightly–though I probably have not been the best ref in every situation–but I have definitely benefited from the process professionally. But with my teaching load, 4 or more a year? Forget it!

  27. I get 2 or 3 a year from good places. About every other year and a half I get a book from a good press. By "good" I mean having occurred in the top 19 or top 10 of the two previous Leiter polls. I also proofread some of my colleagues works (and I think we have an obligation to do so for junior colleagues), which has recently been about 5 papers a year. And I've written letters for two tenure files. I just got tenure so I don't know how small-fishy this is. My unpublished friends coming out of the very top schools seemed to be doing this out of the gate, whereas I've only started doing it as I accrued publications in 19/10 places myself.

    I really want to second Gualtiero Piccinini's point about the Karmic importance of giving the writer detailed suggestions about how the manuscript can be improved. I know from previous debates in this forum (and from some of the reports I've received!) that that is not seen as part of the job by many referees. . . but I think the view that it is is defensible on both utilitarian and roughly deontological grounds.

  28. I'm an untenured lecturer – my rule of thumb is that, if something comes my way and I've the time, I'll do it. Partly this is because I (despite myself) enjoy it; partly it's because I think it's the sort of thing I ought to be doing. And that "ought" cashes out in a number of ways: I ought to be doing it for my own sake – it's a good way to keep in touch with what's new; partly for the sake of the discipline – I like to think I'm playing my part in keeping it healthy; and partly it's because I think it's the sort of virtuous thing I should be doing anyway.

    In terms of feedback, I tend to be quite generous: I tend to think that even a paper I'd reject probably has something going for it, and so by being quite detailed, I can probably help to get it into the public domain somewhere. And since rejectable papers – I'd guess – tend to come from more junior writers, I think it's important to encourage them. A rejection doesn't mean that they should scrap it: it just means that a paper won't be appearing here, yet.

    Yay me.

    (Just on that last point: I remember a paper I submitted when I was just starting out came back, rejected, but with generally encouraging comments. That meant a lot. On the other hand, if you're going to point out that I've made a category error, a hint about where and what it was might have been nice…)

  29. Over the least 5 years my average was 12 journal papers, 1 book manuscript, about 5 book chapters, and about 20 conference papers per year.

    I usually read journal papers for a first time the very day I receive them, take a day or two to digest, read them a second time, and then write the report. I read them just once and then reject them right away if the papers are really bad. I decline to referee if I can't submit a report within a week, don't feel competent, or know the author.

    Benji's claim that grad students don't referee is false.

    I am not an editor of a journal, but would like to ask all editors to consider the following refereeing model:
    An author has to pay 10 units per submission. In return she is
    guaranteed a decision within 8 weeks.
    A referee is paid 5 units per report. Reports are due within 2 weeks.
    If the report is not in within 2 weeks, the referee is charged 5 units and the paper is sent to another referee.
    Authors can only submit if they have no debts. To get the system
    running, each author gets one free submission. This free submission is sponsored by the publisher or, as in the case of "Philosophy of
    Science", the corresponding association.

    Needless to say, one can think of a lot of additions (e.g. editors get salaries, authors of accepted papers get a refund or even bonus) and refinements (e.g. prices depend on the word count). In each case the numbers have to be changed a bit, but the important features remain:
    1) the system doesn't cost anything and so the journal can be open access
    2) the system is efficient and fair: the number of papers which are
    not ready to be submitted is cut down, bad referees get punished, and good referees are rewarded.

  30. This is slightly tangential, but how often does it happen that you get asked to referee a paper for one journal, reject it, then are asked by another journal to referee the same paper when it gets submitted there? Given how specialized most topics are, I would think this must happen with some regularity. If so, do you refuse to referee the paper, reject it automatically, reconsider depending on revisions or what the journal is…?

  31. I referee about 9-10 papers per year, not counting any revise & resubmits that I'm asked to look at again. I'm a lecturer (assistant prof) at the University of Manchester.

    In answer to Matt S's question: this has happened to me a few times now. I alert the second editor that I've already refereed and rejected this paper for another journal, I offer to have another look at it if they would like, and leave it to the second editor to decide whether they want me or someone else to referee the paper.

  32. Christopher Hitchcock

    In answer to Matt S., I personally do not believe in double jeopardy, so I decline to referee a paper if I have already helped it get rejected by another journal. I do inform the editor of my reason for declining, so conceivably the editor could make use of the information. I think that would be hard to do, however, without knowing the journal, or the content of my review (neither of which I reveal). Occasionally, however, I end up re-refereeing what is essentially the same paper, if the author changes the title and/or abstract, and I don't realize it is the same paper.

  33. When I decline a paper because I've seen it before (very occasional), I don't tell the editor the reason; I just decline. I think passing more information on to the editor in such cases seriously jeopardizes the chance that the paper has of being published by the new journal, and since I don't do a close enough reading of the paper second time around to determine just how similar it is to the previous version–just enough to know, ah, I've seen this before and said "no"–I feel like the author is entitled to a fresh shot somewhere else. (Who knows? Maybe my initial rejection really was too rash.) It's true that this may mean that a very bad paper ends up being put through the system again, but then again I temper this with the thought that I've certainly had referees judge a paper of mine to be very bad when, next time round (sometime with very few changes, if any) the paper was accepted by another journal–even a signficantly better journal! There's plenty of noise in the system, and even conscientious referees have blind spots.

  34. RE: Matt S's question: I wonder what the disciplinary consensus is concerning reviewing a paper you've already reviewed for another journal. In one instance, I agreed to review a paper I had recommended for rejection elsewhere but only because the second journal had different criteria for acceptance than the first. In particular, the second journal had a slightly different focus than the first (more of an applied rather than a theoretical focus) and a slightly higher rate of acceptance (10-12% instead of 5%). In that case, I recommended the paper's publication to the second journal even though I recommended against it to the first journal. (It also helped that the author had answered some of the criticisms I had made when I reviewed the paper for the first journal.) In other words, I understand my obligation as answering the question, "ought this manuscript be published in *this* journal?", not "ought it be published?" But I don't know if I'm in the minority in that regard.

  35. One might have many reasons for refereeing: getting access to fresh arguments about topics of interest prior to their (probably much delayed) publication; taking pleasure in reducing the amount of garbage that is published in the journals by rejecting unpromising work; and honing one’s critical skills by routinely bringing them to bear on criticizing and dissecting the work of one’s fellow philosophers.

    That said, I dispute that any one of us has a moral or professional obligation to referee. As a former journal editor, I know firsthand that many philosophers –including some very productive ones—make deplorable referees, just as many authors of good books make terrible book reviewers. If one knows or suspects that about oneself, surely the best thing to do is to refuse requests to referee. I also believe that, in the case of highly productive and original philosophers, it is far better for the profession if they spend their time doing their own work rather than carrying the broom to sweep up behind the rest of us. Finally, making a good argument of one’s own and detecting bad arguments in the work of others are not skills that always go together. Being a good writer doesn’t entail one is a good editor of others’ writings and vice versa; being a good pitcher doesn’t automatically qualify one to be a plate umpire. The idea that each of us owes it to the profession to referee frequently fails to reckon with the fact that the field is full of poor referees, whose highest moral calling would be to refuse ever to take on a task that they are unlikely to discharge quickly and competently.

    It seems to me that the really important ‘refereeing’ or vetting that goes on is not the official sort for journals at all but the kind of feedback networks (often web-based) that philosophers usually set up amongst themselves. If philosophers have any obligation on this front at all, it is not as referees but as authors: they should get drafts of their papers commented on prior to submission for publication by at least half a dozen first-rate philosophers in their area of specialization. In my experience, the opening acknowledgment in a paper of those who have read and commented on it is generally a better guide to its quality (or lack thereof) than is publication itself. If there are philosophers who have no circle of talented, willing and critical colleagues to draw on, they should take steps to remedy that situation.

  36. I have refereed 8-10 papers, four book proposals, and two grant proposals. I'm in my fifth year in a permanent position. In addition, I have editing duties with recent collections I have put together and a journal that I edit.

    I suspect that I am not asked to referee as much as I might others be asked to referee because I am a journal editor. However, and for what it's worth, I almost never decline.

  37. I referee almost everything I'm asked to, but I'm almost never asked. In the last 8 years I've refereed between 1-3 papers a year, except for 2005 when nobody asked me to do anything … a guy could get a complex. (And just in case you're wondering, I write detailed reports when I do referee, for which editors often thank me).

  38. I've refereed 5 journal papers so far this year and 2 book proposals. Some years I get just 4-5 total, others 7-8. I try to give quite extensive comments, since I hate it myself when I get rejected with a few sentences along the lines of 'fundamentally misguided/hasn't read x/obviously rubbish'. Every now and again I get a paper I've already refereed and I always read it again, tell the editor and either adjust my comments or not.

  39. I'm seriously concerned by the last paragraph of Larry Laudan's comment. It suggests that we judge the quality of a philosopher's work by looking at how famous his friends are. The great virtue of blind review is that it prevents that.

  40. Tom, I am sure you know that wasn't Larry's meaning! He was making the evidential claim that the quality of the philosophers who read a piece in draft is (Larry claims) a better indictor of the real quality of a paper than the journal in which it appeared.

  41. Various people have rightly pointed out that one should referee *more* than twice as much as one submits, since younger philosophers create a refereeing burden that they can't repay. Just a small point on this: It seems like this additional burden should be shared equally amongst the profession. That means that it isn't simply that one should referee, say, three times as much as one submits. That would be unfair to those who submit a lot, since they would end up taking a larger share of this additional burden, even though they haven't done anything more to create it. Conversely, it would let low-submitters off the hook too easily: they still presumably read journals, after all, and so have some obligation to take up some of the refeering slack. So the rule should be presumably be something more like: referee twice as much as one submits, plus another five (?) papers per year. The important point from this is that even those who publish very little have some obligation to referee.

  42. Alex,

    “it isn't simply that one should referee, say, three times as much as one submits. That would be unfair to those who submit a lot, since they would end up taking a larger share of this additional burden, even though they haven't done anything more to create it.”

    But surely those who submit a lot contribute more to the burden of refereeing than those who submit little. If, for example, the 10% who submit the most were to stop submitting their work, they would ease the burden of finding and writing referee’s reports to a much greater extent than if the 10% who submit the least were to stop submitting their work.

    “The important point from this is that even those who publish very little have some obligation to referee.”

    One should not run together quantity of submission with quantity of publication.

    Someone who publishes very little might nevertheless submit a lot. And someone who publishes a fair amount might nevertheless submit relatively little. It all depends on the ratio of submissions to acceptances.

    A very distinguished former colleague of mine who publishes a fair amount in refereed journals didn’t receive his first rejection from a journal until something like his 50th birthday!

    It’s not just talent that explains his phenomenal ratio of submission to publication. A large part of the explanation is that he spends an enormous amount of time and energy perfecting his writing before sending it off. Few of us can match this person’s remarkable talent, but we can all match his devotion to getting things right before sending them off to journals. If we all did the latter, then the burden of finding and writing referee’s reports would be much lower, both in terms of sheer quantity and, qualitatively, in the reduction in the amount of dispiritingly half-baked and otherwise flawed stuff we’re called upon to referee.

  43. PS to Alex: I now see that my response to your first quoted point misinterprets you. I failed to register that your point was that high submitters don’t contribute more than low submitters to the number of younger philosophers who submit work without (given their stage) also refereeing as much work. While I largely agree with that point, I’d note that people who submit a lot of half-baked stuff do indirectly contribute to the number of submissions by younger philosophers by setting a bad example for their PhD students and those who are just starting out as junior faculty. Back when I was an assistant professor at UCLA, there was (and I hope there still is) an ethos of only trying to publish what was really worth publishing and not sending something off for publication before it was truly ready. Although I was never able to come close to my former colleague’s astonishing ratio of submissions to acceptances, he and others at UCLA did inspire me to try to publish fewer, better things. As a result, I think I’ve submitted fewer and better papers to journals, and my ratio of submissions to acceptance is higher, than would have been the case if my formative years had been amidst high submitters.

  44. Thanks for the responses Mike – you're correct to think that I was talking only about the additional burden created by graduate students. Sorry about my final sentence, that was a typo: It should have read that even those who *submit* very little have an obligation to referee.

  45. As an ex-editor (ANALYSIS, for twelve years), I've been reading these comments with interest. Two quick points.

    First, it is worth remembering that referees reports are primarily addressed to editors not authors, to give advice as to whether to publish or not. Of course, if referees want to write in extenso, giving that advice in a form that can in part be passed on to authors, that's fine — but surely a work of supererogation. As an editor, I certainly asked referees to strongly prioritize giving fast advice to me (even if telegraphic), rather than delay and write detailed comments for authors.

    In fact, however, referees were quite often generous with their time: but that's because I tried not to tax their patience. For — and this my second point — why suppose that most papers *should* be refereed in addition to being read by an editor (or suitable member of the editorial board)? I can't imagine that ANALYSIS, given its place in the pecking order, is untypical: and in my time, I'd say over two-thirds of submissions plainly failed at the first hurdle for one reason or another. Editors should only use referees to get advice about the clear "possibles".

Designed with WordPress