Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. Fool's avatar
  2. Santa Monica's avatar
  3. Charles Bakker's avatar
  4. Matty Silverstein's avatar
  5. Jason's avatar
  6. Nathan Meyvis's avatar
  7. Stefan Sciaraffa's avatar

    The McMaster Department of Philosophy has now put together the following notice commemorating Barry: Barry Allen: A Philosophical Life Barry…

Refereeing obligations for journals where one has published?

Brian Leiter

We've touched on related topics before, but philosopher Yuri Balashov (Georgia) poses a slightly different form of the question: 

The other day I had to decline another refereeing request from a journal where I earlier published, citing "being too busy."  Then I felt some pangs of conscience.  But I wonder if I should feel that way.  Some of us referee a lot of papers, others very few, others not at all.   Is there a minimal number N of submissions one should feel obligated to referee for a journal (i) where one published, and (ii) where one's own submission was rejected?  I think in the second case the answer is clearly N=2 (assuming the journal sends all submissions to two referees).  What about (i)?  Suppose this is a top journal that rejects 19 submissions out of 20.  Its editors may be disinclined to approach many or most authors of rejected submissions for refereeing.  Of course the editors could approach any number of other people who have never published in or submitted to their journal.  But it seems fair that the authors of published papers could be morally expected to do more work than others.  How much?  I would say that refereeing five submissions would probably fulfill one's moral obligations in such a case, and refereeing ten would be supererogation.  But I'd be curious to know what others think. 

Thoughts from readers?  Signed comments strongly preferred.

Leave a Reply to Brad Cancel reply

Your email address will not be published. Required fields are marked *

15 responses to “Refereeing obligations for journals where one has published?”

  1. Robert Paul Wolff

    However few or many papers you agree to referee, I would strongly urge all tenured members of the profession to identify themselves to the author in the review. It is difficult enough to be rejected by a journal, without having to be in the dark about who axed you. What is the point of tenure if not to encourage this tiny act of courage?

    Reply
  2. Brad

    I do not think the appropriate "unit" of analysis is the journal. That is, we should not referee because we owe a specific journal something for publishing or considering our submissions. Rather one reason we referee (and should referee) is because we are experts on particular topics, and we have an interest in promoting high quality scholarship on those topics. By refereeing we can influence what is published. This includes recommending rejections to poor quality submissions, and helping authors develop their ideas into contributions worth reading. After one publishes in an area, one becomes responsible to some extent for scholarship in the area.

    Reply
  3. Eddy Nahmias

    Perhaps this is a topic for another post, but I was very surprised to see Prof. Wolff's suggestion that tenured referees reveal themselves. I have never heard this suggestion and it seems to conflict with the practices of most journals. Is this something people do? Is it something people think should be done? (I don't have a strong opinion either way. Just curious.)

    Meanwhile, I agree with Brad that we should be reviewing based on expertise and our goal of keeping the quality of research in philosophy and our subfields high. So, we should referee as much as we are asked to review, except for when we can't because of too many other commitments or shouldn't because of conflict of interest or lack of expertise on the topic. (And the fact that you have lots of other commitments does not count as too many.) We've thrown out numbers before but certainly anyone with tenure should be doing at least 4-6 a year.

    Reply
  4. Brad

    There was a question about numbers. So far, in 2012 I refereed 11 papers for philosophy journals. I also refereed an addition 5 papers for a scientific conference. I will begin to decline invitations for the remainder of the year. I have done my share this year, and I do not want to become a crabby overworked referee.

    Reply
  5. Charles Pigden

    Berit Brogaard at http://www.newappsblog.com/2012/05/a-plead-from-the-editor.html#more has a sensible suggestion. She argues that you should referee at least twice as many papers as you submit. Her idea – and it seems to me a sensible one – is that it is your duty to the profession to do as much refereeing work as you generate. Every time you submit a paper this requires two people to write a referee's report. Ergo you should referee at least twice as many as you submit.
    There is a problem with this however. Suppose that most of your papers are accepted by the first or the second journal that you send them to (or that your invited submissions to collections are usually accepted). Then Brogaard's formula suggests that you should referee about four times as many papers as you submit. But I take it that early acceptance is on the whole a sign of quality and that if somebody's papers are multiply rejected this is a sign that their work is a bit sub par. Thus the lightly refereed person whose work is usually accepted after one or two submissions is likely to have a higher reputation than the heavily refereed person whose work is only ever accepted by the fifth or the sixth journal that it’s sent to. But having a high reputation is roughly correlated with the number of referee's reports you are asked to write. Thus the less refereeing work you generate the more likely you are to be asked to be a referee.
    This, I suspect, explains some of the problems with refereeing. A relatively small number of people who don’t generate much refereeing work are doing most of the refereeing work that other people generate. Hence the delays, hence the snarkiness and hence the superficiality. I think it’s a serious problem and I have no idea what to do about it.

    Reply
  6. Charles Pigden

    PS. I guess there is an ambiguity in Brogaards’ principle. Should you referee twice as many papers as you submit or twice as many submissions as you make? Only the second of these two alternatives guarantees that you will be doing at least as much refereeing work as you generate.

    Reply
  7. Neil

    On Robert Paul Wolff's suggestion: a referee for the Journal of Ethics once gave me the most helpful set of comments on a paper I've ever seen, and asked that their name be passed on to me. The journal refused to do so. That is the only time a referee has asked to be identified in my experience in philosophy(though referees for science journals often sign their names).

    Reply
  8. Jamie Dreier

    ETHICS (in contrast, apparently, to the Journal of Ethics) explicitly offers referees the opportunity to identify themselves to authors.

    I think the Brogaard formula needs some adjustments. If someone does a lot of editing for a journal, they can surely count that as part of their journal-related duties. Someone who does little other journal-related service should surely do more refereeing than #-of-submissions X 2.

    Reply
  9. Yuri Balashov

    A quick comment on the "Brogaard formula": suppose X submits papers to every good journal, but they never get accepted. According to the formula, X should be asked and agree to referee as much as he submits. But would it be good for the profession? If not, does it not suggest that the authors of accepted papers should referee more (and perhaps a lot more) that the authors of submissions that are never published?

    Reply
  10. David Wallace

    The "moral obligation" framing of the question seems wrong. Both my refereeing, and my writing, are part of my responsibility to advance scholarship in my discipline, for which I get paid. I'd have a moral obligation to act as a referee in my areas of expertise even if I wasn't publishing at all. (Leaving aside the question of what makes it my "area of expertise" in that case…) There's of course a question of how much refereeing I should really do, consonant with my other research and non-research responsibilities, but it's not a quid pro quo.

    Reply
  11. Landon Schurtz

    In re: the Brogaard formula, I have to ask (as a mere ABD grad student, here), how much control over how much refereeing he or she does might a professor have? I mean, presumably you have to be asked – so what if you aren't? Or, what if you aren't asked a number of times commensurate with your supposed "debt?" Isn't reasonable to assume that some people are asked to referee far more often than they actually can/should, while others (who might still make competent judges) are only rarely asked, owing to differential recognition within the field?

    Note – I'm still fuzzy on how this works, so my questions might be utterly incoherent. If so, just let me know. Thanks.

    Reply
  12. Charles Pigden

    Dear Landon. You ask how much control a professor has over the amount of refereeing that he or she does. The answer is that you can turn down requests, thus limiting the amount of refereeing you do but there is no direct method of soliciting requests thereby increasing the amount of refereeing that you do. So far as I can tell (and editors can correct me if I’m wrong) you get asked to referee a paper under one or more of three related conditions: 1) you have published something in the area that has garnered a couple of citations or has otherwise created a stir, 2) your own work is cited in the paper to be refereed, and 3) you have an established reputation as a reliable referee. You don’t have to have to have published all that much in a given area to meet conditions 1) & 2). For example, I have only one publication on truthmaker theory (a coauthored paper with my colleague Colin Cheyne), yet I’ve had several requests to referee papers on truthmakers and negative facts. Obviously you don’t get to meet condition 3) unless you have already done a bit of refereeing, usually, though not necessarily, for the journal that makes the request. But I rather suspect that I get asked to referee more papers that I would be asked to do on the grounds of expertise alone because I have a reputation among editors as a competent and conscientious referee. (I can’t claim, sadly that there are no blots on my copybook and I am certainly slower than I would like to be, but on the whole, I am fairly reliable.) All this points to a problem with Brogaard’s otherwise very sensible formula. If I understand her correctly, she thinks that as a rule of thumb, you should do at least a much work as a referee as you generate as a submitter of papers, which means twice as many referee’s reports as submissions. But multiple submitters, who can generate a huge amount of work, are unlikely to be asked to do their fair share of the refereeing, since their papers are less likely to be cited. (Of course we can all think of brilliant papers that have been rejected numerous times, but on the whole the better the quality of a paper, the more likely it is to be swiftly accepted and the more likely it is to be cited thereafter. The sad fact is that a great many philosophy papers don't get cited at all. Even for a prestige journal such as Phil Review, the average number of citations per paper after four years is usually less than one.) Thus when it comes to multiple submitters, their debt to society is likely to go unpaid – or rather it is likely to be paid by someone else. Furthermore, the realm of refereeing is a bit like the moral universe of C.S Lewis’s Narnia books. Your reward for doing one good deed is that you get asked to do another. The person who turns in a decent report without too much delay is more likely than the slacker to be asked to do it again. (Thus in the realm of refereeing, virtue had better be its own reward since there isn’t any other!) So what does this all mean? It suggests that Brogaard’s formula faces an ought-implies-can problem. You can’t report on papers that you have not been asked to referee, so there are many big-time generators of refereeing work who will never get to pay their personal debt to society. The work that they generate tends to be done by other people. Furthermore, if the refereeing show is to be kept on the road, those who do get asked (which means, on the whole, the more successful philosophers) will have to do considerably more refereeing than the Brogaard formula suggests. It's part of the price that they pay for their success.

    Reply
  13. Berit Brogaard

    Hi Charles, As you know, I already replied to you in person but I thought I'd post my reply here as well. I agree with the second formulation you suggest (or some version thereof). I also agree with most of the other concerns. If you edit a journal, surely you are not REQUIRED to also referee papers. That leaves more papers for others to referee. So some adjustment should be made for that. Then there is the further problem that junior people encounter, viz, that of not being asked. That is indeed a problem but it can probably be resolved by following my other suggestion: Leave alternative names whenever you turn down a referee request. It would be natural to mention some good junior people, as it is likely that the journal editor has already thought of all of the senior people you could possibly think of.

    Reply
  14. Brad

    Landon,
    I would like to clarify something, given your questions. Until one has published one should not be refereeing. A referee is expected to have expertise on the topic of the paper she referees. As a result, junior scholars referee less, as should be expected. In time, one gets asked to referee more, in part, because one has more publications, and, in part, because one gets a reputation for being a respectable referee. One provides thoughtful and useful comments to the editor in a timely fashion. That is, roughly, how the system works. Some people probably never referee as many papers as they submit; others do far more.

    Reply
  15. Charles Pigden

    Thanks Berit, good to hear from you! But alas, I've thought of another problem! The suggestion of substitutes is only going to bring in the junior philosophers if the first person approached knows who they are. If, like me, they live in the boondocks or at the ends of the earth and don't go to many conferences, the only philosophers in their field that they are likely to be aware of are people that have already been published. Thus the people I am likely to recommend are already well-established figures who are probably doing a fair bit of refereeing anyway. Metropolitan philosophers are in a position to spread the refereeing work around: not so backwoodsmen such as myself. Indeed I am inclined to think that you would have to be an assiduous conference-goer and a highly approachable person to be able to list a reasonable number of unpublished (or underpublished) philosophers in your area. Hopefully there are enough well-published philosophers who are also also metropolitan, conference-going and approachable to solve the problem, but I have my doubts. It's a tough one, and my personal solution is to referee at least three times the number of the submissions that I make – or rather to never turn down a request unless I am absolutely snowed under with work. But then, I am in my fifties and all my children are grown up, so I don't have many domestic responsibilities. I have rather more time to do my refereeing duty than perhaps some others.

    Reply

Designed with WordPress