Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. Justin Fisher's avatar

    To be worth using, a detector needs not only (A) not get very many false positives, but also (B) get…

  2. Mark's avatar

    Everything you say is true, but what is the alternative? I don’t think people are advocating a return to in-class…

  3. Deirdre Anne's avatar
  4. Keith Douglas's avatar

    Cyber security professional here -reliably determining when a computational artifact (file, etc.) was created is *hard*. This is sorta why…

  5. sahpa's avatar

    Agreed with the other commentator. It is extremely unlikely that Pangram’s success is due to its cheating by reading metadata.

  6. Deirdre Anne's avatar
  7. Mark's avatar

Should authors be allowed to opt out of blind review?

Brian Leiter

A junior philosopher writes:

The topic of reviewing has been broached on your blog before, but I’ve recently been invited to referee papers where the authors' identities are explicitly included at the front of the submissions. It seems simply obvious that this approach is open to a number of abuses, yet some journals allow authors to make the final decision.  We can all agree that practical problems emerge even when blind-review policies are in place, but this hardly seems like a good reason to loosen the reins. The practice I'm pointing to may be confined to a minority of journals, but that doesn't mean it's a minor issue: surely there should be an industry-wide uniformity for blind-review. I’m interested to hear what other readers think about this, and what, if anything, we might do about it.

Thoughts from readers?  Signed comments preferred, as usual.

 

Leave a Reply to Bill Edmundson Cancel reply

Your email address will not be published. Required fields are marked *

12 responses to “Should authors be allowed to opt out of blind review?”

  1. Ben Saunders

    I assume the main concern is that referees will be more likely to publish a paper if they know it to be by a 'big name' than if the author's identity is properly blinded.

    Unfortunately, I think it's very hard to be sure that the author's identity is genuinely unknown. There is, of course, a difference between the referee simply happening to know and the author choosing to reveal it (presumably the author expects to benefit somehow), but if the author can't put their name on the cover then they can still try to reveal their identity through less explicit means, such as self-citations.

    Perhaps, in an ideal world, the author would truly be anonymous but, given that seems hard to enforce, I'm not sure it's worth the costs. It strikes me that the most important thing is that the referees' identities are hidden from the author. Double blindness, while something to be aspired to, needn't necessarily be mandatory.

    (This, of course, raises the question of whether referees should be allowed to out their identities to the author.)

    Reply
  2. David Wallace

    I'll make the same point I always make on these discussions: the more that philosophy follows the science model of publishing internet pre-prints, the less realistic double-blind review is. (It's virtually dead in physics.) So whatever the theoretical virtues of double blinding, we need to recognise that we have to make the single-blind referee model work.

    Reply
  3. Daniel Kaufman

    I know this topic has come up here before, but having spent years as a Managing Editor and having walked around in that universe quite a bit, I have zero confidence that there is anything like an "industry-wide uniformity for blind-review," and especially when we are talking about submissions by Top Clique philosophers, whose mere presence in a journal is a major cred-booster, not to mention draw.

    Given how broken the system already is, I don't see how letting people opt out of it would hurt. Hangnails aren't much of a concern, when you've lost the other leg.

    –Daniel Kaufman

    Reply
  4. kurt

    maybe i shouldn't say this. but i reviewed a paper of a guy who was a big shot
    (and not just in his own mind). the paper i reviewed was embarrassingly bad.
    i rejected it (as did the journal), but i couldn't help but feel that i always
    had in mind who the author was.

    Reply
  5. Tim Kenyon

    It's important to steer clear of the reasoning that, because a system doesn't work perfectly, it doesn't work at all. In particular, if there's a worry that sociological elements of the discipline (roughly, the influence of big names or perceived-elite institutional affiliations) make themselves felt too much even with anonymous review, it's hard to see how it follows that eliminating anonymous review will do anything but worsen the problem greatly.

    It's merely a sensible world, not the ideal world, in which authors or editorial assistants delete all self-references, and remove identifying meta-data from files. It's not pie-in-the-sky to suppose that editors will explicitly instruct referees not to do internet searches for quoted sentences to try and find who wrote an anonymous submission. That doesn't always happen by accident, after all — presumably reviewers sometimes make specific efforts to defeat anonymization. Yet there are lots of ways to defeat or undermine various other forms of academic or intellectual integrity too; I conjecture that the reason most people do not exploit these (if indeed they do not) is because they have been appropriately professionalized not to do so.

    So maybe a bit more discipline-wide professionalization is in order, emphasizing the key role that anonymous review plays in quality control, fairness, and bias-reduction. Those are goals and values that I think we give up at our enormous peril, in terms of both the intellectual health of the discipline, and the public perception of the value of our research.

    Reply
  6. Steven Hales

    I agree with Tim Kenyon. I've never tried to hunt down the identity of the author of a paper I am refereeing. The fact that someone could accidentally figure out the author, or subvert double-blind with internet searches, is not a good argument for abandoning it. My students are capable of plagiarism, but I still discourage it and expect them to do their own work. Is that hopeless idealism? Anyway, I think one serious problem with voluntary opting out of blind review is the risk to junior scholars. Big shots will all opt out, in the expectation that their names will sway referees into giving them the benefit of the doubt and a thumbs-up report. Junior people will remain anonymous in the hopes of not being discriminated against. Which means that referees will know that an anonymous submission is from someone junior or unknown and thereby allow for the very sort of bias that blind refereeing is supposed to mitigate.

    Reply
  7. Bill Edmundson

    I had an experiance similar to kurt's. Like kurt, I don't know whether I should say this, but…. I once reviewed a paper by an author whose identity was unknown to me, and I recommended against publishing it. Many months later, I recognized the submission in the pages of the very journal I had refereed for. It was, of course, signed, and the name was a big one. A member of the journal's Editorial Board, too.

    Perhaps coincidentally, I was not asked again to review for that journal for many years. My opinion of the article itself has never changed; but I am ashamed to say that I doubt that my recommendation would have been the same had I known whose work it was I was judging. ("Revise and resubmit" is the last refuge of scoundrels.) It is possible that I was dead wrong, and that the merits of the piece were so great and so obvious that my review was justly discounted. Readers may judge from their own experience how often their work has muscled its way into the pages of a premier journal over an inept referee's flat recommendation to the contrary.

    The integrity of blind review requires blindness at the highest editorial levels. Ideally, a journal would have each submission go through an "anonymity check" by someone not otherwise involved in the decision in any way. Submissions that fail that check should not be allowed to go either to the editors or to referees.

    Reply
  8. Daniel Kaufman

    In reply to Kenyon and Hales:

    I would submit that the process is more debased than you describe. As a Managing Editor, my job was partly to actually receive and process the submissions. In short, you know who wrote the paper when you receive it. And if you think that a paper written by a rock star doesn't get handled differently, once it is received, than a paper written by a nobody, then I have several bridges to sell you.

    This may not be as much of a problem at the very top journals, whose pages are routinely filled with the writings of rock stars and who have very established reputations and readerships. But for the next tier down or two, there is often a great deal of incentive to publish rock stars, whenever they submit to you, because you too would like to become a top-tier journal.

    The point regarding "the ideal" versus "the reasonable" is out of place here. Part of the problem with the current system is that it further entrenches elites and further alienates those who are not in the elite. To suggest that we should keep it, then, is to suggest that we should allow this ridiculous aristocracy…which is based largely on publishing rates and venues…to continue. I for one, would be very happy to see a lot more papers by nobodies and a lot fewer papers by rock stars.

    –Daniel Kaufman

    Reply
  9. Peter McBurney

    To Bill Edmundson: I am editor of a journal in AI, and I disagree profoundly with your final paragraph. Someone INVOLVED in the decision needs to know the identity of each submitted paper's authors, so that the reviewers assigned to the paper are not, for example, the authors themselves, or close colleagues of the authors', or somebody known to bear a grudge against the authors. Reviewers cannot be allocated to papers by a machine – or, at least, to do so risks many other types of unfairness and injustice.

    Reply
  10. Jamie Dreier

    I think it's important to point out that this:

    "In short, you know who wrote the paper when you receive it."

    is not true at ETHICS.
    Our managing editor screens papers without knowing who wrote them. The ones that pass his screening go to associate editors, who also make their decisions without knowing who wrote the paper. And then, of course, the referees do not see who wrote the paper.

    There is some danger of an editor or referee picking up on clues in the paper itself and having at least a guess at the author; this problem worries me, but I don't see what can be done about it. But in any case, the practice that Daniel Kaufman followed is, happily, not universal. I know that Social Philosophy and Policy follows a 'triple blind' (if that's the right term) policy similar to ETHICS'. Maybe others could comment on the policies at other journals.

    p.s. The 'preview' function here is not working for me. Maybe it's just my browser.

    Reply
  11. Daniel Kaufman

    Ethics is a top tier journal. I was pretty careful to acknowledge that this is not a universal practice and that the top journals are probably less inclined to unfairly favor the big shots than those journals that would like to make it to the top. I do believe, however, that it is far more widespread that is admitted. And that comes, in part, from talking to the very people running these journals, who tend to be more candid in private than on public message boards.

    I would maintain that publishing rates and quantities also provide evidence that there's something of a fix at work, but we've had that argument here before, and there are enough other variables involved (like teaching loads, access to Graduate Assistants, etc.) to make the claim difficult to demonstrate in any conclusive way.

    Incidentally, on the more general topic of whether elites, in whichever institutions, tend to rig the systems they work within, in order to maintain their own status and keep others out, there was an interesting Bloggingheads on the subject recently: http://bloggingheads.tv/videos/10015.

    –Daniel Kaufman

    Reply
  12. Mark van Roojen

    If I have to choose between the argument that things are somewhat rigged so we should give up on trying to exclude the influence of stature on publication prospects, and the argument that things are somewhat rigged and we should try to make them less so, I think the latter position is more sensible than the former. I have trouble seeing how that isn't the obviously the right response. We know the world isn't perfect and we try to make it better nonetheless.

    Reply

Designed with WordPress