Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. Binary choice's avatar

    If you were the head recruiter for a professional basketball team, would you require height and weight measurements for prospective…

  2. Jeremy Telman's avatar
  3. Roger of Invisible America's avatar
  4. Brad's avatar
  5. LC's avatar

    I concur to a certain extent. I also scored very low on the writing section. Back then, GRE was required…

  6. Mark's avatar
  7. Anonymous's avatar

    I’m a graduate student in philosophy at a well-known terminal MA program who is applying to PhDs this fall and…

Time to Boycott “Journal of Philosophy”?

Brian Leiter

More authors are reporting horror stories with J.Phil. here.  Add to this the fact that it's an in-house operation, and the fact that there are now lots of high-profile and high-quality alternatives, that are also generally efficient (like Nous, Philosophy & Phenomenological Research, Philosophers' Imprint etc.), and one wonders whether a boycott isn't the right response.  (See also the comment from Barry Lam [Vassar] regarding on-line sources for collecting information about journal practices.) 

Thoughts from readers?

Leave a Reply

Your email address will not be published. Required fields are marked *

50 responses to “Time to Boycott “Journal of Philosophy”?”

  1. Paul

    As a grad student, I haven't yet done the whole journal thing, so I am sure there is much that I do not know or understand. But the numerous discussions on this blog and others really suggest that the profession has a serious problem.

    But it seems to me that there is a collective action problem here. Tenure track professors (and grad students) are under enormous pressure to publish in big name journals. If there is some sort of a boycott, certainly it will be individually rational for those people to submit to JPhil. And as we all know, the suspicion that others will be acting on their own self-interest will inspire others to submit as well.

    Reply
  2. Keith DeRose

    The account in the main post is sobering — over 2 years for the first paper with no decision! (That it was at JPhil is revealed in comment #9.) But the experience reported by another philosopher, involving both MIND and JPhil, in comment #12 is even more amazing.

    In response to the first comment above, much of the burden should fall on older, more established philosophers. When I learned about some of JPhil's practices a couple of years ago, I decided I wasn't going to send them anything, and it hasn't been hard to stand by that. And that was before I heard about these problems with long decision times. Now that there is that self-interest consideration to throw into the mix, it should be even easier to refain. More established philosophers often get lots of invitations to publish papers in places where they don't have to go through the often long & frustrating submission process. I think it's important that they support properly (mostly this means blindly) refereed journals, and so philosophers who could easily avoid it should nonetheless submit much of their best work to properly refereed journals. But then there's little reason to submit a paper to a journal that doesn't employ decent procedures — especially if one is quite possibly subjecting oneself to an experience like those reported in these horror stories. None of this is to deny that there might be established philosophers who might find it in their own interest to send a paper to JPhil. But the sacrifice might not be as big as one might think. Of course, we should all cut a lot of slack to those early in their careers.

    Reply
  3. Neil Sinhababu

    Even if full-fledged boycotts don't result from this, many people will fear submitting papers to a journal that might sit on them for a ridiculous length of time.

    Being a young fellow myself, I appreciate Keith's charitable attitude towards those early in their careers. But junior people with tenure or promotion deadlines ahead will be especially keen to avoid having a journal do nothing with our work for two years. That could leave us with no way to get credit for it before we're evaluated.

    Reply
  4. Colin Farrelly

    Perhaps the best solution to the problem of “in-house” philosophy journals is for the members of the editorial boards in question (i.e. the faculty at Columbia and Cornell) to suggest they hand over the journals to the APA (and for the APA to accept the kind offer). So JP and Phil Review would become the official journals of the APA (just like the APSR is for the American Political Science Association).

    In-house editing might have been appropriate when these journals began over a century ago, but such practices have no place in the 21st century. Voluntarily handing the journals over to the APA to setup a new, rotating editorial arrangement would be best for the journals in question (by preserving their high status as flagship journals in the discipline) and would create enormous benefits to the discipline (by having transparent and accountable editorial practices for two of the flagship journals of the discipline).

    If the editorial boards reject this proposal, it would be interesting to know what their reasons for doing so would be (other than “we enjoy the power and prestige we currently have in the discipline!”)

    Cheers,
    Colin

    Reply
  5. Justin

    I suspect that the negative reputations of places like JPhil. and Mind when it comes to their review time, failure to respond to emails and the like will only discourage younger folks such as myself from submitting. Tenured faculty for the most part needn't worry about the delays and may choose to publish in venues like JPhil. Am I wrong about this? I also suspect that because younger folks won't send their papers to places like J.Phil. and Mind, other top venues, especially Nous and PPR, will be flooded with papers. Indeed, they already are. This is unfortunate as it places an even heavier burden on quality journals who offer prompt responses to submissions. Needless to say, it would benefit all parties if certain journals got their act together.

    Reply
  6. jon kvanvig

    Colin's idea of handing journals over to the APA is not a good one, I think. The history of the national office with respect to, for example, the handling of Jobs for Philosophers is bad enough that the idea would have to be discounted on that ground alone. Moreover, there are costs associated with journals, and I doubt there is enough confidence in APA administration at this point for members to welcome increased dues to fund such a plan.

    I agree with Keith's point that the burden should fall on more senior philosophers to engage in boycotts. I committed to such a boycott regarding several journals long ago, and will continue it. I wonder if a Hermes-like petition site might be useful to show how serious a matter this is. (Lest I be misunderstand, it is nowhere even close to as serious as is the issue involved in the Hermes petition; it is just that a united front done in a public way is better than individuals grumbling.)

    Reply
  7. Laurence McCullough

    Here’s how it works in the world of medical journals, in which philosophers, myself included, regularly publish. The first step is an internal review, usually completed in 3-4 weeks. The paper is either sent for outside review or rejected. In the latter case, the author receives a form-email immediately. The only things one risks are a short delay (4 weeks or so) in sending the paper to the next journal on one’s list and a bruised ego. Outside review usually is completed within eight weeks, sometimes faster. Responding to peer reviews, either for resubmission or submission at another journal, in my experience always results in an improved paper. One major advantage of this system is that, given the speed of internal review and its low risks, junior faculty are encouraged to start with top journals. The promotion and tenure process can therefore reasonably have exacting standards for publication venues.

    Here is the description of the review process at Philosophical Review from its website (http://www.arts.cornell.edu/philrev/info/editorial.html, accessed July 21, 2009):

    “Unfortunately, the editors are not able to provide comments or a rationale for their decision, except in the case of a paper that is being encouraged; they simply don't have the time. We hope that authors will understand and excuse this limitation, and that it is somewhat mitigated by the quick turnaround time for the initial review.”

    The editors’ statement they do not have time for other than standard rejection after internal review is true, given the volume of submissions at a top journal and their policy is in this respect is not out of line with medical journals. The amount of time such internal review takes at PR and other philosophy journals appears to be way out of line. It is also a major problem for younger colleagues for whom long delays in internal review can be very costly, a risk that is compounded by the very long review time for papers than a journal wants to “encourage” (a word choice that I have never seen from any medical journal and that drips with arrogance).

    The unacceptable review practices of some philosophy journals discourage submission to them by junior faculty. Brian’s posting raises the issue of a boycott, a fine idea provided that (1) the leadership of departments of Philosophy meet with deans to set out processes and standards for promotion and tenure of junior faculty that take into account unacceptable long delays at top journals and resultant altered expectations of department leadership and deans regarding publication venues that do not engage in such unacceptable review practices and (2) senior faculty, who can take the risks of prolonged review, nonetheless and publicly join the boycott.

    Reply
  8. jon kvanvig

    It is worth noting that the information at the philosophy journal wiki accords with the judgment that JP and Mind are the most egregious journals on this score. See here:
    http://www.wikihost.org/w/philjinfo/start#headline47

    A cursory glance, using my own quite lenient standard that 6-9 months is still acceptable, Mind responds appropriately in only 50% of the cases cited, and JP in only about 1/3 of the cases. The administration of Mind is responsible to the Mind Association, and I would think they would want to take some action on this score. It is unfortunate that the administration of JP is responsible to no higher philosophy association, so far as I know, though I expect they are answerable in some way to the University itself.

    Reply
  9. Andrew

    My suspicion is that although there may be some problems at the journals themselves, we need to devise a new way to encourage outside referees to read and comment on papers in a timely fashion.

    Here's an idea I've discussed with some colleagues: suppose that PR, PPR, Nous, Mind, and JPhil were collectively to start charging $25 per submission. That would cut down on the many frivolous submissions that are now clogging up the system. Then, assuming that less than half the submissions that remain would be worth sending out, the editors could offer each of two referees $25 to send their review within a month. What counts as a 'good' review would be up to the editorial staff.

    $25 is not a lot of money, but it may be just the incentive required for busy people to move refereeing to the front of their to-do list. Just a thought.

    Reply
  10. Evan Thompson

    I fully support a boycott of J Phil. I will not submit anything to J Phil, Mind or Phil Review because of their deplorable editorial practices. I work in the cognitive and brain sciences as well as philosophy, so I have a fair amount of experience reviewing and publishing articles in a wide variety of science journals. In these journals there are often three reviewers representing different perspectives and areas of expertise; the comments are usually helpful; the decision turn-around time is usually 6-8 weeks (often faster); and the publication is timely (with advance on-line publication shortly after return of corrected proofs). A number of philosophy journals strive to follow some or all of these practices – for example, Nous, Philosophical Psychology, Philosophy of Science, and Phenomenology and the Cognitive Sciences, to name four journals that regularly publish important articles in the fields in which I work. I have resolved to support these (and other comparable) journals and not to send my work to journals whose editorial practices are simply unacceptable.

    Reply
  11. Paul

    With respect to Andrew's idea: Right now, referees know they are being irresponsible, creating hardship for young philosophers, and contributing to other people's aggravation and needless stress. My guess is that most of the referees have some pangs of conscience about this, but not enough to, you know, actually referee articles in a timely manner. I would worry that offering them a nominal monetary "incentive" would make them feel *entitled* not to do their job. They could say to themselves "Well, I'll just forfeit my $25, and not do the refereeing that I was supposed to complete this month."

    Reply
  12. Clayton Littlejohn

    "Here's an idea I've discussed with some colleagues: suppose that PR, PPR, Nous, Mind, and JPhil were collectively to start charging $25 per submission. That would cut down on the many frivolous submissions that are now clogging up the system. Then, assuming that less than half the submissions that remain would be worth sending out, the editors could offer each of two referees $25 to send their review within a month. What counts as a 'good' review would be up to the editorial staff.

    $25 is not a lot of money, but it may be just the incentive required for busy people to move refereeing to the front of their to-do list. Just a thought. "

    This idea of charging a fee to submit papers for review comes up a lot and I've always said that I oppose it because of its effects on those of us at the bottom (i.e., those of us who are paid the least and who seem to need the publications the most). The trick is to find a way to provide the right sort of incentives and disincentives without putting an undue burden on those of us who suffer significant financial strain already because we're grad students, lecturers, or adjuncts. I doubt that it's possible.

    Reply
  13. Mark van Roojen

    Andrew,

    I think that with PPR and Nous which are as far as timeliness goes not among the journals that seem to have a problem, they partly get that timeliness by combining the two stages in a way that speeds things up. They send you a paper and ask if it should be fully reviewed. And then if your answer is yes you are supposed to say if you could do it and to suggest another who could if you could not. Given that you've already read it at this stage, it is pretty likely that if you think it deserves a review you are also going to say you'll do it (absent conflicts of interest and the like). I'm not clear on what sort of screening is done prior to this stage, but it also allows papers that clearly don't make it to be returned without comments but quickly.

    On the general matter and no longer addressing Andrew especially,

    Even with the journals wiki (which is a good idea) I'm not sure we on the outside know what the problem is. I've refereed for a large number of journals (but not the mostly in house ones) and my general policy is to meet deadlines, but that they are set by the journal editors. These have varied from 3 weeks to 3 months. I've missed a few deadlines, but none by a large percentage of the allotted time. I don't know if I am typical, but if I was I would think that the problem is not on the referee side of things.

    As I recall there was a statement from the APA when I was a young pup setting 3 months as the reasonable expectation for a response to a paper. So that is always what I have regarded as a perfectly reasonable time frame for refereeing by a journal. I don't get all that frustrated until it goes past 4 months. (And to be honest I've never waited much over 6 months that I can recall.) But I think that Jon's 6-9 month time frame is very generous.

    So what's my point? I guess it is that we should know a bit more about what the root causes of delay are before we say what exactly journal editors should do. I've heard people postulate duties to referee within a week to a month. I think that is a bad idea, at least if I'm typical. If I never referee again I will have refereed more than I have submitted. I am generally on time when I accept a refereeing request. Yet telling me I have to do it within a too short time frame will make me turn down refereeing. If there are other people like me that will decrease the referee pool. (I may not be typical.) So my thought is that that sort of specific proposal for a reform is not useful absent some showing that the problem lies with journals letting referees take 6 weeks or 2 months (the most typical time frames from my experience).

    What we know is that the current average times to response at a number of journals, including some of the historically most prominent, is beyond the realm of acceptability, and that the editors of those journals have a responsibility to identify the root problems for their journals and work to fix them. And it is not unreasonable for folks to take the reputations of journals for prompt refereeing of papers into account in deciding where to submit. Whether it warrants an organized boycott would depend partly on whether editors are making a real effort to fix things or not, and also on whether the evidence we have about individual journals is reliable. I'm a little worried that even the wiki is not entirely reliable as a window into the overall situation.

    That's a little rambling and a bit vague, but the general point is that getting too specific about appropriate remedies in the absence of more information might be getting ahead of ourselves.

    Reply
  14. Keith DeRose

    As I indicated already, I decided a while ago against submitting papers to JPhil. But I don't think that their being edited "in-house" is itself a problem, and I have no problem with the Phil Review, which is also edited in-house. The most important thing is that the Phil Review uses blind review. After that, while it *may* matter whether they're an "in-house operation," it isn't clear to me whether that's better or worse than the usual way journals are done. IN-HOUSE EDITING: Editors are extremely important. The real decisions of whether to publish papers come down to the editors, using the referee's reports as advice. At the Phil Review, the editorship rotates around among members of the Cornell philosophy department. I don't see why that would be worse than the usual way journals are edited — which I guess is usually just one person doing the job for a long old time. IN-HOUSE REFEREEING. The Phil Review also often referees papers in-house, using Cornell philosophers for the task when they have suitable experts available there — though what they report this: "Occasionally, papers are refereed by other members of the Sage School; more often, they are refereed by philosophers not at Cornell." Still, I imagine you get a good deal of refereeing done by Cornell philosophers — probably more than any small group of philosophers does for other journals, though I can't be sure about that. There may be a disadvantage in that. But there would also seem to advantages to doing a good deal of the evaluating in-house. I imagine one of the toughest jobs for an editor is finding referees who will do a good & timely job. And I imagine it helps here to have colleagues that you see every day, and who are a bit invested in a journal, as it is the journal run by and associated with their own department. I imagine it would be a little motivation to getting the job done in a timely fashion that the editor to whom you owe the report is someone you see every day & work with — and that might also provide some extra motivation to do a good job. When a journal belongs to a department and is a matter of pride to the department, I'd have to think the members of that department might do a better job of running it than would, say, referees who get an e-mail from some editor they never see and maybe hardly know asking them if they might be so kind as to do some hard work for no reward. At least, it isn't clear to me why the in-house way is supposed to be inferior. On the other hand, if such a journal were run in a shoddy way, that would reflect poorly on the department that runs it.

    Reply
  15. Christy Mag Uidhir

    Here’s an idea that if widely implemented might help control for both referee delays and submission gluts.

    Referees have six months in which to render a decision. If no decision is rendered after six months, the referee receives a warning and is given an additional three months to render a decision. If no decision is rendered after nine months, the referee incurs a 1-2 year ban on submissions to that journal. Any referee receiving two (consecutive) six-month warnings will incur a 1-2 year submission ban. Any author of two consecutive submissions rejected at the preliminary vetting stage will incur a 1-2 year submission ban.

    Reply
  16. Thom Brooks

    A few thoughts on the conversation thus far:

    1. I agree with John that no journal should be run by the APA and for many of the reasons he offers.

    2. I entirely disagree with the view that journals should charge a fee for submissions. First, there need not be any problem with frivolous submissions: if submissions are immediately identifiable as substandard, then they should be instantly rejected. This takes little time and clogs nothing. Secondly, it is bad enough that referees, authors, and even editors (all things considered) earn nothing at all from the labour they provide — why then add to the already substantial costs incurred? Thirdly, I doubt it would prove any substantial deterrent from submitting papers given the importance of publications. Finally, I doubt it could be enforced. Unanimity is hard to come by. Moreover, it is one thing to get the philosophy journals in agreement — but what about related journals that are in related fields (e.g., law, politics, classics, etc.) or that interdisciplinary?

    3. I continue to believe more journals should publish their submission and review data. For this reason (and not unlike many others), the Journal of Moral Philosophy publishes our submission data and review times annually.

    4. Readers may be interested to learn that Carol Gould and I co-chair the reformed Association of Philosophy Journal Editors. We are in the process of contacting all philosophy journal editors now ahead of a meeting to be held at the APA-Eastern this December. We believe that editors have much we can learn from each other and that there are good reasons to join forces on various issues where possible. Hopefully, much good will come from our collective efforts.

    Reply
  17. Thom Brooks

    In response to Christy, why wait six months before contacting referees again? Why not check in after about six weeks instead? In my view, six months should be the absolute limit on how long a paper might be under review. Moreover, I wouldn't penalize the referee: if an editor knows a referee has sat on a paper too long, then it may be the case that the editor should find a new referee. Granted, this takes a lot of time — the problem of selecting referees is quite an important one — but it would solve this problem of 6+ waits without anyone being penalized.

    For what it's worth, I entirely agree with Keith DeRose on in-house editing.

    Reply
  18. Colin Farrelly

    There is much that could be said about the vices of in-house editing of flagship journals (journals that carry so much weight in tenure and promotion decisions, the RAE, etc.). But I’ll limit my comments to just two points.
    I find it odd that many seem to hold the following two beliefs:

    (a) the official professional organization of philosophers, which is composed of 10 000 members of the profession in different countries from all areas of philosophy, is grossly incompetent and could not be trusted to oversee and run a journal in the discipline.

    AND

    (b) the handful of editors and reviewers at (one or both) Columbia and Cornell possess the epistemic capacities and resources needed to design, implement and oversee the efficient, fair and inclusive review of hundreds of submissions in all areas of the philosophy.

    I can appreciate the tendency to hold belief (a), but other disciplines seem to manage, like APSA and ASA. So why are philosophers so different? And what I find really surprising (and troubling) is how many philosophers hold (b) (yet this post shows at least some do see problems with the status quo).

    I suspect what is happening is that many philosophers in the profession simply *idealize* the epistemic capacities of the faculty in these two departments (are they not prone to bias, self-selection, time-constraints, cronyism, etc. just like “real people” are?), and yet take an overtly pessimistic view of the general profession’s ability to offer anything better.

    Given the choice between an editorial procedure that places almost all (if not all) weight on the judgement of the faculty it its own department, and one that relies much more heavily on the opinions of a more numerous and more diverse group of philosophers, I’d go with the latter.

    As for the alleged “incompetence of the APA”, I guess an organization is only as good as its members. Perhaps the stakes currently at risk (prompt and accurate publication of JFP) are not sufficiently high enough to motivate members (beyond those in junior positions) to care much. If given the responsibility of overseeing the discipline’s flagship journal(s) perhaps things would change. But maybe not. Maybe the best the profession can hope for is the current system. But if so, that is very sad and does not reflect well on the profession.

    Cheers,
    Colin

    Reply
  19. John Turri

    Christy seems to be on to something, but I'd suggest a slightly different approach. If we punish tardy referees with submission bans, then I suspect at least a substantial minority of would-be refs will simply stop refereeing. We need a system that will not only discourage poor refereeing, but also encourage good refereeing *and discourage non-refereeing*.

    Instead of a ban, we might implement an "incentivized multi-track system." Your *submissions* will be prioritized in the review process to the extent that you:

    A. Referee more papers,
    B. Referee papers quickly, and
    C. Referee papers thoughtfully.

    In short: more, better, and quicker refereeing would "fast-track" your submissions, whereas fewer, worse, and slower refereeing would "slow-track" your submissions.

    Of course there's the conceptual problem of trying to come up with some weighting formula that would assign people "scores" in the system. A and B would be no problem. Give people 50 points per paper, and between -100 and 100 points depending on how many days it takes them to complete the assignment: 100 for completing it the first day, 99 for completing it the second day, …, 0 for completing it the hundredth day, -1 for completing it the hundred and first day, …, -100 for completing it the two-hundredth day. Assigning values to C would be trickier, but here's an idea: let the author herself assign a value to the comments between -100 and 100. Or let the author and the editor each assign a value between -100 and 100, and then average the two.

    We might also want to add in a penalty for "inactivity" or too high a ratio of refusals per request-to-review. (The latter would obviously need to be waived if you refereed some absolute minimum, say, four per year or whatever.)

    There's also the logistical problem of implementing such a system. Maybe Thom and other editors on this thread will comment on whether something like this is even feasible.

    [P.S. Brian: let us know if you would like to save discussion of such measures for another thread, since it's clearly not directly relevant to the topic of the post.]

    Reply
  20. Robert Gressis

    I like John Turri's suggestion, though I wonder if it could end up being a "the rich get richer and the poor get poorer" scheme. After all, the people at the top departments or who have senior positions often, from what I gather, have more time to devote to research than junior people or people at teaching-heavy departments. Consequently, it appears as though it could result in people who already have the most time to do research also having their own publications fast-tracked as well.

    That said, refereeing papers more quickly should count for something.

    Reply
  21. Aldo Antonelli

    Let me try to put a bit of damper on the referee-bashing.

    Refereeing is a thankless task, and good referees are few and far between. The punitive measures being advocated would turn off people from undertaking refereeing jobs (a system of positive incentives, on the other hand, might work better). That would hardly be better than the current situation.

    Yes, sometimes referees are slow, and sometimes unresponsive. There is not much that can be done about this by the editors, except perhaps constant (but polite) nagging of referees.

    Occasionally papers will take an inordinate amount of time to get refereed. What I do find unprofessional in the stories reported, though, is the lack of communication between editors and authors. If the editor is dealing with an unresponsive referee, let the author know, keep him or her in the loop!

    Reply
  22. Allen Coates

    The one worry I have with John Turri's suggestion is that people cannot choose how often they are asked to referee a paper. If you are asked to referee very frequently, you may have to say 'no' fairly often, and this would be punished in Turri's system. If you are very rarely asked to referee, this too would be punished. Turri's system would work best if we were all asked to referee with moderate frequency, whereas I think that the frequency with which people are asked to referee varies widely.

    Still, perhaps something like Turri's suggestion could be made to work.

    Reply
  23. Michael Cholbi

    I'm not enthusiastic about either charging submission fees or giving cooperative referees fast track reviewing on their own manuscripts as ways of encouraging better and more prompt refereeing. Submission fees would have to be more than pro forma in order to deter submissions, and I'm not confident that the submissions so deterred would be the frivolous or ill-conceived manuscripts. Instead, the manuscripts that would end up not being submitted are likely to be the promising work from younger scholars not able to bear the submission fees and that might most benefit from careful feedback. That would an especially unfortunate result for the discipline. The fast track proposal also seems to discriminate against younger scholars, since more established scholars are more likely to be selected as referees in the first place. If anything, given tenure timelines, we should try to find ways to accelerate the reviewing process for the work of younger philosophers, rather than slowing it down!

    But I wonder if some *positive* reinforcement for referees would speed things up, encourage better comments and fairer evaluation, etc. Some ideas along these lines:
    1. Most journals publish a list of referees annually, but perhaps a separate letter could be mailed with an issue (or an e-mail sent) listing the referees in a given year.
    2. Journals could establish criteria for high quality refereeing (call it something like "Gold Medal" refereeing) and send a thank you letter to those who meet this standard in a given year.
    3. Referees could get a free subscription to the journal or to the online version. The latter costs the publisher nothing. Or perhaps the referees could be entitled to a special 'sneak peek' subscription, where they have access to forthcoming articles sooner than normal.

    Reply
  24. jon kvanvig

    I don't see how this is a problem about refereeing. If it were, every journal should be subject to the same abysmal, pathetic record that Mind and JP are showing. One can always try to improve on present methods, but focusing on refereeing looks more like ignoring the elephant in the room by insisting that we need to do some remodeling to increase our square footage. What is needed here is for Mind and JP to start acting responsibly, in the way other journals seem to be decent at. The problem is simply not one to be laid at the door of referees.

    Reply
  25. Doug Portmore

    I don't think the problem lies solely or mainly with referees. Presumably, Nous and Mind draw from the same potential pool of referees with vastly different results in terms of their review times. And Nous succeeds in being much speedier without sacrificing the quality of what they publish. This leads me to think that the problem doesn't lie with referees, but with the editors who run the journals.

    Aldo Antonelli writes, "Yes, sometimes referees are slow, and sometimes unresponsive. There is not much that can be done about this by the editors, except perhaps constant (but polite) nagging of referees."

    I completely disagree. One thing editors can do is give referees deadlines and then remind them when their deadlines have past, telling them that have, say, a one-week grace period to turn in their reports — otherwise, the journal will need to find replacements for them. If after the grace period is up the report still hasn't been filed, then the editors should write the referee telling him/her that he/she need no longer submit his/her report, for his/her failure to file the report within the agreed upon time frame has forced them to seek a replacement. (Shaming referees in this fashion may help motivate them.) My sense of it is that most editors don't do this. It seems that many just wait for months past a deadline without seeking out some more reliable replacement.

    Also, different editors give referees different deadlines. Some editors ask me to file my report within four weeks, others within six weeks, and others still within some longer time frame. I've always kept to the imposed deadlines, but I'm much more likely to file my report within four weeks if given a four week deadline than I am if given an eight week deadline.

    There are other things that can go wrong that have nothing to do with the referees. For instance, I find that Mind sometimes takes a month just to acknowledge receiving my referee report. If it takes them a month just to acknowledge receiving my report, I wonder how long it takes them to read the report and then render a verdict.

    And I've heard that some journals are willing to wait weeks just to hear back from a potential referee as to whether he/she is willing to review a submission. If they then have to go through four to six potential referees just to find two that are willing to take on the task, then just this part of the process can take weeks and perhaps even months.

    So there are many things that editors can do to speed up the process: (1) choose reliable referees; (2) impose strict and timely deadlines on referees; (3) communicate effectively with potential referees, where this includes giving them a deadline to respond to a request to referee a submission; (4) have something like Nous's two-tiered process of review, where the initial reviewers are asked only whether the paper deserves to be given a more thorough review; etc.

    Reply
  26. Aldo Antonelli

    Doug Portmore writes, "One thing editors can do is give referees deadlines and then remind them when their deadlines have past, telling them that have, say, a one-week grace period to turn in their reports — otherwise, the journal will need to find replacements for them."

    I see that Doug and I agree about the nagging, except for the part about being polite.

    Threatening referees with being replaced will obviously lead nowhere, as many would be all too happy to be let off the hook. And now suppose that you (the editor) have just told a referee that you are going to find a replacement. How exactly does that make you better off? How does that help the author?

    Reply
  27. John Turri

    I completely agree with Jon and Doug — this is not a refereeing problem. And it probably wasn't productive for me to offer my earlier suggestion on this thread.

    Reply
  28. Mike

    "The problem is simply not one to be laid at the door of referees."

    "I completely agree with Jon and Doug — this is not a refereeing problem"

    These are strange conclusions. It is _at least_ a referee problem. (Hard to take seriously the plea, "don't blame me, he didn't force me to get it done"). Of course, it's likely something more. Suppose the set of referees for the problem journals just is the set of referees for the non-problem journals (or, something close enough). Are we supposed to believe that these good referees act significantly out of character when the pressure less perceptible? Implausible. Seems more likely that in-house relationships at (some of) these journals make the ease of finding a referee worth the trade-off of finding a less-than-virtuous one.

    Reply
  29. Doug Portmore

    Aldo,

    I've had some journals wait for over six months for one referee. If a journal set a sensible deadline of say six weeks and then replaced them after seven weeks, they could go through three referees in the time that that one journal took waiting for just one. I think that it's much more likely that at least one of three referees will file a report than that one will, especially when the one has already shown a lack of responsibility in not completing the report within the time frame that s/he was agreed to. It seems to me that journals would operate more efficiently if they replaced irresponsible referees.

    This policy amounts to more than mere nagging. And there's nothing impolite about replacing someone who can't keep to his or her commitments.

    Reply
  30. Neil

    The difference between Mind and the rest is, as other have pointed out,　not a refereeing problem. But the difference between prevailing standards with regard to turn around time and other disciplines, especially the sciences　probably in part a refereeing problem. In part, it is a problem with regard to what all sides find reasonable. There is no reason at all for us to tolerate a 6 month turn around time with submissions. Many fine articles have been written from scratch in the time it takes many referees to get around to putting a few words on paper.

    Reply
  31. Mark van Roojen

    Mike,

    It could be not a refereeing problem, if for example a few small steps to remind referees could make a significant difference.

    Let me give one sort of example. I believe I've only once really seriously blown past a deadline and it was because the paper was on line rather than in the mail and I had forgotten that. (This was several years ago when just a few journals were starting to put papers on websites. And my old system for keeping track relied on a pile of papers.) I searched my office for it for an entire day but all I could remember was that I had a paper to referee, but not for whom. So I had no way of getting in touch with the relevant journal. I finally decided I was wrong. Within a week or so I had an email from the journal asking where the report was. That was enough to allow me to figure out it was still on the website, to download it and get it done within another week. Even if forgetfulness and disorganization count as states of character, they are hardly insurmountable by the right bureaucratic procedures.

    Just by sending the reminder email that journal (Phil Studies) got things back on track. My negligence delayed things by only two weeks as a result. It would have gone on for much longer without that email. Nous, one of the ones we all seem to agree are doing alright, seems to email on the day of the deadline or a day later.

    So Jon may be more right than wrong about referees. In any case, as I sort of said above, we'd need more information to know where the problem lies. (Thom Brooks, care to tell us what works for your journal?) Jon is also right, I think, to emphasize that we don't really need to know the details to know that something in the editorial offices is likely to explain the patterns when there are wide and systemic differences between journals. OTOH, I still have some worries about how good our evidence is about the patterns.

    Reply
  32. David Hunter

    I referee for about 15 different journals including "proper" philosophy journals as well as non-philosophy journals such as the British Medical Journal. Likewise I've submitted papers to a range of journals that I referee for. What I have noticed, both in my own reviewing behaviour (although I try to do a review within two weeks of being asked if I can, often this slips out to the deadline) and in my papers being reviewed that the review takes however long the journal allows. So the British Medical Journal asks for a review within two weeks, it takes two weeks to get the review done. The Journal of Medical Ethics asks for a review in 6 weeks it takes 6 weeks…

    This isn't surprising, since academics tend to be busy people, and given that a review tends to take the same length of time roughly to do, and other tasks might be more pressing, the longer we are given the longer we take.

    So rather than trying to incentivize reviewing further why not simply drastically decrease the time given to produce a review? This shouldn't be a problem since people know what they are signing up for when they agree to do the review.

    Reply
  33. David Wallace

    I publish mostly in philosophy-of-science journals, and they usually manage a six-week turnaround time.

    I actually find it difficult to see why anyone thinks they need a six-month window to referee something. If I accept an invitation to write a paper I can see why I might need a long period. Maybe the actual task of writing it is going to take a non-trivial amount of time, and given I can only devote N% of my time to research, and some fraction of that N% is already committed, so that it's going to take six months to finish it. Or maybe I've already got my research plans fixed for the next four months, and won't be able to turn my attention to the new paper till then.

    But refereeing isn't like this: it takes maybe a couple of hours, or an afternoon at most, even if you do write a detailed report. It's just a matter of finding a slot to do it in, and anyone can find a couple of hours over the course of a month or two – unless they've got some really major commitment over that month or two, in which case they can just turn it down. Yes, we're all busy, but that's a reason not to do too much refereeing, not a reason to spend too long on any given bit of refereeing. My own experience – and that of colleagues I've spoken to – is that what actually happens is that you ignore the refereeing job until the deadline is looming, whenever that is. For me, the pattern is pretty systematically:

    i) agree to write report within ~4 weeks
    ii) ignore paper for 3 weeks
    iii) receive ever-so-polite reminder email from journal that I'm a week from deadline
    iv) write report sometime in the next fortnight

    Reply
  34. Mike

    Mark,

    It looks like, on the one hand, you want to say that any refereeing failure that is correctable by an editor is a problem for the editor and not the referee. The assumption seems to be that editors are obliged to fulfill a wheedling requirement. I don't think they are; of course, I'm not denying that their doing so might go some way toward solving the problem. That editors can take extra measures to solve the problem doesn't mean it's _their_ problem. On the other hand you offer an example in which disorganization explains a refereeing delay. Sure, editors can take extra measures to accommodate the disorganization of referees. But I'm hardly led to conclude that it an editorial failure if they don't. What's next, an obligation on contributors to cajol editors to cajol referees?

    In any case, I said that it's "at least a referee problem". This is consistent with it also being an editorial problem.

    Reply
  35. Bredo C. Johnsen

    I too was caught in this maddening situation: two years between submission and getting a "revise-and-resubmit." But I'm appalled at some of the attitudes evinced in comments here. Surely, with a journal as distinguished as JPhil, one begins with confidence that no one is more frustrated than the editors in such circumstances, and that they are doing their level best. Anyone who wants to is free to withdraw a paper at any time, and I'm sure many did, but given the topic of my paper, I wanted it in JPhil if at all possible, and the decision to wait it out was a no-brainer. Incidentally, John Smylie (managing editor) was a paragon of forthrightness, sympathy and general helpfulness throughout our correspondence. Would I submit another paper to JPhil? It's already there.

    Reply
  36. John Turri

    Bredo,

    You're "appalled" that some people think a two-year delay is beyond reasonable, regardless of how distinguished the journal or how frustrated the editors themselves might be about it? Really?

    And somehow I doubt that anyone was *more frustrated than me* about how long it took with my submission.

    Reply
  37. Bredo C. Johnsen

    John,
    I neither denied that the delay was beyond reasonable, nor suggested that the editors were *more* frustrated than submitters. The attitudes that appall me are those that seem to presuppose that there couldn't possibly be an acceptable explanation of these enormously frustrating events. Perhaps the editors should have declared a moratorium, and even explained the reason(s) for it. But they didn't; they kept at it. Meanwhile, we were all free to choose: wait or withdraw. My "r-and-r" letter was accompanied by extensive thoughtful comments. A friend received one of the most judicious editorial letters I have ever seen when he raised a complex issue concerning their initial response to his paper. I continue to give them the benefit of the doubt on the question whether they are doing their level best.

    Reply
  38. Thom Brooks

    Mark – Sadly, I have no genuine secret with the Journal of Moral Philosophy. I spend time initially reading papers and doing my best to find suitable referees before writing to them. I ask for reports in about one month and I make a note to chase up reports when they have not yet been received. The result is that 80% of papers are refereed in two months or less. Of course, that leaves 20% (often papers submitted as part of a special issue that requires additional coordination), but I think authors submitting papers have been pleased — plus, I make a point of thanking referees in person for helping us with papers whenever and wherever I can.

    Again, there will be a roundtable panel at the APA-Eastern of the relaunched Association of Philosophy Journal Editors. This panel will discuss these and other issues of concern to both contributors and fellow editors. I do hope attendance will be strong.

    Reply
  39. John Turri

    Bredo,

    I very much doubt that anyone here presupposes that there couldn't possibly be an acceptable explanation.

    'Doing their best' is ambiguous, I think, between trying their hardest and promoting the best (or close enough) outcome they could. I don't doubt for a moment that they're trying their hardest (or close enough). But I'm absolutely certain that people could be promoting much better outcomes.

    P.S. My earlier question was prompted by this statement of yours: "one begins with confidence that no one is more frustrated than the editors in such circumstances." Perhaps I misinterpreted you here, but I believe I'm a counterexample to that generalization. (Technically, given your literal words, my objection should have been, "I doubt that anyone is more, or equally as, frustrated as me …".)

    Reply
  40. Joseph Diekemper

    Bredo C. Johnson,

    It strikes me that portraying the editors of JPhil as heroically battling a frustrating problem is somewhat of a misrepresentation, since they are most likely the source of the problem (given JPhil's editorial structure). Here I am distinguishing 'the editors' from the managing editor, John Smylie. And this seems really to be the problem. With most journals (as I understand it–no doubt someone will correct me if I'm mistaken), the managing editor or editorial assistant can complain to the editor about a problem referee, and the editor then can exercise some kind of authority over the referee (precisely as is being urged by some of the earlier commenters). In the case of JPhil, however, the editors *are* the referees!

    Throughout my 25 month saga (from submission to final decision) with JPhil, I continually got the impression that Smylie was doing his 'level best' to cajole the editors, but that they were the source of the problem, and that his hands were more or less tied. Of course, this was just an impression, and I don't have direct evidence for it. Having said all this, I have to acknowledge that I, too, received extensive, thoughtful, and very helpful comments in my R&R letter. Those comments unquestionably improved the paper, and led to its eventual acceptance.

    Perhaps the chief problem with JPhil is not specifically the in-house review system, but the fact (if it is a fact) that there is no single person who has the authority to get things moving. If this is the problem, then it wouldn't take much effort to do a little restructuring; and the effort would be well worth it if it solved what is clearly a terrible problem.

    Reply
  41. Sebastian Lutz

    Bredo,

    You write that `we were all free to choose: wait or withdraw.' But while it would be nice to have the choice to submit the paper to the journal with the fastest response time, the choice between waiting and withdrawing a paper is extremely unpleasant: By the time it is bad enough to withdraw, one has already waited a long time, and to start the process over is not a great alternative (reminds me a bit of downloading files: Do I stay with my current connection or do I start over: http://xkcd.com/612/ — make sure to read the mouse-over comment).

    Reply
  42. Andrew

    The reason that Christy's idea (or John Turri's revision of it) is unlikely to work is that many referees are distinguished senior philosophers who rarely submit to refereed journals anymore. They're the major experts in the field, and so editors want them to be referees. But they also receive so many invitations to contribute to special editions and edited collections that they don't need to bother with the frustrating process of submission and revision at a journal.

    So I really do think that money has to be involved. Sites like http://www.Stickk.com (which was created by a Yale economist) show that even just a small financial incentive can help overcome serious procrastination problems. Try it next time you get a refereeing assignment!

    Reply
  43. mark lance

    The variety of editorial responsibility is obvious from both the refereeing and the submitting end. Many journals now ask you to seriously consider whether you will finish in – specify time here – and remind, then politely encourage, then strongly encourage you to do what you said. A former editor of Nous – who I've known a long time – once wrote to me an email "send me the fucking report now." He meant it with a smile and I very much appreciated it. I also sent the damned report that evening.

    While I disagree with an earlier poster about a report only taking a couple hours – I don't see how any paper can be seriously read and commented on in that time, and technical ones clearly take longer – it is extremely rare that one spends more than a day on a report. So what is absolutely true is that a deadline of 6 months is absurd. You will just wait 5 before thinking about it. If one doesn't have a free day or half day to devote to the project in the next 3 or 4 weeks, one should say 'no.'

    Since other fields have established this as a practice, it can be done. It should be.

    (Oh, and I'm strongly opposed to filing fees for submissions. Remember that there are lots of grad students, some unemployed philosophers, and some economically marginal junior and adjunct faculty out there. It is simply wrong to make it functionally impossible for them to submit.)

    Reply
  44. Peter Alward

    Jeez, compared to some of the rest of you, my 19-month-no-decision (with-occasional-responses-to-queries) experience with JPhil seems like something of a cakewalk. I am already holding my own personal boycott of JPhil, along with the other members of my do not submit list — CJP and (lifetime achievement award winner) Linguistics and Philosophy. I expect PPQ (23 months and counting) will be joining them shortly.

    Reply
  45. former J Phil submitter

    The title of this blog post expresses an excellent idea.

    From the handful of comments I've read so far here and over at Certain Doubts, it feels amazing how many others have been through the same horror stories with the Journal of Philosophy as I have. I myself waited for 19 months until autumn 2008, in a pivotal point of my early career, and all queries were to no avail (though the editorial secretary did write me long emails, apologizing for what he admitted to be inexcusable, and explaining the situation).

    After several editors had read the paper (I knew this from the status reports I got from the secretary), the paper even went out to an external referee. Even so, however, I only got a brief in-house comment accompanying the eventual rejection, which was incompetent, rather vicious, and one statement of which read as if a certain editor was miffed about some distinction of his own having gone unmentioned (which, because it's irrelevant, nobody else cites in the pertinent literature either).

    Definitely the worst journal, J Phil.
    They've clearly taken that #1 position from Mind.

    Philosophers often don't seem to realize that the regular occurrence of such perversely protracted manuscript treatments is not some regrettable part of standard scientific practice which we must shruggingly accept. This cannot be accepted.

    And given the Web, it should be easily possible for someone to put up a website which lets people exchange information about turnaround times at journals. I hope someone competent at doing this actually does, and would be happy to hear about it on this blog.

    Brian Leiter is right about suggesting a boycott of J Phil.
    Thanks for addressing this so clearly.

    Reply
  46. former J Phil submitter

    I've also read Brian Leiter's critical blog post from Sep. 2006 now, the one linked to in the post above, on in-house reviewing with J Phil and Phil Rev. And it occurred to me that none of the comments I've read so far has mentioned another glaring deficit of the Journal of Philosophy:

    J Phil is, to my knowledge (and that of other philosophers who've complained about it to me), the only major journal which doesn't give a bleep about whether manuscripts are anonymized or not. Talk about cronyism!

    I can confirm that AJP, by contrast, is excellent indeed. The turnaround time there for the same manuscript I'd submitted to J Phil was something over 6 weeks, with several pages of comments so competent it made the J Phil look ridiculous by comparison. The paper was rejected, but the reason given was on a completely different level.

    And how do their standards of anonymity compare? The AJP writes back to the submitter if there is even so much as an anonymized reference to a forthcoming paper of the author in the bibliography, requesting that the name of that journal be removed (!) and the manuscript resubmitted accordingly. They really do go to such lengths. J Phil, on the other hand, is so unprofessional that it's hard to see why they are still regarded highly at all.

    Reply
  47. Clayton

    Apologies to Brian for a slightly off topic comment, but I wanted to pick up on something FJPS said. I believe that when papers are submitted to AJP the handling editor doesn't know the author's identity, and I think that this isn't the norm. Shouldn't it be?

    I can't think of any good reason for the handling editor to know the author's identity when giving a paper an initial screening to see if it should be sent out for review and deciding which referees to assign to the paper if sent out for a review. I can, however, think of some reasons to keep the process blind at this stage. A few years back, I thought of writing to complain about a bad referee's report but was told by a friend that it would be unwise to do so as complaining about that report might offend and/or annoy the editor making it harder to publish something edited by this particular editor. It could be that my friend's worries were groundless, but I started to worry that things might work the other way as well. If someone has a positive relationship with an editor, does this influence the initial screening process and the selection of referees? Setting aside these sorts of concerns, it seems there's good reasons for editors not to know an author's race, sex, and institutional affiliation but it seems that as most journals are now run, this information is all available to an editor when deciding whether to send out a paper for review and choosing which refs to assign to the paper. Wouldn't this be very easy to fix?

    Reply
  48. Jeremy Fantl

    As the author of the "even more amazing" comment that Keith DeRose refers to in comment #2, above, I feel it is my duty to note here that I completely misremembered the time to my original decision from Mind. It was just less than 3 months, not a whole year. I apologize to the editors at Mind for my error. See the more complete comment at Certain Doubts here: http://el-prod.baylor.edu/certain_doubts/?p=1245#comment-9212.

    Reply
  49. Tom Baldwin

    As editor of Mind I have followed these postings with considerable dismay at the complaints about Mind’s performance and I have placed a ‘reply to my critics’ on John Turri’s blog http://el-prod.baylor.edu/certain_doubts/?p=1245. I don’t claim that Mind’s recent record has been flawless; but, as Jeremy Frankl’s recent posting shows, some of the critics have made mistakes too. But what I most want to communicate is the commitment of Mind’s editorial team to dealing properly and promptly with submissions. Believe me, we do devote a great deal of time to it, including evenings and weekends (ask my wife about it!). There is still some space for papers submitted now to appear in 2010 and our procedures certainly allow for this possibility.

    Reply
  50. Brian Leiter

    Professor Baldwin's reply to the concerns raised here and on the "Certain Doubts" blog are actually here:
    http://el-prod.baylor.edu/certain_doubts/?p=1265

    Thanks to Professor Baldwin for addressing these issues. Hopefully the J.Phil. editors will see fit to speak to these problems as well.

    Reply

Designed with WordPress