Leiter Reports: A Philosophy Blog

News and views about philosophy, the academic profession, academic freedom, intellectual culture, and other topics. The world’s most popular philosophy blog, since 2003.

  1. David Wallace's avatar
  2. Catherine Sutton's avatar
  3. Mark's avatar
  4. Brian Leiter's avatar
  5. Roger of Invisible America's avatar
  6. Mark Engleson's avatar
  7. s. wallerstein's avatar

Improving peer review

Brian Leiter

Philosopher Leora Dahan Katz (currently a Polonsky Fellow at the Van Leer Institute in Jerusalem, soon to join the Hebrew U faculty) writes:

It has been my experience that submission rejections (in philosophy and legal philosophy journals) that are not accompanied by reviewer comments are not only frustrating but also highly inefficient. Clearly whomever decided on the rejection had a reason for doing so, and yet blank rejections encourage resubmission without editing given that the author has no information about why the piece was rejected (e.g. whether it was a matter of quality or journal mismatch). It seems to me it would be fairly easy and low cost to relay the relevant information thereby both helping the author and preventing repetitious resubmission of pieces whose fault is obvious to editors.

For example, the disciple might adopt a list of basic reasons for rejection, one of which would be marked on any given rejection.

I have little experience editing the but the list might include some of the following:

  1. Quality of the piece found wanting
  2. Argument doesn’t work
  3. Not for our journal
  4. Too long
  5. Too broad
  6. Not ambitious enough
  7. Doesn’t engage relevant literature
  8. Not original/ No contribution to field 9. Badly written 10. Unclear 11. Not publishing paper of this kind at this time 12. Other

The final category of “other” would be necessary to keep the practice low-cost where the reason for rejection is non-standard. But authors would be no worse off than now, and in all those cases where rejection is for a “basic” reason, they would be far better off; as would the journals, since they would presumably receive less re-submissions of papers that should surely have been edited/submitted elsewhere/repurposed before resubmission.

I’m sure I’m not the first to have thought of this, but I do wonder why we don’t already have such a practice.

Readers?

Leave a Reply

Your email address will not be published. Required fields are marked *

28 responses to “Improving peer review”

  1. Jj

    I think some journals do blank rejections to speed up the review time. So you get a blank, but within 2-4 weeks. I think that is acceptable. A blank one but after more than 1 month much less so.

    Reply
  2. Joona Räsänen

    Antti Kauppinen suggested something similar in the Daily Nous some time ago. The biggest problem, according to the comments (some of them from journal editors), was that such practice would encourage authors to argue against the editors. With current practice, most authors simply move to a next journal after the rejection, but if the editor tells the author that, for example, the paper doesn’t engage relevant literature the author is more inclined to respond: "But yes it does!, what do you mean by that? Tell me who I have not cited that I should have cited!11!"

    Reply
  3. David

    I have to say that I'm flabbergasted by this. In my own field (Classics), I have never received a rejection that was not accompanied by detailed comments, addressed to the specifics of my paper. From the other side, I have acted as a referee for dozens of journals, and when advising rejection have always provided a detailed (2+ page) account of the problems with the paper that led to my decision, written explicitly to be transmitted to the author (whose identity I never know – double-blind refereeing is the norm in Classics). I had assumed that this was the universal practice across humanities disciplines; all that I can say is that it should be, and that Dr. Katz's modest suggestion doesn't go nearly far enough.

    Reply
  4. David Beard

    Non-philosopher here, always fascinated by the world of philosophy.

    Reviewers reject articles without justification? That seems both unhelpful to the author and unhelpful to the editor.

    Sometimes, an editor might truncate the reasons for the rejection [as a cold, hard, abrupt "no" prevents an author from resubmitting under false sense of encouragement], but that feels like a rare choice.

    In any case, if you are going to abdicate your responsibility to actually respond to a MS, I guess the checklist is better than nothing.

    Reply
  5. Marcus Arvan

    I appreciate the thought behind the proposal, but I agree with David and David Beard: a checklist lets reviewers off far too easy.

    A reviewer's job is not (or at least should not be) to simply report what they think by checking a box. Their job should be to provide an actual referee report providing a reasoned justification for their editorial recommendation. This is in part so that editors can judge the credibility of their report in light of the other reviewer reports. Given the prevalence of bias in human judgment, "trust my judgment" simply isn't good enough: https://philosopherscocoon.typepad.com/blog/2016/03/on-journal-refereeing-why-trust-me-isnt-good-enough.html

    I think a better answer is a system that actually holds reviewers responsible for the timeliness and competence of their reviews. One such system might be one where editors and authors rate the quality of a given review. Reviewers would then receive Overall Reviewer Scores on the basis of their general performance as reviewers, and then be matched with similar-performing reviewers when they submit as authors. This would plausibly incentivize better quality (and timeliness) of reviews across the board, as no one wants to be paired with a bad reviewer as an author–and in this system the only way to ensure that would be to be a good reviewer oneself!

    See: https://philosopherscocoon.typepad.com/blog/2018/12/incentivizing-better-reviewer-behavior.html

    Reply
  6. Steven French

    I'm commenting as a Co-Editor of a journal: we do 'desk reject' papers, for all sorts of reasons ranging from 'falls outwith the remit of the journal' to 'does not advance the debate'. We do not pass these comments on, precisely for the reason given on Daily Nous – that in our experience this leads to unhelpful and sometimes angry exchanges with authors. On the positive side, as Jj indicated, given the frequency of our editorial meetings if an author is 'desk rejected' they'll know within two weeks.

    If a submission is sent out to referees then of course we pass on their comments with our decision but we definitely do not expect 2+ pages of detailed analysis – that way lies madness! In my experience what is wrong with a philosophy paper can usually be expressed in a couple of paragraphs (weak argument, failure to engage with relevant literature, inappropriate case study/example …) and thats enough for editors to make an informed decision.

    Reply
  7. Daniel Kaufman

    I am of very mixed feelings on this. On the one hand, it just seems like common sense that if one rejects a submission, one should explain why, even if just as a matter of basic manners. On the other hand, having received copies of referee reports, I often wished I hadn't, given how shoddy they were. Particularly abused is the "hasn't engaged with the relevant literature," which overwhelmingly means, "hasn't engaged with the literature *I* [the referee] care about." Citation in academic publications has gotten completely out of hand to the point of being a positive barrier. Indeed, I find it harder and harder to motivate myself to do this sort of work for this reason alone, beyond all the others.

    So, more and more, I am leaning to the side of "quick turnover, with no reason given." Does this contribute to the deterioration of the intellectual culture in our profession? Absolutely. But that ship has already long since sailed. The absurd overproduction of professional papers; the even more absurd job availability/PHD production ratio which makes said overproduction even worse; the no-simultaneous submission policies; the increasing intrusion into the editorial process of political considerations on the part of ideologically compromised referees. The whole thing stinks. No wonder that more and more people — myself included — are putting their passion and energy into D.I.Y. public intellectualism, where this whole miserable, corrupt system can be avoided.

    Reply
  8. Henry

    The obvious solution to that problem is to accompany the rejection explanation with a statement that the rejection is final and responses to it will not be considered.

    Reply
  9. Jj

    I don't think it is reasonable to always expect an explanation from a journal – the journals do not owe the authors anything of that sort (unless it is a requested or commissioned piece, obviously). Of course, the journals have good reasons to provide useful feedback to authors but they must weigh that against number of other considerations, including availability and willingness of reviewers, turnover time, number of submissions, and so on. I have received about 30 papers to referee in my life, out of which only about 15 were, at least in my view, worthy of submission (5 I recommended). The rest were just not ready for submission to any professional journal and should have been weeded out early on by the editors (though I understand that that's harder than it sounds). Providing comments on those would take forever – there were too many things wrong/lacking. Journals are not there to educate authors how to work in their field, even if publishing is a learning experience.

    Reply
  10. Mark

    I assumed my blank rejection from BJPS was an ironic commentary on the limitations of naive falsificationism.

    Fully agree with David's comment that Katz's suggestion does not go far enough.

    Reply
  11. John Schwenkler

    I disagree with Marcus concerning what is "the job" of a reviewer. Reviewers have many jobs (qua reviewers), and what these are will vary according to the policy of the journal. But surely the *principal* job of any reviewer is to provide sound advice to journal editors concerning whether a paper should be published. It's fair to expect that this be accompanied by some sort of explanation of why the advice is what it is, and it seems to me that a checklist of the sort envisioned in the original post could be a reasonable way to accomplish this. However, I think it's a bad idea to hold that a reviewer should always (or even nearly always, or in general) have the responsibility of suggesting (especially in detail) ways that the author can make the paper better. (No doubt it can be *good* for reviewers to do this, and I often do it myself — my concern here is only with what should be required as a matter of policy or professional expectation.)

    If a journal wants to have a policy of requiring that reports are substantial and offer detailed advice, then that's fine — though I expect that they will have a harder time finding reviewers, and that their reports will then be slower in coming. In the end, however, the responsibility of getting feedback falls to the author himself or herself, and it is unreasonable to expect — as many philosophers freely admit they do — that reports from anonymous reviewers are going to serve this function. It's even more unreasonable to expect this while also complaining, as again some of those philosophers frequently do, that the reviewing process takes too long.

    Reply
  12. David

    [To Prof. Leiter: Apologies – I mixed up refereeing with reviewing in my previous post. Could you delete it and substitute the one below?]

    Just to follow up on my earlier comment, I thought readers might like to see the official "code of practice" for refereeing of the Society of Classical Studies (the major professional body for classicists in North America). It's part of a wider document concerning all aspects of professional ethics, endorsed by the entire membership of the SCS (https://classicalstudies.org/about/scs-statement-professional-ethics ). The relevant portion reads as follows:

    Editorial Policy and the Refereeing Process

    Journals should have well defined and clearly stated editorial policies and should publish for readers the procedures for submission of articles. Editorial decisions should be informed by appropriate scholarly criteria. Editors have an obligation to ensure that all submissions are properly refereed. When inviting referees, editors should specify their requirements and ask for a commitment that a decision be rendered within a stated period of time (preferably within 6 weeks). In order to encourage fair and objective judgments, manuscripts should always be assessed as anonymous submissions, and the substance of referees' assessments should be conveyed in writing to the author. Editors should choose referees from qualified scholars who are open minded, impartial and not unsympathetic to the subject matter and approach of the manuscript. A good referee's report is one that provides a fair, accurate, and informed assessment of the submission under consideration. Scholars who are asked to referee a manuscript should decline in cases where they might have conflicts of interest which would impair their ability to make an objective assessment, or when such requests cannot be completed within the time requested. A referee has an obligation to inform an editor in cases where that referee has previously reviewed a manuscript. A commitment to publish a manuscript, once communicated to the author, is binding on the journal or publisher.

    ******

    Just a couple of things to draw attention to. First, refereeing is required: the editor(s) should not reject an article without having it refereed. Second, referees are expected to provide more than a bare decision on "accept or revise & resubmit or reject"; they should give "a fair, accurate, and informed assessment of the submission under consideration" and "the substance of referees' assessments should be conveyed in writing to the author".

    Does the APA not have a similar code of practice for philosophy?

    Reply
  13. Marcus Arvan

    Hi John, I appreciate your viewpoint. However, I have a question. You write, But surely the *principal* job of any reviewer is to provide sound advice to journal editors concerning whether a paper should be published.” How is an editor, or anyone else (including the author) to have any idea whether a referee’s judgement on a paper is “sound advice” in the absence of some kind of detailed report? A detailed referee report demonstrates whether it is sound advice: the editor can evaluate it in light of other referees’ reports, as well as in light of their own reading as an editor. In the absence of a detailed report, all one really has is the referee’s word (their bare judgment) that the paper is or isn’t worth publishing. Given the biases that can affect people’s evaluations of other’ work (both explicit and tacit biases), I worry about a system that permits people to simply give a recommendation without attempting to justify it. Also, for what it is worth, the system I am proposing also promises to address the problem of turnaround times—since it would both incentivize reviewers to give good reports, but also timely ones.

    Having sent papers out for in other fields (Psychology), and having a spouse who works in that field, I have not once encountered a case of submitting a paper and not getting a fairly detailed report. If people in Classics, Psychology and other fields can do it (with decent turnaround times to boot), why couldn’t/shouldn’t our discipline do the same?

    Reply
  14. Skef

    There is a fundamental, unresolvable tension in current practice that comes from the difference in what *excuses* desk rejection (which is how most of these no-comment rejections arise) and why desk rejection is now so common.

    The excuse or justification always offered is that so many papers are now submitted to journals in an unpublishable form. However, the *need* for desk rejection — which is clear from the comments editors always offer when the topic comes up — is a result of the sheer number of papers now submitted to journals. Desk rejection is ultimately a means of workflow management.

    The offer of the excuse holds out the hope that the entirety of "excess" in submission rates is made up by clearly unpublishable papers. But this is a false hope. Any doubt about this can be cleared up by the simple observation that unpublishable papers could be easily labeled as such: an editor could, for example, just cite the first five serious issues, and if those all came on the first two pages that would be enough.

    Requiring a clear, accurate, and appropriate reason for rejection is therefore untenable, because the whole purpose of the system is to avoid reading all submissions thoroughly. Editors appear to mostly excuse this situation to themselves, on skimming one or another paper, with the reason "not appropriate for this journal at this time". When the reader is an editor that reason becomes axiomatic: they're rejecting it, they're the editor, therefore it is not appropriate. You also hear, in the periodic threads on this subject, the reason "we may have published an article on this subject recently", as if there are still a significant number of philosophers who care about the annual composition of this or that particular journal.

    The bottom-line is that the first stage of review at most philosophy journals is that a paper is skimmed and only moves forward if the editor or their delegate is happy with the result of the skim. That process is not robust enough to provide an accurate reason for rejection that is acceptable to either the author or to a third party observing the process. Until that changes, no reason will be offered.

    Reply
  15. John Schwenkler

    Surely one way for an editor can have some idea about the soundness of a reviewer's advice is by knowing the reviewer's expertise and trusting (which can be a way of knowing) that she has read the paper carefully and given her advice in good faith. That's an imperfect measure, of course — but then so is anything that we finite beings are going to manage, and the fact is that there will always be trade-offs between thoroughness and efficiency.

    Reply
  16. D. Ross

    I'm chief editor of a (mathematical) logic journal, and desk-reject quite a few papers. I'm not going to waste the time of my editors or referees by sending them a paper with no hope of acceptance. In many cases the papers are simply junk, and my rejections don't give much detail on the rejection because there is no nice way of telling someone their paper is junk. In other cases the problem is something else, and I do try to explain why as part of the rejection; over my time in this position I've developed some boilerplate for this, filling the same role as the proposed checklist.

    For papers that we reject but don't desk-reject, I try to get useful information from my referees, but there is only so much one can twist their arms, and occasionally I have ended up sending rejections that are quite like those that have infuriated me as an author.

    Reply
  17. jbr

    Some food for thought. In neuroscience and psychology (my other fields) if a paper goes out to review, it always receives comments. The Journal of Neuroscience, the flagship journal for the Society for Neuroscience (which I review for fairly often), requires all reviews to be completed within 14 days, WITH comments. Despite the large number of submissions and relatively small size of the editorial office, this standard is by-and-large met. In fact, I get the distinct sense that many authors submit to jneuro (even when a manuscript is still a bit rough around the edges) because they know they will be substantive feedback in short order. Yes, sometimes the decisions and comments are frustrating.

    I think it is worth considering what it says about (the personalities of) philosophers such that many would be seemingly resistant to, or are incapable of satisfying, such a standard.

    Reply
  18. Michael Brent

    The sort of view suggested by John Schwenkler is one that, in my experience, is widely shared, i.e., that “the *principal* job of any reviewer is to provide sound advice to journal editors concerning whether a paper should be published”. My concern is that is too narrow a view of the role that we play as reviewers. In fact, when we provide advice to editors about a paper, the typical options are not merely accept or reject. Rather, the typical options are more nuanced than this, including outright acceptance or rejection, but also “revise and resubmit”, “accept with minor revisions”, “accept with major revisions”, and so forth. In these more typical cases, when providing our assessment of the paper’s status, we can offer critical feedback to the author, feedback that is intended to help the author improve their paper during the revision process, while simultaneously justifying our assessment of its quality. In fact, this is just the sort of feedback that we provide to students when grading their essays: feedback that aims to help them improve their draft during the revision process while assigning them a grade. I think we can, and should, aim to institute this sort of practice as referees, drawing on the skills that we have as teachers when providing feedback to our students.

    Reply
  19. Michael Magoulias

    A viewpoint from a publisher: This is ultimately a practical matter that reflects several factors: the number of submissions a journal gets, the size and frequency of its issues, and most importantly, the size of its editorial team. One major philosophy journal I worked with in the past had a single editor who was tasked with accepting roughly 25-30 papers a year for a quarterly journal. The typical volume of submissions in a given year was 500. It's simply not possible for one person to write substantive comments for that number of papers. The obvious solution is to expand the editorial team, but that is not always as easy as it sounds, especially, if one is dealing with a journal that is not affiliated with a society, and therefore lacking in a pool of specialist talent already committed to the organization sponsoring the publication.

    Reply
  20. Marcus Arvan

    In case anyone's interested, here's an (admittedly optimistic) proposal for a system that might incentivize authors to submit fewer 'unready' papers for peer-review, thereby lessening the workload of editors and reviewers (while simultaneously incentivizing quicker and more thoughtful reports by reviewers): https://philosopherscocoon.typepad.com/blog/2019/01/how-to-incentivize-better-author-submissions.html

    Reply
  21. David Wallace

    At least when I referee, I don't really consider that I owe the authors anything very much – but I do owe the editors an argument to support my recommendation, and it's usually then very little effort to write that argument in a form that is suitable for the author. (That's perhaps relevant to Jj's comment (echoed by others) that some papers are so bad that giving full feedback would take forever – if the feedback just needs to be sufficient to establish to an informed third party that the paper is unpublishable, that's much more feasible.)

    Reply
  22. Graeme Forbes

    All the discussions I’ve read or listened to on this sort of topic have been dominated by aggrieved authors and fantasy-world suggestions. This discussion is no exception.

    The problem is surely minor. Rejections not accompanied by reviewer comments are in almost every case desk rejections. So Dr. Katz’s suggestion is in effect that “efficiency” will be improved by giving an author a “basic reason” for a desk rejection. I don’t see how. If the reason is Dr. Katz’s 3, 4, or 11, an editor is going to say that anyway — I certainly do. That leaves reasons like “the argument doesn’t work”. As others note, this is even more likely to provoke tiresomely disputatious emails than a bald rejection would. “Henry” replies that we can avoid this by making a statement that the rejection is final and responses to it will not be considered. Uh, not really. The same number of quarrelsome responses will be received, except that those correspondents who actually noticed the statement will begin their missives “I know you said the rejection is final but…”. One can ignore such emails, but that’s already current practice. So nothing changes.

    “David” quotes from a policy document of the Society of Classical Studies according to which “Editors have an obligation to ensure that all submissions are properly refereed”, where the next sentence makes it clear that desk rejection doesn’t count as proper refereeing; rather, referees should be invited. This is fatuous beyond belief, certainly if applied to philosophy journals. Any journal that’s got “Philosophy” in its title is going to receive plenty of submissions that are little more than the ravings of lunatics, and plenty more where all the author wants is to tell readers about “his philosophy” (or hers, tho’ it’s usually a he). I am supposed to invite referees to advise me about such dross? Maybe qualified referees grow on trees of a kind I haven’t heard about.

    Re the difficulty of getting qualified referees: Marcus Avran proposes a system “where editors and authors rate the quality of a given review. Reviewers would then receive Overall Reviewer Scores…blah blah blah”. Oh Jesus. Of *course* authors who’ve just been rejected should be asked to evaluate the quality of their negative reviews. How stupid not to have thought of that oneself. (Slaps his forehead.) Anyway, we’re talking about desk rejections, so external reviews don’t happen. But if you want to discuss the case of papers sent out for review, same question about trees. Does anyone suppose that experts in a field are going to be encouraged to accept review requests if they’re told their reviews will be *graded*?

    And then there’s “Skef”, who opines that if an editor says the paper isn’t appropriate for the journal, “that reason becomes axiomatic: they're rejecting it, they're the editor, therefore it is not appropriate.” No. If I say “wrong journal” to an author, it isn’t true by definition because it’s the editor saying it. It’s true because the paper’s topic doesn’t fall in any of the areas listed on the inside front cover of every issue of Ling&Phil (a list which Ede Zimmerman and I inherited and, iirc, haven’t changed). If the author had simply read the goddam list, he (again, it’s usually a he, disproportionately) might have saved everyone time and trouble.

    “Skef” also says that the whole purpose of the system of desk rejection “is to avoid reading all submissions thoroughly”, which is fair enough, but he (perhaps she) says it as if he thinks it’s objectionable. Apparently, “Skef” does not understand that there’s a whole slew of people out there so in the grip of self-delusion that they cannot see that their research has simply no chance at all of ever appearing in a good journal. Let me repeat my main point there, in case someone missed it: their research has simply no chance at all of ever appearing in a good journal. If reading “thoroughly” means word by word from the beginning to pretty much the end, why in god’s name would anyone, least of all the editors of good journals, be obliged to read such stuff thoroughly? Turgid mediocrity usually announces itself in the first couple of pages, and rejection without a stated reason does not fall short of what it’s entitled to. I agree it ought to be within the author’s lifetime.

    Some contributors to this and similar discussions seem to imagine that editing a journal is the editor’s day job, or that the nights are 48 hours long. Please move back to Planet Earth from wherever you are.

    Reply
  23. Steven Yates

    As an ex-academic philosopher who these days writes mainly on his own blog (aaaah, the freedom!!!) & hasn't submitted to a refereed journal in years now, I'm not sure I have skin in the game here, but these suggestions may be helpful to someone.

    From reading the comments above by those associated with journals, it sounds as if the referees & sometimes the editors themselves are simply overwhelmed. Hence the impulse to take short cuts by resorting to forms, or simply not supplying authors with referees comments at all. Obviously no one can read & supply detailed critiques of 500 submitted papers in a year; even a staff of, say, ten, is likely to be overwhelmed by that kind of onslaught. Especially if, as I suspect is the case, the majority of submissions start at mediocre & go downhill from there, meaning that referees wanting to do a good job will have much to sort out. (It is not that the authors of mediocre papers are "bad philosophers" necessarily. They, too, are overwhelmed in the present economy, juggling jobs on multiple campuses, perhaps managing a full-time job search, possibly working at something outside academia to pay the bills, etc. They simply don't have the time it takes for "good" philosophy which I hope everyone reading would agree is not easy to produce. Yet the temptation is there to try for a "rush job" & hope for the best! Or at least a good critique!)

    I would suggest that there are simply too many papers being submitted for publication in journals, a product of the absurd level of competition for jobs, awareness that being rejected for tenure can be career-ending for the few lucky enough to find something tenurable, etc., that has overwhelmed academic philosophy.

    This, in turn, is a product of the publish-or-perish mindset — although you can publish & still perish if your publications aren't in sufficiently "prestige" journals.

    Now some institutions *have* opted out of this mindset. Before leaving academia I taught at three different community-type colleges, & over a period of seven years not one person mentioned publishing. Their interest was, "Can you teach?"

    Suppose more major state-level four-year colleges & universities decided to ratchet up teaching over "research" (most of which, let's face reality, isn't likely to be read by anyone anyway)? Why not suggest *eliminating* the need for strings of publications in "prestige" journals as a condition for employment and/or tenure, or some other long-term contract, at all except (perhaps) Ivy League institutions?

    While I enjoyed most of the academic writing & publishing I did (around 20 articles & another dozen or so book reviews), I was aware that many (most?) of my colleagues did not. Some had not written a line of philosophy since being granted tenure. But they were effective in the classroom. One such person who retired while I was at his institution in the early 1990s had never published, & admitted frankly to me in private that had he not been born when he was & entered the field when he did when jobs were plentiful, he would not have survived in the field. This fellow had no publications, but he did have teaching awards & was loved by his students.

    Seems worth thinking about. If both the field itself & large institutions could be encouraged to emphasize teaching more, & publishing less, those who had little or no interest in publishing would not be forced to grind stuff out to survive. Meanwhile, those who *wanted* to publish could do so without pressure. Their enthusiasm would show in their results. My guess is that journals would be less overwhelmed with paper, & the overall quality of what they accepted would go up.

    Reply
  24. Skef

    In reply I'll point out that it is obvious no one agrees with these points in general. I doubt that Graeme Forbes agrees with them in general. The almost universal advice in philosophy to *authors*, on hearing that one has received a desk rejection, is to submit the paper unmodified somewhere else. And not necessarily any lower on the prestige scale. Are we to suppose that Forbes, on hearing of a desk rejection by another journal, advises the colleague or student "Oh, that probably means the work is unpublishable. That's occasionally about fit, but it's more likely your paper is a work of turgid mediocrity."

    Will anyone reading here admit to taking a different view of desk rejection when acting in any capacity other than journal editor?

    Reply
  25. Steven French

    Thanks Graeme! You've hit the nail on the head. In my case it was 'A Poem in Praise of Perpetual Motion' that did it … (a genuine submission).

    In case anyone might be interested, here's what we put up on the BJPS blog on 'How to Avoid a Desk Rejection':

    https://thebjps.typepad.com/my-blog/2015/01/deskrejectionfrench.html

    Much has already been said about the difficulties in finding referees so I'll just note that I am currently on invitation no. 10 for one paper (and not one that deals with a particularly esoteric issue). And as Graeme also notes, all of this, finding referees, chasing them up when they're late, reading their reports, making decisions etc etc is undertaken in addition to my 'day job', as neither I nor my Co-Editor get any 'workload relief' from our respective institutions (yes, I know, we should but we don't and we won't under the 'current budgetary circumstances').

    And in response to Skef *my* advice to my PhD students who get a desk rejection is not just to submit it immediately elsewhere, but to consider carefully the remit of the journal, the standard that is expected etc. etc.

    Finally, getting back to Prof Katz's original concern, examples of papers that have gone out to referees and have been rejected without some commentary seem few and far between – we would never do that at the BJPS and when it comes to my own papers, sent to a wide range of journals over the years, although many were rejected of course, none, so far as I can recall, were turned down without comments.

    Reply
  26. Marcus Arvan

    Graeme: if authors grading review reports is problematic (something I raised questions about myself and which we have discussed over at the post I linked to), then perhaps a better option is leaving the grading to editors.

    Also, on the proposal I floated there are *incentives* for reviewers to accept reviewer assignments despite knowing they will be graded. Their Reviewer Score would depend in part on how many review assignments they accept (reviewers who constantly turn down review requests would get bad scores). Since their experience as authors (the kinds of reviewers they are matched with) would depend on their reviewer score, that would incentive people to accept reviewer assignments and put together good reports (since both would play a role in Reviewer Scores).

    I don't see what is fantasy-world-ish about this in the slightest. It seems to me a system that could realistically be put in place, and one that would set up a number of good-behavior-generating incentives, both for authors and for reviewers–making everyone's lives in the peer-review process go better.

    Reply
  27. Henry

    In 22, Graeme Forbes writes, "“Henry” replies that we can avoid this by making a statement that the rejection is final and responses to it will not be considered. Uh, not really. The same number of quarrelsome responses will be received, except that those correspondents who actually noticed the statement will begin their missives “I know you said the rejection is final but…”. One can ignore such emails, but that’s already current practice. So nothing changes."

    If that's the case, then what's the problem? If quarrelsome responses can be ignored, then the fact that they will be sent does not constitute an argument against explanations of rejections, unless one finds it overly burdensome to delete quarrelsome responses.

    Reply
  28. David Velleman

    I suspect that this discussion has not attracted many comments from journal editors because it is mostly out of touch with what is entailed in editing a journal in this age of submission madness. In order for each philosopher to receive more informative feedback on his/her submissions, all philosophers, collectively, will have to submit many fewer papers. There aren't enough qualified referees in the profession to read and comment on all of the submissions currently being received.

    Some editors have discussed forming a consortium that allows a paper to be submitted to only a limited number of journals within the consortium: N rejections from journals in the consortium would rule out further submissions to journals within it.

    Of course, this proposal, like others that have appeared, will be met with howls of moralistic protest. Every proposed solution to the problem is received as an unjust imposition on the rights of individual philosophers. But the problem is a multi-party arms race — a collective action problem. Each person submits more and more papers, and turns each rejected paper around more and more quickly, as the probability of acceptance falls in consequence of more and more papers being submitted and turned around more and more quickly. The problem will not be solved until philosophers are willing to accept some limits on their freedom to ratchet up their rate of production and submission.

    Anyone who complains about how they have been treated by journals should offer a solution to this collective-action problem. Demanding better treatment from editors and referees is pointless without some constructive proposal for how to make such treatment possible.

    —–
    KEYWORDS:
    Primary Blog

    Reply

Designed with WordPress