Editor's Virtual Forum on Peer Review

We are interested in finding out editors’ views on the various issues documented in Editors’ Update. We’ve created the Editors’ Virtual Forum to enable you to have your say. In each issue, we'll nominate a topic for discussion and ask you to share your thoughts with other editors.

The current topic nominated for discussion is: Peer review. Do you agree with answer A, B or C?

Discussion question: How many referee reports does an article require?

• A)   An article should only require two referee reports. That's excluding the editor's review because that should not be counted.
• B)   The editor's review is as valid as any other referee report. As a matter of fact, if the editor first reviews the paper and it is not according to standards the paper can be rejected immediately.
• C)   If you receive more than two referee reports by sending out requests to as many referees as possible, that is acceptable.

Please email your answer and rationale to editorsupdate@elsevier.com. Both your answer and rationale will be posted. Reactions will be posted below and at the end of the Referee Research results article. Each reply published will display the contributor's name and journal.

Note: Elsevier reserves the right to edit comments.

--------------------------------------------------------------------------------

Contributions to Editors' Virtual Forum

27. Chris Morfey, European Editor, Journal of Sound and Vision

I have been acting as European Editor for Journal of Sound and Vibration for the last six months. One issue that has caused me some trouble is the submission of linked papers. Other editors' advice on this would be welcome.

The difficulty arises when authors submit 2 (or even 3) papers simultaneously, dealing with the same subject and linked together, usually as Part I, Part II, etc. This means that

(1) the total manuscript is generally very long, 60-100 pages being typical;

(2) the papers cannot easily be split between different reviewers, as each one refers to the other(s);

(3) reviewers are often reluctant to take on such a large assignment; or if they do, it may take them a long time to get round to providing a report.

I wonder what other editors' policy is on linked submissions. Related to this, do they have a policy on maximum paper length; or can they suggest ways of deterring authors from submitting long manuscripts?

26. Kung, Gori, Hamp-Lyons, Wiebe, Machado, Ricker, Lever, Roe.

I agree with nearly everything said by your contributors so far, particularly Riley.

25. Emeritus Professor Chris Morfey, Acting European Editor, JSV, Fluid Dynamics & Acoustics Group

The Editor's role needs to be kept distinct from that of reviewer. Prescreening by the Editor is essential to avoid wasting reviewers' time. I have a team of Editorial Advisors to whom I can turn for advice on (a) technical suitability of a manuscript for the Journal, (b) suggested reviewers. In exceptional cases, the Editor may act as reviewer by writing an anonymous report in the usual way.

24. James Heath, Editor, Thermal Biology

Item A. generally the procedure I have followed.

Item B. I review each paper to select appropriate reviewers. On occasion I will reject a paper outright because 1. It does not fit the theme of the journal. 2. It is clearly not timely research or is of poor preparation or structure or it does not meet the quality standard of the journal.

Item C. I am opposed to "broad casting " to find a reviewer.

23. Thomas Knudsen, Editor, Reproductive Toxicology

Disagree with answer A

Agree with answer B

Disagree with answer C

22. S.C. (Steve) Sheppard, Editor-in-Chief, Journal of Environmental Radioactivity

My approach is to do a quick review of the paper myself to ensure it is worth the reviewer's time. I reject about 10% outright. For the remainder, I ask 4 to 6 people if they will do a review, and then I send the paper to all who accept. I require at least 2 reviewers, and I rarely get more than 3. Only once was one poor author lumbered with 6 reviews, his paper was excellent and unusual, and that is why so many reviewers were interested. I always provide detailed editorial and review comments along with those of the reviewers, and only rarely disguise myself as one of the reviewers.

21. Martin R. Okos, Editor, Journal of Food Engineering

A) An article should only require two referee reports. That's excluding the editor's review because that should not be counted.

B) The editor's review is as valid as any other referee report. As a matter of fact, if the editor first reviews the paper and it is not according to standards the paper can be rejected immediately.

C) If you receive more than two referee reports by sending out requests to as many referees as possible, that is acceptable. I actually agree with all three statements depending on the circumstances. Two referee reports are usually enough however there are situations when a third report is valuable especially if one of the reports has few comments or if two reports have vastly different conclusions. Many times as a matter of course, I send to three reviewers in case one reviewer cannot review or is unduly delayed. Also an editor should look at an article and determine if it fits the editorial policy of the journal and thereby saving the time and energy of the reviewers.

20. Eric Johnson, Editor in Chief, Environmental Impact Assessment Review

My answer is B. It is a waste of reviewers' time to act otherwise. Answer A is usually valid, but not always. Sometimes one good review is enough, sometimes 3 or even 4 reviews are needed. Answer C is oddly phrased. Yes, more than 2 reviews is fine, but "as many referees as possible" is odd. If you mean, say, 30-40 referees, then probably no.

19. Brian Shackel, editor, . International Journal of Human - Computer Studies

My choice of the three alternatives offered is B. However, I personally in addition to taking the view of B also always sent to three referees if at all possible, because my opinion might be biased and I would like to have a clear decisive view from at least 2 out of 3 scientists to guide my final decision.

18. Ian D. Wilson, Co-Editor, Journal of Pharmaceutical and Biomedical Analysis

Option B. If, on reading the manuscript, it is not in the scope of the journal, or is incomprehensible because it falls below minimum standards of English, then the Editor has a duty of care to the referees not to waste their time in asking them to do something on a paper that either will not be published (because it is outside the scope) or cannot be published because it is unreadable.

If a paper is outside the scope of the journal the editor should explain this to the authors and suggest suitable alternative publications, if unintelligible the editor should offer to re-evaluate a new version of the manuscript written with the help of a native English speaker.

17. P.A. Pirazzoli, Editor of Global and Planetary Change

In my opinion, at least two reviews are necessary and the editor can be one of the reviewers, but not the only one. If there is strong disagreement between the two reviewers, a third review is necessary before a final decision is taken.

16. Anna Wang Roe, Editor, Neuroscience Letters

Requiring two non-editor reviews should be standard. Requesting three is somewhat beyond what I would expect for submissions to Neuroscience Letters. Even very high profile journals typically request only two. My strategy is to invite only two or at most three reviewers at a time. If three reviewers accept (which is unusual), I typically decline the last acceptance. I also believe that if the editor also has expertise in the particular area, that should and would be factored in to the editor's decision, especially in cases in which the two reviewers do not agree. Furthermore, if a manuscript is obviously flawed or poorly written, it is the editor's prerogative to reject without review.

15. Ellen Thomas, Editor, Marine Micropalaeontology

In my opinion one can not have a fixed answer to this - it varies.

I usually do not count my editorial review; in most cases I reinforce what the reviewers stated, and look at the technical aspects (adhesion to editorial policy). There are exceptions: when I experience long delays in obtaining a second review, and the topic of the manuscript is directly within my own expertise, and the first reviewer has a positive opinion with which I fully agree, I will write my own review and count it as such in order to expedite publication. I do not do so when I disagree with the first reviewer.

In cases where a manuscript covers a wide range of expertises I may ask 3 or 4 reviews in order to make sure that the manuscript does not suffer a major error in one of the topics.

And then, one sometimes ends up with more than two reports because a reviewer may be very late, a new reviewer found, and the first review still arrives. In that case I do send on all reviews.

14. Barry Lever, Editor, Coordination Chemistry Reviews

While two reports would be 'normal', there is no reason not to have more- three is not unreasonable but 4 or more would be overkill except in special circumstances where the paper is very controversial or out of the mainstream. Certainly an editor should feel able to reject a paper without external review if he/she judges it unacceptable.

13. N. Lawrence Ricker, Editor, Journal of Process Control

Answer B. The editors view should count, and the editor should reject a sub-par paper immediately without requesting peer reviews.

Three peer reviews is better than two.

Requesting reviews from more than the required number will cause bad feelings in the long run. With today's rapid communication media there's no reason to do this. One can solicit the desired number, then ask others if the initial requests are refused.

12. James Mayer, Editor, Materials Science & Engineering B

Review procedure acceptable. Two referees plus editor is acceptable.

11. Chris Ryan, Editor, Tourism Management

a) editors should at the very least skim articles and then read in more detail if it is felt an article needs to be rejected. Referees need to be protected against poor work!

b) I send articles to 3 or 4 referees, but will work on two referee reports if there is a congruity between those reports as authors deserve 'good' turn round times. Subsequent referee reports are sent to authors when received if an amended paper has not yet been received.

10. Antonio Machado, Editor-in-Chief, Journal for Nature Conservation

Answer A is OK, but it is important to distinguish two roles played by the Editor-in-Chief. As Editor-in-Chief I check all submitted papers for consistency with the editorial policy of our journal, but this cannot be considered properly a review. At this point (1 step), a given paper may be rejected despite its excellent quality (I normally give some information to the authors about other journals that are better suited for the content of their papers). Only when I consider the paper's theme and scope "suitable" for our Journal, the process of peer reviewing will initiate (2 step). Occasionally, I may act as a reviewer if the topic fits my specific speciality, but normally it will be forwarded to two specialists. These specialist are chosen either from our Editorial Board or from the outside, depending on the topic.

9. Leonard I. Wiebe, Editor, Applied Radiation and Isotopes

I believe that the Editor’s opinion regarding suitability for the Journal is very important. As a reviewer for other journals, I concentrate mainly on the manuscript/content, leaving the question of suitability to the Editor. If the Editor deems a MS inappropriate, and rejects the MS on this basis, this saves everyone, including the referees, a lot of work. As an Editor, I find it increasingly difficult to enlist the services of top referees, and I certainly don’t want to waste their time on a MS that has a high risk of rejection because of the poor fit with the Journal’s objectives. Vote: B.

8. Liz Hamp-Lyons, Editor, Journal of English for Academic Purposes, and Assessing Writing

A is certainly the primary answer. B is in 2 parts; yes, the Editor's initial reading and judgement of whether the article meets the basic requirements of this journal (relevance, adequate use of English) is important, but then, I think, no, the peer reviewers take over and the Editor should not consider her/himself one of them [although sometimes one Co-Editor may serve as a peer reviewer for a ms a different Co-Editor is handling]; it is sometimes necessary to get more than 2 reviews if there is wide disagreement and if there are different disciplinary paradigms or knowledge areas within the paper or the reviewers' stances. Fundamentally, the handling editor of any submission should hold her/himself at a remove from the articles s/he is handling.

7. Professor Keith Attenborough, Editor-in-Chief Applied Acoustics, Associate Editor (Atmospheric Acoustics) Acta Acustica.

I prefer option (c). Obviously the more reviewers the better but typically it is difficult to obtain more than two. If the subject of the paper is close to the editor's research expertise then it is sensible for the editor to supply one of the reviews. Typically I manage to find time for this on very few occasions but one of my Associate Editor colleagues reviews every paper that he receives!

6. Professor Ann M Thomson, Editor-in-Chief, Midwifery (an International Journal)

It depends, on the article. A minimum of 2 but I have used up to 4 when the paper had relevance to more than one type of country and where the referees did not have the methodological expertise to review all the methods.

5. Dr. Gio B. Gori, Editor-in–Chief, Regulatory Toxicology and Pharmacology

The number of referees is a judgment call for each submission. Two referees may be a reasonable rule of thumb, but more may be needed if some reviews are superficial, if there is controversy among referees, or if the submission is multidisciplinary. The editor’s first review is sufficient to reject incompetent submissions without further peer review. Initially I seek consent to review from at least three peers, and usually receive two competent reviews within 5-6 weeks, a cut-off time that is seldom exceeded.

4. John P. O'connell

I agree with Dr. Riley; there should be no rigid policy with regard to the number of reviews. I feel that options B (Editor's review counts) and C (use as many reviews as received) can be appropriate, depending upon the breadth and depth of the paper, the expertise of the editor about the subject, and the availability of reviewers. In any case, we always should be prepared to explain our procedures and decisions to our authors and other editors.

3. Harold Kung, Editor Applied Catalysis A: General

A paper should have at least two referees, and the editor's opinion is as important as any other's. Two independent referees would give the paper a better chance of a fair consideration than only one. However, the more important factor is the quality of the referee, that is, the choice of referee is critical. The editor's opinion counts heavily in interpretation the referee's comments, in deciding the relevance and interest of the topic, and especially when the editor is knowledgeable in the subject. It is highly undesirable to send out requests to many referees and use the first two for decision. Good referees receive many requests, and we should all make use of their service in as optimal a manner as possible.

2. Chris Bagnall, Ph.D, Editor-in-Chief, Materials Characterization

We favor Option B. We have arrived at this method of operation over a period of years, as a result of changing patterns in manuscript submittal and an increased spectrum of manuscript quality. Time to publication is every editor's enemy and we now find it expedient to save on time investment that would otherwise be incurred by associate editors and reviewers by doing some "weeding-out" up-front. Reviewers are rebelling against the significant increase in lower quality work; sending this material out without an initial editor appraisal has two consequences: the reviewers are becoming frustrated by effort required to write copious notes for either rejection or a major revision, and they then begin to lose their positive feelings regarding the technical stature of the journal-because they see too much of the dark side, which should not/may not/will not make it to print. One of the few compensations for EAB members is that they have the opportunity to review new contributions to the literature in a field of study of significant interest to them. If too many manuscripts fall short of the term "contribution," there is a very real danger of losing the support and good-will of these hard-working reviewers. Multiple reviews tend to be self-defeating in that they generate controversy and result in mixed messages sent back to the authors. In the long run, if a manuscript does get into print with data or conclusions that raise serious questions of doubt, a "Letter to the Editor" is a valuable forum for offering alternate opinions and corrections. Especially now that manuscripts are accessible "on line," this form of peer review is potentially more valuable than any number of pre-publication reviews could possibly be.

1. David Riley, MD, Editor-in-Chief, Explore - The Journal of Science and Healing

Different papers require different numbers of reviewers. Papers covering multiple subject areas with methodological issues may require more than two referees. If the editors feels a papers does not meet minimum standards for publication in the journal it should be rejected by the editor or the editorial team before going out for review.