Peer review

Reviewers play an essential part in research and in scholarly publishing. Elsevier, like most academic publishing companies, relies on effective peer-review processes to uphold the quality and validity of individual articles and the overall integrity of the journals we publish. However, we know how difficult it can be to not only find and keep reviewers, but encourage them to meet your deadlines. Below we cover a range of topics from sourcing good reviewers to some of the avenues we are exploring together to improve and streamline the peer-review process.

Finding reviewers

We appreciate that finding new reviewers can be challenging. Here are some of the ways we can help:


  • ees-scopus-search-barVia the Scopus search bar in your EES assignments page, you can search Scopus to identify potential reviewers, link to their published work and citation histories, see who their co-authors were and set up citation alerts to keep up to date with who is citing which research.

  • Additionally, reviewers can access the full text of articles for Elsevier publication in ScienceDirect, significantly helping reviewers in the peer-review process.

  • Your publishing contact can help you create a reviewer classifications list for your journal, from which reviewers indicate their areas of expertise and use this to search for matches with the manuscript classifications.

  • Within EES, you can build a database of relevant reviewers by assigning classifications and adding keywords or notes to the people notes field, search for reviewers matching the manuscript's keywords or classifications, automatically un-invite reviewers when they don't respond in sufficient time, and set up automatic alternate reviewer invitation.


editor-top-tipsEven when you have a useful pool of reviewers, selecting the right one can require some thought. Editors have advised us that the following might be helpful points to bear in mind:

  • Try to select reviewers who are doing research in a related area; they are more likely to find the paper relevant and interesting and so respond promptly, and are also best placed to spot missing references and other shortcomings.

  • Make use of Editorial Board members for reviewing, and consider rotating off Board members who are not regularly refereeing.

  • Think twice before using reviewers who have not been active in research in the last five years.

  • The best and most willing reviewers are often young professors, researchers, postdoctorates, emeritus professors and authors who have recently published in the journal.

  • Some of the slowest reviewers are mid-career professionals, executives and people who have never published in the journal. However, these individuals can be very good at referring manuscripts to other people and expanding your pool of reviewers.

  • You should invite only as many reviewers as you will need. Inviting more reviewers than are needed and using only the first reviews to be returned can cause reviewers to feel unappreciated, and conflicting reviews can come in after the author has already been informed of your decision. EES has an automated function that can help manage the invitation process.

Supporting reviewers

rfp_headerSince February 2006, we have been surveying reviewers from a number of journals across a range of subject areas, trying to discover how we can best offer them the support they need. The objective of our Reviewer Feedback Programme* is to assess how satisfied reviewers are with the experience of reviewing. Below are some of the key findings, which we hope will provide more insight into reviewer motivation.

Relevant manuscripts

Reviewers are under considerable time pressure and would prefer not to receive manuscripts that are of such poor quality that they could be rejected outright**

“I think that nowadays our reviewing overload (I usually have at least one manuscript on my desk continuously to review) should be reduced by a first screening of the submitted manuscripts.” (Aged 36 to 45 from Italy)

“Bolster a triage system to remove papers that have no chance of being accepted.” (Aged 36 to 45 from United States)

What can we do?

  • We have a technical screening programme which subjects all submitted manuscripts to an initial screening, purely on the basis of technical standards – adherence to the Guide for Authors and English language quality. Only those manuscripts that pass this screening will be passed on for peer review. For more information see our technical screening page.
  • Reject very poor or out of scope papers outright without sending them to a reviewer. Research into reviewer motivation shows that receiving too many bad papers is the strongest demotivating factor.
  • Your publishing contact can advise you about cleaning your EES database of reviewers and assigning keywords 
     

Feedback

Reviewers would like to know the final editorial decision of the paper

  • 87% would like to see other reviewers’ comments**
  • 90% of reviewers would like to be able to see the final decision**

“It would be useful to know the final decision and be able to see the other reviewer's comments... This would give me confidence that there was agreement or allow me to understand why a difference of views was expressed - and so improve my ability to review.” (Aged 46 to 55 from United Kingdom)

“It would be nice to know what happened to the article: Was the decision taken to publish it? What feedback did you get regarding my comments to the author? I would have liked to receive a copy of other reviewers’ comments... It feels like we send information that gets lost in a black hole!” (Aged 46 to 55 from United Arab Emirates)

What can we do?
Your publishing contact can ensure that this functionality is switched on in EES for your journal.


Recognition

Reviewers are grateful for efforts, however small, to show appreciation of their work**

“… I feel reviewers should receive some form of recognition for their time commitment. Perhaps a listing of all reviewers used in each issue …” (Aged 36 to 45 from Canada)

“I would like to know if you publish an annual list of the reviewers. It would be very good for people working at universities to have the opportunity to demonstrate that we are reviewers of a journal.” (Aged 46 to 55 from Spain)

What can we do?

  • Annual listing of reviewers in the journal; or
  • Certificates - click here for an example.


Reviewer policy: Reviewers would benefit from having more information and guidance from the outset of the review process, including timely and useful reminders**

“As far as I know there is no guidance offered as to what is expected of the reviewers except to get the review done by a certain date.” (Aged over 65 from Canada)

“More specific guidelines for how long the review should be, if there are particular aspects of the article I should focus on most…” (Aged 26 to 35 from United States)

“I would have preferred the reminders to come before the deadline had passed ...” (Aged 26 to 35 from United States)

“Sending a reminder letter before the deadline of the review process will be helpful to keep the deadline.” (Aged 26 to 35 from Japan)

What can we do?

  • Establish a peer review policy outlining what is expected of reviewers which can be published in the journal and on the journal’s homepage, and linked to from the reviewers' invitation letter. Click here to see a template review policy, which can be used as a starting point.
  • Develop a set of clear reviewer guidelines – your publishing contact can provide examples to use as a starting point. You can also view the Elsevier Reviewers' Guidelines on Elsevier.com.
  • Findings suggest that reviewers are sometimes unaware of the deadline to return their reviewer report. Including the deadlines in the reviewer invitation letter would help address this.
  • Set up automatic reminders for reviewers and customise reviewer letters to include deadlines, what is expected of them and where they can turn to for help e.g. Scopus.


* The Reviewer Feedback Programme is an online survey, similar to our established Author and Editor Feedback Programmes, which includes questions covering topics such as: reviewers’ overall satisfaction, their willingness to review again for the journal, the influence of the perceived reputation of the journal, interaction between both the journal editor and publisher and the quality and relevance of the article. Reviewers are also asked to make comparisons with non-Elsevier published journals. The aim of this programme is to get an understanding of reviewers’ wishes, their opinions of our online peer review system, EES, and their view of the overall peer review process. The feedback we receive is used to develop EES and other reviewer offerings, as well as to provide an insight into reviewer motivations.

**Findings are based on the June 2006 round of the Reviewer's Feedback Programme, drawn from 5,030 responses, where the error margin was ±1.2% at 95% confidence levels. The results were representative with reviewers drawn from all the major science disciplines.

Innovation

The expansion of the global research community and the year on year increase in the number of published papers means the pressure on the peer-review system has grown. We are always seeking new ways to work together with you to improve and streamline the peer-review process, ultimately easing the burden on both reviewers and editors.

Over the years we have launched a number of pilots and initiatives, some of which have eventually been rolled out to a wider group of journals. A number of those trialled recently are outlined below. If you have a suggestion that could improve peer review, please do contact your Publisher.

Peer Review Grand Challenge

peer-review-challengeLaunched in the summer of 2012, this web-based Challenge invited submissions on any aspect that could significantly add to the current peer-review system. Entries could range from designing a completely new system, to working within an existing peer-review method (like the single blind system).

The Challenge also welcomed entries that explored how publishers and Editors can help early career researchers become reviewers, or how reviewers can be recognized by either their institutes or publishers.

Details of the three winning entries can be found on the Challenge website.


Cascading of manuscripts

article-transfer-serviceAs an editor, you may frequently be confronted with manuscripts that are out of scope or are simply not suitable for the journal; however, they still contain sound research. With this in mind, we have developed the complementary Article Transfer Service (ATS) which allows the paper to be moved to a more appropriate journal. Completed reviews are also transferred, reducing the burden on reviewers. 

For more information read our Reviewers' Update article Article Transfer Service

 

The Reviewer Guidance Program

From feedback we know that reviewers, especially those new to the task, would value more guidance on how to peer review. This program, which is still in the developmental stages, has been created to answer that need and will consist of both theory and hands-on practice.

For more information read our Editors' Update article Exploring Improvements to the Peer-Review System


Published reviewer reports

Reviewers play such a vital role in the peer-review process yet their contribution often remains hidden. In addition, open reviewer reports increase peer-review transparency and assist good articles to gain authority. With that in mind, we thought why not publish reviewer reports alongside the final article on SciVerse ScienceDirect?

For more information read our Editors' Update article Exploring Improvements to the Peer-Review System


Open peer commentary format 

In this pilot, we ask experienced researchers to submit a one page comment on a (review) article for the journal Physics of Life Reviews. On average, five comments are published with the article and the author can write a rebuttal article.

For more information read our Editors' Update article Exploring Improvements to the Peer-Review System


PeerChoice

peerchoicefaqTraditionally in peer review, editors have chosen to approach reviewers they consider are suitably qualified to comment on a manuscript, or who would find the subject matter interesting. But what if the reviewer could select the manuscript themselves? We have been experimenting with this additional peer-review system on the journal Chemical Physics Letters.

For more information read our Editors' Update article Exploring Improvements to the Peer-Review System

Further reading

2009 Peer Review Study with Sense About Science

SASLogo_100In September 2009, Elsevier partnered with Sense About Science, an independent NGO working to promote the public's understanding of 'sound science' to launch the 2009 Peer Review Study, the largest survey ever international survey of authors and reviewers providing insights into questions such as: Should peer review detect fraud and misconduct? What does it do for science and what does the scientific community want it to do? Will it illuminate good ideas or shut them down? Should reviewers remain anonymous?

Sense About Science also produces booklets. Those with a focus on peer review include:

Read more about Sense About Science's work on peer review


Other resources

ReviewersUpdateLogo200x100Is Peer Review in Crisis? This 2004 paper by Adrian Mulligan examines the role of peer review and discusses if, and how, it could be improved.

Reviewers' Home - our Elsevier.com pages offering support to the reviewing community.

Reviewers' Update - regular newsletter for reviewers offering information on developments in peer review and the Elsevier submission & peer review system.

Webinar – A 20:20 Vision on the Future of Peer Review - discusses the challenges editors face with the traditional peer review system vs. emerging models and analyses the impact of the peer review system on you as an editor and what role Elsevier can play to help overcome challenges.