Elsevier Connect
Skip Navigation
Peer Review

Peer review: new improvements to an age-old system

As pressure on the system mounts, Elsevier tests initiatives to make it more efficient and appealing

Philosophical Transactions, Vol. 1 (1665)

A version of this article appeared in Editors’ Update.

The global research community has expanded enormously since the first peer-reviewed journal was published in 1665. Every year, more than 1.3 million articles are published in peer-reviewed journals, and the number continues to rise. The sheer volume of papers is putting pressure on the system, leading many to question its effectiveness.

Some are worried about bias and concerned it is not objective. Others are anxious about the length of time it takes for an article to go through the process, and some criticize its efficiency.

These and other issues emerged when more than 4,000 researchers responded to the Peer Review Survey 2009, which Elsevier conducted with Sense About Science.

In response, peer review continues to evolve. At Elsevier, we are working with our journal editors and reviewers to improve and streamline the process, ultimately easing the burden on both reviewers and editors. In this article, we describe our pilot programs and evaluate their progress.

Peer Review Challenge

This web-based challenge, which closed in May, invited submissions on any aspect that could significantly add to the current peer-review system. Entries could range from working within an existing peer-review method to designing a completely new system. The challenge also welcomed entries that explored how publishers and editors can help early-career researchers become reviewers, or how reviewers can be recognized by either their institutes or publishers.

Judges have selected 10 finalists, whose ideas are posted on the challenge website. After inviting comments from the research community, they are evaluating the finalists and will post the prize-winners in mid-August.

For more information on this initiative, please contact Clare Lehane, Executive Publisher, STM Publishing, c.lehane@elsevier.com

[note color="#f1f9fc" position="right" width=400 margin=10 align="alignright"]

Tell us what you think 

All these pilots have been launched with one aim in mind: to support and improve the peer-review process for the benefit of editors, authors and reviewers. e would love to hear your thoughts on these new approaches and your suggestions for improvements. You can either post a comment below or submit an idea for a story.[/note]

Cascading of manuscripts

If you’re an editor, you may frequently be confronted with manuscripts that contain sound research but are out of scope or are simply not suitable for the journal. With this in mind, we have developed the Article Transfer Service (ATS), which allows the paper to be moved to a more appropriate journal. Currently, editors in the fields of Pharma Sciences, Physics and Immunology are able to offer authors this option and, if the author agrees, we can promptly transfer the manuscript on their behalf.

The system has several advantages:

  • Editors can make faster, more informed decisions on manuscripts.
  • Authors receive faster decisions without the need to reformat or resubmit.
  • Reviewers benefit from a lighter burden due to a reviewer sharing policy where reviews have already taken place.
  • Authors can publish in a journal that maximizes the impact of their research. [caption id="attachment_11168" align="alignright" width="271"]Article Transfer Service at a glance Article Transfer Service (ATS) at a glance[/caption]

Results so far

  • Editors have offered to transfer up to 35% of rejected manuscripts, and up to 35% of offered transfers have been taken up by authors.
  • Up to 20% of those transfers have been accepted by the receiver journals.

We also surveyed a number of participants in the ATS scheme and discovered the following:

  • 67% of editors think the ATS benefits the authors, while 75% agree that having reviewer reports is beneficial.
  • 55% of authors are active promoters of the scheme.
  • 86% of the reviewers are willing to recommend an alternative journal to the editor.

For more information on this pilot, visit the ATS website or contact John Lardee, Senior Project Manager, Publishing Services, j.lardee@elsevier.com.

Results so far: Since the pilot was launched in January 2010, the journal has had an increase in papers (85 in 2011 and 74 in 2010, up from about about 12 papers a year previously. There has also been a sharp increase in usage: about 3,000 downloads per month compared to 2000 per month in 2009.

For more information, contact Charon Duermeijer, Publishing Director, Physics, c.duermeijer@elsevier.com[divider]

Reviewer Guidance Program

From feedback. we know that reviewers, especially those new to the task, would value more guidance on how to peer review. This program, still in the developmental stages, has been created to answer that need; it consists of teaching theory through workshops and providing hands-on practice through mentorship:

  • Reviewer Workshops. By attending a workshop, participants will be introduced to the concept and background of peer review as well as peer-review fundamentals, publication ethics and the role of a reviewer. They will also examine a specific case study. Reviewer Workshops have been taking place for a while now, and participants have told us that they feel more confident after attending one. Since it is not always possible to physically attend a workshop, we are now looking into the possibility of offering an online alternative.
  • Mentorship. This part of program is intended to give participants the experience of independently reviewing at least two manuscripts inside a specially-created EES (Elsevier Editorial System) site. Each trainee is supported by a mentor who discusses the reviews with them and gives feedback and guidance. The mentor finally decides when a trainee has gained enough experience to review live manuscripts. After the program, each trainee receives a certificate of participation from Elsevier. We began piloting this module at the end of last year, and the first feedback is promising. One trainee commented: “I’m now more familiar with rating papers and I’m more critical when I read papers.” The mentors involved in this module, often journal editors, also see the benefits of this initiative; one remarked: “This module is a nice opportunity to learn how to efficiently review manuscripts. Often junior scientists have no idea how it works. As well, they can better understand how their manuscripts will be reviewed.”
[/note]

During the Reviewer Guidance Program, we will guide participants in how to write review reports in such a way that they answer the needs of both the editor and the author. The program should also contribute to increasing the number of trusted — and usually enthusiastic — reviewers for editors to call on. In the words of Irene Kanter-Schlifke, who is involved in the pilot as a Publisher for Pharmacology and Pharmaceutical Sciences at Elsevier: “The experience not only helps early career researchers become better reviewers, but also to be more critical in analyzing their own papers before submitting. In addition, this is a great opportunity for junior scientists to network with their more senior peers.”

If you are interested in organizing a Reviewer Workshop at your institute, please contact your publisher at Elsevier.

Results so far: We are evaluating feedback and expect to do a further pilot in due course.

For more information on this program, please contact Irene Kanter-Schlifke, Publisher Pharmacology & Pharmaceutical Sciences, STM Publishing, i.kanter@elsevier.com, or Angelique Janssen, Project Manager, Publishing Services, a.janssen@elsevier.com [divider]

Published reviewer reports

Reviewers play such a vital role in the peer-review process yet their contribution often remains hidden. In addition, open reviewer reports increase peer-review transparency and assist good articles to gain authority. With that in mind, we thought: Why not publish reviewer reports alongside the final article on SciVerse ScienceDirect?

At the beginning of this year, we began doing that in the journal Agricultural and Forest Meteorology.

We know from the feedback we have received that editors welcome such a public acknowledgement of reviewers’ contributions; we hope this step will enhance the quality of the review reports and help attract good reviewers for the journal. 

How does it work?

The journal’s authors and reviewers are informed about the new process, and reviewers can indicate whether they want their name disclosed on ScienceDirect. Editors then decide if the reviewer reports are appropriate to publish alongside the article as supplementary material.

Results so far: The pilot launch attracted positive international media attention. It was also suggested that open reviewer reports could play a useful role in training early career scientists as reviewers. So far, reviewer reports have been published alongside about 13 manuscripts.

For more information on this pilot, please contact Gilles Jonker, Executive Publisher, Physical Sciences, g.jonker@elsevier.com [divider]

Open peer commentary format

In this pilot, we have asked experienced researchers to submit a one-page comment on a reviewarticle for the journal Physics of Life Reviews. All those comments are published in the same issue as the article. On average, five comments are published with the article and the author can write a rebuttal article.

[caption id="attachment_11170" align="alignnone" width="557"]An article with open peer commentary on SciVerse ScienceDirect

An article with open peer commentary on SciVerse ScienceDirect[/caption]

Results so far: Since the pilot was launched in January 2010, the journal has had an increase in papers (85 in 2011 and 74 in 2010, up from about about 12 papers a year previously. There has also been a sharp increase in usage: about 3,000 downloads per month compared to 2000 per month in 2009.

For more information, contact Charon Duermeijer, Publishing Director, Physics, c.duermeijer@elsevier.com[divider]

PeerChoice

Traditionally in peer review, editors have chosen to approach reviewers they consider qualified to comment on a manuscript, or who would find the subject matter interesting.

But what if the reviewers could select the manuscripts themselves? For a year now, we have been experimenting with this additional peer-review system in the journal Chemical Physics Letters. Each week, a selected pool of reviewers receives an overview of the new submissions. If they like a paper because it matches their expertise and interest, they can decide to review it. Because they make the decision themselves, we ask them to review the manuscript within a week.

This idea was inspired by feedback in the Peer Review Survey 2009. “We learned that a significant number of reviewers were sometimes hesitant to review an article because of a lack of expertise in that particular field,” said Martin Tanke, Managing Director of Elsevier’s STM Journals group. “In addition, researchers made clear they want to improve peer review by improving article relevancy and speeding up turnaround time. PeerChoice can contribute to solving both issues.”

[caption id="attachment_11172" align="alignnone" width="744"]An example of the email overview a reviewer receives

An example of the email overview a reviewer receives[/caption]

Results so far: The time taken to review the manuscript has been slightly reduced, while the time taken to accept an invitation has been halved.

For more information on this pilot, please contact Egbert van Wezenbeek, Director Publication Process Development, Publishing Services, e.wezenbeek@elsevier.com[divider]

Related links

A Helping Hand for Early Career Reviewers – Editors’ Update, Issue 32, June 2011

Article Transfer Service: Helping Authors Publish in the Right Journal – Reviewers’ Update, Issue 9, December 2011

[divider]

The Authors

[caption align="alignleft"]John LardeeJohn Lardee[/caption]

[caption align="alignright"]Adrian MulliganAdrian Mulligan[/caption]John Lardee is Senior Project Manager for Elsevier Publishing Services. For the last 15 years, he has been involved in managing projects to improve the experiences of authors, editors and reviewer experiences with Elsevier’s products and services. Recent projects include the Article Transfer Service and the Find A Reviewer tool. John’s approach to project management is an agile one: “To develop services and products iteratively together with our editors, authors and reviewers.” He has a Master of Science degree in informatics from the Delft University of Technology in the Netherlands. He is based in Elsevier's headquarters in Amsterdam.

Adrian Mulligan is Deputy Director for Research & Academic Relations, based in Elsevier's Oxford office. He has 14 years of experience in STM publishing. The last 10 years he has spent in research have given him the unique opportunity to study the scholarly community. In partnership with Sense About Science, Adrian helped design and conduct the Peer Review Survey 2009, which examined attitudes of more than 4,000 researchers towards peer review. He has presented on peer review at various conferences, including STM, ESOF, Association of American Publishers (AAP) and APE. Adrian’s background is in archaeology, and he has a BA Honours degree and a Master of Science degree from Leicester University. He also has a diploma in market research from the Market Research Society.



comments powered by Disqus