What you think about the peer-review process
We turn the spotlight on the results of Elsevier studies reflecting the views of more than 3,000 active researchers
By Adrian Mulligan and Dr. Joris van Rossum Posted on 14 September 2014
Peer review is the cornerstone of scholarly publishing, but it can be a source of frustration for researchers. Criticisms levelled against it include claims that it slows down publication speed, consumes too much time and is inherently biased.
In this article, we draw upon the results of Elsevier studies that reflect the views of more than 3,000 active researchers. We not only examine their thoughts on the current peer-review landscape, but explore their expectations for the future.
What did we find out?
Most researchers – 70 percent – are happy with the current peer-review process; a satisfaction rate higher than those recorded in similar 2007 and 2009 surveys. When asked if peer review helps scientific communication, 83 percent of those we surveyed agreed, with comments such as, "I have had reviews that were very insightful. When researchers get their nose caught in the lab book, we cannot see the forest through the trees. Having a peer look at your science helps expand the overall view". (Researcher in Environmental Science, Switzerland, aged 36-45.)
However, there is room for improvement; a third of researchers believe that peer review could be enhanced. The improvements they suggested included reviewers being given more guidance, and incentives to encourage reviewers to volunteer their time. Moreover, 27 percent of respondents felt that peer review is holding back scientific research. A Computer Science researcher from the UK explained: "Peer review for journals is often slow and biased. Radical ideas will be rejected as peer review is inherently conservative."
Is pressure increasing on reviewers? Perhaps surprisingly, only 29 percent of researchers surveyed believed that to be the case. However, that is 10 percent more than in the survey conducted in 2009. Reasons cited include time pressure, lack of incentives, lack of specialist knowledge and too many (poor-quality) papers being sent for review.
If we take a look at which countries are bearing the brunt of increased pressure, it seems that US researchers are reviewing more papers than they are writing. On the flip side, Chinese researchers are writing substantially more papers than they are reviewing. The reason for this gap is that Chinese researchers are not being asked to review as frequently, even though they have one of the highest review acceptance rates for any country.
Anonymity appears to be important to researchers when it comes to the review process. We asked them about peer-review options ranging from double blind and single blind to various forms of open peer review and discovered that, as transparency increased, the likelihood of authors submitting papers decreased; 46 percent said they were unlikely to submit to a journal which published the reviewer's name alongside reviewer reports. It would appear some authors do not like the idea of their mistakes being made public. A slightly greater proportion – 49 percent – indicated they would be less likely to review on a journal that offered such open peer review.
Peer review has many purposes from improving the quality of a paper to detecting fraud. Clearly, high expectations are placed upon the process and for the most part researchers believe peer review delivers. However, there are two areas where respondents indicate it is failing - detecting plagiarism and fraud.
Listen to our free peer-review webinar
The contributors to this article, Adrian Mulligan and Dr. Joris van Rossum, recently hosted a webinar The peer review landscape – what do researchers think?. The archive version is now available to view.
What is Elsevier doing to address some of the peer-review issues identified?
There are a number of projects underway at Elsevier to support reviewers and the peer-review process.These include:
The new Reviewer Recognition Platform
This initiative provides participating reviewers with a personalized profile page where their reviewing history is documented. They are also awarded reviewer statuses based on the number of reviews they have completed for a specific journal – see Elsevier's Reviewer Recognition Platform prepares for next phase in this issue to discover how the project is evolving.
Integration of CrossCheck into EES
The plagiarism tool CrossCheck has now been integrated into EES (Elsevier's Editorial System) for a large number of journals. Once an author has submitted a manuscript, EES will automatically upload the editor PDF to CrossCheck's iThenticate website, where it will be checked against a huge database of publications. Editors can then view an automatically generated similarity report within EES. Over the past few months, several feature enhancements have been introduced and we will continue to roll out the tool with the aim of making it available to all journals by the end of 2014.
A new direction for the Article Transfer Service (ATS)
As outlined in the article How we can better support and recognize reviewers in Reviewers' Update Issue 18, for some time now, editors of 600 Elsevier journals have been able to suggest alternative journals to an author whose paper they have just declined. If the author agrees to submit to the new journal, any completed reviews can be transferred along with the paper.
In the article, we mentioned that a new experiment with six Elsevier soil science journals aims to improve on the service by offering editors who decide not to accept a paper, two important new options in EES; decline and reject. Decline simply says the paper is not suitable for the journal and allows the author to submit the paper – and reviews – to another soil science journal in the pilot. Reject means the author will not be invited to submit elsewhere.
Early results are proving encouraging with the number of manuscripts offered for transfer doubling under the new system. Dr. Joris van Rossum, Director Publishing Innovation at Elsevier, commented: "Not only do we see significantly more manuscripts offered a transfer, we also find many more authors accepting the offer to submit the article to another journal. We are hopeful that, in this way, authors will find the final destination for their manuscript faster and more efficiently."
Adrian Mulligan is a Research Director in Elsevier's Research & Academic Relations department. He has more than 15 years' experience in STM publishing and much of that time has been spent in research. He oversees Elsevier's Customer Insights Programs, ongoing tracking studies used to drive action in the business and help shape Elsevier strategy. Alongside these, Research & Academic Relations works in partnership with external groups to deepen understanding of the scholarly landscape across the industry. He has presented on a range of research-related topics at various conferences, including STM, ESOF, AAP, SSP, APE and ALPSP. Mulligan's background is in archaeology; he has a BA Honours degree and a master's of science from University of Leicester. He also has a diploma in Market Research from the Market Research Society.
For the past 12 years, Dr. Joris van Rossum has been involved in the launch and further development of many products and initiatives within Elsevier. From its inception, he worked as a Product Manager on Scopus, the largest abstract and citation database of peer-reviewed literature, and he worked on Elsevier's search engine for scientific information as Head of Scirus. Later, he developed the Elsevier WebShop, which offers support and services for authors at many stages of the publication workflow. In his current role as Director Publishing Innovation, Dr. van Rossum focuses on testing and introducing important innovations with a focus on peer review. He holds a master's of science in biology from the University of Amsterdam, and a PhD in philosophy from VU University Amsterdam.