Elsevier’s Reviewer Recognition Platform prepares for next phase
Supporting and recognizing reviewers
By Professor Lutz Prechelt, Dr. Joris van Rossum and Dr. Bahar Mehmani Posted on 10 November 2014
Earlier this year, we reported on the new Elsevier Reviewer Recognition Platform in the Editors' Update article How we can better support and recognize reviewers.
Since the platform's launch, we have been monitoring the feedback and advice you have shared with us and we are now ready to begin work on phase 2. Recently, Professor Dr. Lutz Prechelt, Professor of Informatics and Head of the Software Engineering Research Group (AG SE) at the Freie Universität in Berlin, joined the program as an advisor.
In this article, Elsevier's Dr. Joris van Rossum and Dr. Bahar Mehmani take a closer look at reactions to the platform's launch and Professor Prechelt talks about our ambitious plans for the initiative's future.
What is the Reviewer Recognition Platform?
We know from reviewer feedback that while they find reviewing important, they can feel their work is near-invisible to the outside world and is hardly rewarded. The Reviewer Recognition Platform aims to change that. The platform provides participating reviewers with a personalized profile page where their reviewing history is documented. Moreover, reviewer statuses are awarded based on the number of reviews they have completed for a specific journal; a reviewer who completes at least one review within a two-year time period becomes a 'Recognized Reviewer', while those in the top 10th percentile become 'Outstanding Reviewers'. The platform also offers various discounts for Elsevier services, such as the Elsevier WebShop, and reviewers can download a variety of certificates. Future extensions will also describe the quality of reviewing (by means of percentiles among the reviewers of the particular journal) measured with journal-specific criteria.
Initial reactions to the platform
When the Reviewer Recognition Platform went live in March this year, there were 40 participating journals. It now features more than 260 Elsevier journals from a variety of disciplines and that number is growing - each month, we add the five-year review history of reviewers for 50 new journals. When a review has been completed, reviewers receive an email providing them with a direct link to their Elsevier review profile on the platform. So far we have sent emails to 7,700 reviewers.
Our survey results show that reviewers find the Reviewer Recognition Platform a valuable initiative. They have given the platform a score of 8.20 out of 10 (n=488). More than 68 percent of respondents found the information contained in their profile useful and more than 41 percent mentioned they plan to share their profile and status with others (their department head, colleagues, followers and friends on their social media channels, etc.). We have also received a number of novel ideas and suggestions from reviewers about how we can target rewards and further recognize their invaluable efforts.
Professor Dr. Lutz Prechelt's story
I joined the Reviewer Recognition Platform project as an advisor, because I have been thinking about initiating a similar program – the Review Quality Collector – since 2012. I would like to ensure scientists are publicly recognized for good reviewing performance, journals receive reviews of the best possible quality, and research institutions include reviewing quality information in their evaluation of scientists' overall performance. Elsevier's Reviewer Recognition Platform had already started and could be extended to include the Review Quality Collector's goals, so we decided to join forces.
During the next stages of the project, we want to onboard more journals to the platform. Also, we want reviewers to be able to download their review history so that the reviewing record can easily be shared.
The crucial next step, however, will be the introduction of quality elements in the reviewer status. So far, the status is based on the number of completed reviews only. We want editors and authors to provide feedback about the helpfulness of the review and also record timeliness so that we can take these important aspects into account when recognizing reviewers.
We have already started the process by asking editors and authors of selected journals a few simple questions right after the reviewers have delivered their reports. This includes asking them to rate the submitted manuscript reviews in a generic fashion. Over the next couple of months, we expect to tweak the questions, analyze the results, and hold discussions with editors. Our aim is to arrive at a criteria framework that can be customized for each journal in such a way that it will provide a sound method to gauge the qualitative contribution of a reviewer.
In the long term, we hope the statuses and procedures we develop will evolve into industry standards that will be adopted by other publishers. It is time reviewers are appropriately recognized for their important contribution to the progress of research.
Professor Dr. Lutz Prechelt became full Professor of Informatics at Freie Universität Berlin in 2003 after working as a manager in the software industry for several years. His PhD in Informatics is from Universität Karlsruhe (now KIT), Germany. His research is primarily concerned with understanding the human aspects of software development better. He has always been deeply interested in issues of research methodology and research quality. He considers himself a thorough reviewer.
Dr. Bahar Mehmani is Publishing Innovation Manager in the Innovation and Publishing Development department (IPD) at Elsevier's Amsterdam office. She is working on a number of reviewer-related projects, all of which are designed to recognize reviewers' contribution to the progress of science. She received her PhD in Theoretical Physics from the University of Amsterdam (UvA) in 2010. Before joining Elsevier, she was a postdoc researcher at Max Planck Institute for the Science of Light (MPL).
For the past 12 years, Dr. Joris van Rossum has been involved with the launch and further development of many products and initiatives within Elsevier. From its inception he was a Product Manager for Scopus , the largest abstract and citation database of peer-reviewed literature, and he worked on Elsevier's search engine for scientific information as Head of Scirus. Later, he developed the Elsevier WebShop, which offers support and services for authors at many stages of the publication workflow. As Director Publishing Innovation, Dr. van Rossum is focused on testing and introducing important innovations with a focus on peer review. He holds a master's of science in biology from the University of Amsterdam, and a PhD in philosophy from VU University Amsterdam.
Prof David J Lowe says: November 14, 2014 at 11:44 am
I applaud this initiative. I enjoy reviewing (mainly) but it is almost always after hours. Some recognition would be great.
Where can I find my reviewer profile please? I review 6 or more papers each year for Elsevier (Quaternary journals, soils etc).
Reviewer Recognition team says: November 17, 2014 at 8:23 am
Dear Professor Lowe,
Thank you very much for contacting us and your kind words. The Reviewer Recognition platform is currently in its Beta version. This means that not all Elsevier journals have yet joined the initiative. However, we are adding new journals on a regular basis and the chances are high that your journals will be in the next couple of batches – probably early next year. We will inform you as soon as your profile is updated.