Editors understand that attracting and retaining the best reviewers is based, in part, on how reviewers feel about the experience. But what do reviewers want and expect, and how do they really feel about reviewing? We ask Adrian Mulligan, Associate Director of Research & Academic Relations, and Laura Hassink, VP Strategy and Journal Services, about Elsevier’s Reviewer Feedback Program.

As an editor, you may already have experience of the Reviewer Feedback Program (RFP), either because your journal is in the program or because you have participated directly as a reviewer for another journal. Certainly, as an editor, you understand the challenges involved in attracting and retaining the best reviewers. Improving the peer-review experience is one way that Elsevier can support reviewers and editors, and the RFP is how it collects feedback on how reviewers feel about the process.

The initiative is entirely voluntary. Editors elect to have their journal included, and with almost 900 already participating, enthusiasm is high. Reviewers on participating journals are sent a questionnaire after they have reviewed a manuscript for the journal, but no reviewer is asked to provide feedback more than once a year, regardless of how many reviews they perform. Again, interest is strong, with a 40% response rate.

So far, the results have been good: over 90% of reviewers would be happy to review for again for the journal and 89% are happy with the overall process, says Adrian Mulligan, Associate Director of Research & Academic Relations. “We are up three points in two key service areas over the last 12 months: communication between editors and reviewers, and support for tools and services provided by the publisher. And it is also good to see that such a high number of reviewers - 91% - agree that the articles they are sent to review are appropriate for their areas of expertise,” he adds.

Responding to feedback
The RFP tracks key parts of the review process, such as the quality of the articles they are sent, the clarity of instructions, the relevancy of the article, the reasonableness of the deadline, as well as any support tools. Feedback is measured, reported and benchmarked carefully.

The RFP has established that reviewers value feedback from editors and the opportunity to see reports from other reviewers. And it might be surprising to learn that the relevancy of manuscripts is a far more important criterion when deciding whether to review than quality.

“Quality is still important though, and we can improve in this area by making sure that fewer poor quality manuscripts are sent to reviewers,” says Mulligan. A possible solution is to increase the number of manuscripts that are rejected prior to peer review. On some journals, editors are rejecting as much as 30% of manuscripts at this stage. For journals with an above-average number of poorly written papers, Elsevier has introduced technical screening. This service prescreens papers (and returns poor-quality manuscripts to the authors) before they land on a reviewer’s desk.

Not only is quantitative data captured in the RFP, so too is the voice of the reviewer. Comments made by reviewers are particularly valuable, as Mulligan explains: “If a reviewer disagrees with a statement, he or she is invited to explain why, the insights collected are assessed, and are used to help target improvements.” They have been used to help develop the peer-review process in terms of the technology, reducing bureaucracy and speeding up the process.

Endlessly improving
The RFP also gathers data on peer-review generally. While there are issues common to all, what reviewers think is important can be specific to the field, and can vary from journal to journal. For instance, various communities have expressed different preferences for the type of peer review (open, blind or double blind).

“Our aim is to improve quality and speed, but this means quite different things for different journals,” says Laura Hassink, VP Strategy and Journal Services. “We need to listen to as many points of view as possible, so we can effectively target weak areas and tailor our responses to meet the needs of each community.”

Hassink believes that this is a mutual and ongoing exercise: “We are working closely with editors and reviewers to improve both the peer-review process and our services to reviewers.”

Editors want to attract the best reviewers and make efforts to improve the peer-review process on their journals; and for this, they have Elsevier’s full support. From collecting and analyzing feedback to offering guidance, and solutions, the RFP is providing the tools necessary to identify strengths and weaknesses in every journal’s peer-review process.

To cite this article, please use: Michelle Pirotta, “Reviewing the review process”, Elsevier Editors’ Update, Issue 27, August 2009

References:
1. Reichheld, F.F. (2003) ‘The one number you need to grow’, Harvard Business Review, HBR, December 2003.