Adrian Mulligan, Associate Director of Elsevier Research & Academic Relations, explains how data collected from authors and reviewers can have an impact on specific publications, or on the publishing process itself.

“Through our various feedback programs, we can clearly see the issues and concerns most relevant to key contributors, and we can address them directly,” Mulligan says. “This ensures that we have a benchmark for performance and that we are constantly honing and improving our processes.” In this way, he explains, Elsevier remains connected to the community it serves.

Listening to our authors
The Author Feedback Program (AFP) is designed to collect relevant data from authors about their recent experience of publishing in a journal. It covers the whole range of the publishing experience, from submission through the peer-review process to final publication.

More than 86% of the journals eligible for involvement in the AFP are currently enrolled. “Each month, around 15,000 authors are invited to complete an AFP questionnaire,” Mulligan says. The corresponding author on each article is approached. “We have an average response rate of about 33%, which is remarkable for this type of survey.”

By collecting and analyzing feedback from journal authors, Elsevier is uniquely positioned to help guide development to make real improvements. Consistent negative feedback about peer review, for example, would likely mean an evaluation of that journal’s peer-review process was required. The program highlights specific weaknesses and is a starting point for discussion on how best to improve performance. In addition to the snapshot reports available every six months, journal trend reports are also available. Journal performance can be seen over a three-year period, and reports show significant trends in overall journal performance. Impacts of any new processes can be easily discerned over these longer periods.

Data collected through the AFP is not only used to facilitate development on a specific journal’s performance, but affects global Elsevier policies and procedures as well. For example, consistent feedback about the importance of publication times ensured that Elsevier invested in and developed online submission channels. The introduction of this online service meant significant time savings during the entire publishing process, and led to higher author satisfaction. “Compared to previous methods of submitting manuscripts, namely through regular or electronic mail, the online submission system offers significant time and cost savings, and allows for a more streamlined and efficient submission process,” Mulligan explains.

Reviewers’ review
Similar to the AFP, the Reviewer Feedback Program (RFP) was established in June 2006 as a way for Elsevier to collect and analyze feedback from a reviewer’s perspective. The unique and critical role of reviewer carries with it a specific set of responsibilities – and concerns. By tracking reviewer opinion, we are ensuring that we are best placed to provide appropriate support.

“Reviewers have a specific set of expectations and criteria for determining whether a review process was satisfactory or not, and this carries over into whether they are willing to review for a particular journal in the future,” explains Mulligan.

When invited to review an article written by a peer, a reviewer is most concerned about the relevance of the article. This makes the search and selection process critical to choosing the right reviewer for a manuscript.

“Reviewers are very busy people,” Mulligan elaborates. “They are first and foremost concerned with their own research and writing, and the good reviewers are often inundated with requests to review. Unsurprisingly, how relevant an article is to their expertise is absolutely key to the reviewer’s decision making.”

In addition to relevance, reviewers consider the quality of the article, the amount of time allotted for the review process and the reputation of the journal as key factors in their decision.

By employing the RFP, a journal can measure the effectiveness of their reviewer search and make adjustments to ensure that the right reviewers are engaged to review the right articles. Changes ranging from minor adjustments – such as revisions to keywords in databases – to substantial revisions to the peer-review process can result from the data collected and distributed by the RFP.

The core of the matter
In addition to the targeted surveys aimed at Elsevier authors and reviewers, Mulligan and his team are also involved in broader-scale, research projects. These broader studies seek to collect quantitative data regarding worldwide trends and general issues relating to publishing.

A recent study Elsevier commissioned investigated current trends and concerns in the industry – not just Elsevier publications. The results of this study were, at times, surprising. For example, it had been assumed that the technological advances of the last decade – the advent of the Internet and developments in electronic communications – would have a massive effect on the motivations and behaviors of researchers, and might even have an adverse effect on the publishing industry.

“What we came to discover,” Mulligan reveals, “is that although technology and electronic communication have improved the ways in which we transmit and process information, and certainly sped up the process, it has not radically changed authors’ motivations for publishing, nor has it altered their desire to do so.”

Ten years ago, authors published in scientific journals for four key reasons:
1) to facilitate dissemination of their research;
2) to have their research published in a reputable and respected journal;
3) to make their findings readily and publicly available; and
4) to ensure long-term archiving and a historical record.

The core trends study indicates that these are precisely the same reasons authors publish today. Despite radical changes in the way authors submit and process their manuscripts, their motivations remain remarkably similar.

Peer review study
Elsevier also participates in studies via the Publishing Research Consortium (PRC), an organization of society publishers, industry bodies and publishing houses, which supports global research into scholarly communication in order to enable evidence-based discussion. A recent study in which Elsevier participated concentrates on the peer-review process. “Peer review has been the focus of a number of discussions and debates recently. The PRC undertook a broad-scale, in-depth look at peer review,” Mulligan continues. “The study examines all aspects of the topic – the merits or otherwise of different types of peer review and the attitudes towards post-publication review. The study provides benchmarks for various aspects of the peer-review process and, importantly, will provide baseline data that will facilitate discussion.”

This international study will report back on the opinions and perspectives of reviewers. Through this data, Mulligan hopes to identify the key issues and trends that will help Elsevier develop more effective and useful tools for facilitating the peer-review process.

“Like all of our studies, this research will help further understanding. It will also help ensure that we provide the best support through our innovative and leading-edge solutions.”

To cite this article, please use: Toni Bellanca, "Feedback with impact", Elsevier Editors' Update, Issue 21, February 2008

Useful links:

  Publishing Research Consortium (PRC)

  "Is Peer Review in Crisis?"

"  Journal Futures: Researcher Behavior at Early Internet Maturity" - 2006 UKSG Annual Conference Presentation

"  Opportunities in Network Publishing" – STM Conference

  Sense About Science's Voice of Young Science Workshops
 

Please send responses to: EditorsUpdate@elsevier.com