Benefiting from the Author Feedback Program

Twelve weeks after an article is published in one of 1,159 selected Elsevier journals, the author receives a questionnaire that asks them to rate their satisfaction with the way their article was managed through the publishing process, from submission to final publication. In this way, with the publishing experience freshly in mind, the author can seize the opportunity to react to the process and the quality of publishing via Elsevier’s Author Feedback Program (AFP).

The AFP is a structured means of obtaining information about the author experience in scientific, technical, and medical (STM) publishing. Adrian Mulligan, Associate Director of Academic Relations at Elsevier, explains, “Live since 1999, the information gathered from the AFP has added significant depth to the understanding of journal trends from the author perspective. This allows the publisher and editor together to identify areas of strength and weakness, and to concentrate resources where they are needed.” That said, the AFP is an author-facing initiative. “With new editors coming on board and new journals launched or acquired, a few editors may not know about it.”, says Mulligan.  

Primary research journals are included in the AFP but not automatically. The journal editors are consulted prior to a journal being included. “We know that the program is well-received by editors since the journals in the AFP represent approximately 82% of those eligible for it,” explains Mulligan. “It is a massive endeavor, set up to fulfill the needs of the community we serve by providing close monitoring of and tracking of author satisfaction and publishing performance. This helps us to ensure that we continue to meet the needs of the research community.” With, on average, 1,000 reports produced every six months, the data is significant. Journal-level reports are produced detailing both the performance of individual journals and the performance of competitors in the research area. Publisher-level reports are produced to evaluate Elsevier’s performance as a whole.

On average 15,000 AFP questionnaires go out each month. The data is collected anonymously and the response rate hovers at around one third (33%). This is high for a survey of this type and not surprising given that the target group is one for whom publishing is an important part of professional activity and for whom the journal’s field is already an area of engagement. In all, more than one million authors have been contacted since the program’s launch. Reports are sent, in PDF, to editors either on request or if a need is identified.

Central information
Fundamentally, it is the author’s perceptions and experience that are polled. “Information generated by the questionnaire is spread between the quality of journal editing that the author perceives, the treatment of the submission that the author perceives, and the journal performance that the author perceives,” says Mulligan. This provides a basis for journal improvement.

Respondents rate 29 different service points grouped into eight factors or attributes. From this, the AFP has found that authors most value the following:

• First—The speed of the refereeing process.
• Second—The quality of the peer review process.
• Third—The reputation of the journal.

In fact, every year since going live, the results have consistently been these, and almost every time they have occurred in this order of importance. In 2001 there was a slight variation when the quality of the peer review process took the first position, and the speed of the refereeing process took the second position jointly with the reputation of the journal.

Considering that referees, in the course of their review process, decide whether an article is accepted or rejected – that is to say, whether the author must move on and consume valuable time submitting the article elsewhere – it is no wonder that the speed with which this is completed is so important. Adds Mulligan, “A rapid turnaround relieves the worst part of the process: the waiting.

“The quality of the peer review, on the other hand, is down to the content of the report generated for the article, which is to say the usefulness and quality of the comments returned to the author,” continues Mulligan. “If the author believes that a rigorous and thorough review took place, and if the comments and corrections are judged beneficial by the author, the journal will be viewed as more successful and of greater value to the author and, by extension, to the community.”

The reputation of the journal is linked to a number of aspects, and is a combination of being well known by researchers in the area, its impact factor, and the quality of the editor and the editorial board. For a journal to be perceived highly based on this attribute, typically the impact factor for the journal would be good, the journal must be well known and esteemed within the field, the editor and editorial board must be perceived as respected, and the editor must be perceived to be managing the journal well.

Peripheral information
Other valuable information is also collected. Mulligan explains, “AFP reports supply us with insights into the author community that we would never normally get. From demographics to other journals to questions of funding, the AFP has allowed editors to get a sense of certain trends in their field more easily.” In addition, information is gathered about other journals in the author’s field. “To make an effective comparison, questions are asked about the author’s experience publishing with journals other than the current one and the data is used to improve Elsevier’s journal performance.” Some technology-influenced questions have been added to the AFP questionnaire, too. Internet usage and other peripheral information is gathered this way but the core information remains the same for data to accumulate and allow for successful benchmarking.

AFP reviewed
Tests have been carried out comparing AFP data to empirical data. “In a test case, for example, non-responding authors were tested against responding authors,” explains Mulligan. “The non-responding authors were contacted in an effort to make sure the data returned reflected their perceptions, too. There was found to be no significant difference at 95% confidence levels.” Author experience and author perception patterns as reported in the AFP are consistent and reliable.

Practical application
AFP reports can pinpoint a certain area to help a journal editor find out why author perception of it is low. “In the case of one journal, Polymer, production speed was perceived to improve over the four-year period from 1999 to 2004 to a much more acceptable level,” explains Mulligan. “Polymer’s editorial team knew, from the AFP reports, where to focus.” The team began investigating why the journal was perceived as being slow for production speed in 1999. They found three key problems:

• A backlog of articles waiting for action.
• Sixty percent of articles submitted by editors were incomplete.
• Editors had sent outdated copyright forms to authors.

Immediately, a page-budget increase for 2000 was introduced. Issues around submission and copyright were introduced. Finally, a Service Level Agreement was introduced stipulating that production times not exceed 12-14 weeks. From a high of 32 weeks in 1999, production was brought down to a low of 8.5 weeks in 2004. Author’s perception of production speed on Polymer also improved during this time. From being in the bottom third of journals in the field in 1999, to being in the top third in 2002, and has settled in that position since. Moreover, as a result of the journal becoming more attractive for its faster publication time, submissions to Polymer have increased and the rejection rate has doubled from 30% to 60%. The improvement of production speed has not been limited just to Polymer as a result of investments in new technologies: “Author perception of improvement in production speed for Elsevier globally, has gone up by 6.6% since June 2002,” says Mulligan.

As it continues, the AFP provides ongoing feedback to publishing staff and editors. When necessary, it identifies where a journal is underperforming. At the same time it facilitates the development of a solution to the problem by showing the publishing staff and the editor where to focus and allocate resources. Finally, ongoing AFP data continues to measure author perceptions, simultaneously measuring the effectiveness of the solution.


****

Please send responses to this article to EditorsUpdate@elsevier.com