The Editors’ Forum in the last Editors’ Update focused on how best to emphasize journal performance. Some 93 editors participated, giving their views on the following question:

Measuring journal performance is important for the editor and publisher, not only to review the journal’s position but also to allow them to take decisions for its future development. When discussing the performance of my journal I prefer to emphasize:

A Usage statistics - this reflects readership and trends which tell me whether we are on the right track. Usage figures are also current;

B Citations to articles in the journal - to me it is important to know how the journal and the articles are being used in the science community. This tells me that we publish the right articles and also helps me establish scientific market trends;

C Author feedback - It is important that the authors are satisfied with the journal. Highly satisfied authors are loyal and also recommend to others to publish in my journal.

More than 49% of respondents selected option B, although many of these also clearly acknowledged the potential pitfalls involved in citations statistics. Some comments for this choice included: “Option B gives editors (and authors) a good overview of how often, and in what context, the articles are used by other authors and journals,” and ”citations lead on to impact factor, which has become an important criterion for choice of a journal for submission of papers.”

In contrast, although still claiming article citation as the most important factor in measuring performance, many respondents conceded that this was due more to standard practice than to the accuracy of the measurements.

Citation engineering
Respondents in favor of option A, usuage statistics (15%) included editors of young journals (less than 3 years in publication), or those working in specific industries, who do not have access to relevant citation data as yet. Still others who selected ‘A’ argued that high usage/citation statistics are not necessarily an indication of quality. “Many times, indexes measure the applicability of the topic rather than the real quality of a journal,” according to one editor.

Some respondents claimed even stronger positions. “A paper that has errors and/or erroneous interpretations may attract more attention than an important result that is ahead of its time,” one respondent said. “In addition, the competition for citations has created a trend of ‘citation engineering’, whereby all kinds of tricks are used to attract citations. Editors often come across reviewers who insist on citing their [own] papers. However, it is difficult to propose an alternative measure.” Several authors mentioned the trend of ‘artificial citation numbers’ as a serious problem.

Author satisfaction key
A small number of respondents (3%) stated that author satisfaction was key. “If authors are loyal and active, and new authors are attracted on a regular basis, this is the best sign of journal health... The fact that people are happy to do reviewing and that referees take their task very seriously is important as well.”

Around 11% of editors who responded said all three indicators were important. This was summed up by an editor who wrote: “Usage statistics are the most transparent approach with regards to the journal. Citation results must also be taken into account, and are generally reliable, but do require some care. Author feedback is also useful, but is more a reflection on the editor rather than the journal, and can be strongly influenced by side factors.”