As the Head of Elsevier’s Usage Research Group, Hanneke Steuten is responsible for analyzing and reporting on the usage of Elsevier journals and articles through ScienceDirect and Scopus. She then reports her team’s findings back to customers – predominantly subscribing university libraries – on a monthly basis.

“We monitor the number of articles downloaded per journal - as well as the number of searches made, articles most read, etc. – and produce reports that our customers can retrieve via our website,” Steuten explains. “Each customer has access to up to 20 reports per month, which are created automatically, on request.”

Measuring impact
“In the past, before our journals were available online, we weren’t able to gather any usage information whatsoever,” Steuten continues. “Customers subscribed to certain journals, which were placed in their libraries, but we had no way of knowing how many people were reading them. Our main source of feedback on the audience and impact of a journal was provided via bibliometrics based on citation data, which is still very important. We rank the journals we publish based on the impact factors provided by Thomson ISI on an annual basis.”

A high impact factor for a journal or a high number of citations for an article provides an indication of its importance within the academic community, thereby adding prestige to the article. “Citing an article is a very formal way of acknowledging the significance of that article and citation data represent ‘author behavior’. At the same time, usage data are also an acknowledgement of the value of an article or journal, but in a much broader and less formal sense, representing ‘reader behavior’.”

In certain scientific fields (physics, for example), there is a large amount of overlap between the author community and the reader community of a journal. In other fields, the majority of readers do not publish articles and the reader community is much larger than the author community. “Lots of people outside academic circles read medical journals, for example, because they work in the field or they have an interest in a certain disease or new form of treatment,” Steuten explains. “Physics journals, on the other hand, are very specialized and do not appeal to such a wide audience. They attract a higher percentage of citations per reader than medical journals, due to the academic nature of their readership.”

Vast improvement
“The World Wide Web has changed the way journals are read. Since the Internet is now the main way to access journals and articles, this provides us with a much more accurate overview of how they are being used. Usage can differ from journal to journal – some have a large number of personal subscribers, for example – but most subscribers are libraries and a lot of the time, articles are accessed through Web searches. Consequently, we now have a wealth of information on the amount and frequency that articles are read, which universities use which journals most, and the number of readers of journals and articles."

Statistical benefits
As Steuten explains, there are many advantages to be gained from the ability to gather accurate and current usage information. “As far as our customers (university libraries and R&D departments) are concerned, they can use the information to make informed decisions, such as whether to renew their subscription to a particular journal. Other factors are involved, of course, such as the faculty’s opinion of the journal and its impact factor, but they can now make decisions with the additional knowledge of how often the journal is referenced.”

“Internally, publishers can see which universities use which journals the most, which articles are downloaded more often and which have the highest number of readers. This information can be used to assess whether special theme issues are more or less popular than standard issues, for example, thus influencing the editorial policy of the journal.”

“It’s also important for society journals to know how often they are used. In general, they aim for as broad an audience as possible and we can offer them information not only on the number of readers, but also on their geographical spread. In recent years, for example, China has become a large and growing user of our journals and articles. Around 12% of our worldwide download usage now stems from China, compared to 24% in the United States and 26% in the whole of Europe.”

“Finally, accurate usage statistics are also useful for our own sales organization. We can show customers exactly how journals and articles are being used as an incentive to renew contracts.”

Hot topics
Nothing stands still for long where the Internet is concerned, and the gathering of usage statistics is no exception. “Researchers at the Los Alamos National Laboratory, in the United States, have set up a group called the MESUR Project,” Steuten explains. “They are carrying out research into whether impact factors can also be based on usage information, rather than citation data alone. They are using Elsevier log file information from ScienceDirect and Scopus to create formulas to measure the comparative impact of journals and articles within the same field. These alternative impact measurements would be very useful to librarians, who currently rely on citation impact data, mainly.”

Another organization, called Counter, is also researching the possibility of creating impact measurements on the basis of usage data. “Counter is a co-operation between publishers and librarians that has introduced standards for the usage information provided by publishers,” Steuten continues. “In the past, different publishers used varying criteria when compiling usage statistics, making it difficult to compare journals and articles from different sources. Counter has developed a standard, with which Elsevier and the majority of publishers now comply.”

Work in progress
“Since all of our journals and articles have been available via the Web, through resources such as ScienceDirect and Scopus, the use of journals, in general, has increased considerably,” Steuten concludes. “This is due to various factors. In the past, for example, readers needed to visit a library to find a printed copy of a journal, whereas now they can access virtually any journal or article from their desktop. In addition, the increasing efficiency of Web search technology, the fact that our journals are covered by widely accessible abstract databases, like Pubmed, and the fact that they are now also indexed by Google has made it quicker and easier to locate specific articles on the Web.”

“The whole process has become much more efficient and accessible. As a consequence, usage rates have been growing at a remarkable rate since 2000, when we first started tracking usage statistics through our databases. In the early years, usage doubled year on year, while in 2007, for example, download usage via our websites still increased by 24% in a single year and I wouldn’t be surprised if 2008 also produced double-digit growth.”

To cite this article, please use: Gary Rudland, "Measuring usage", Elsevier Editors' Update, Issue 21, February 2008

Useful Links:

  Usage reports website

  What Counts and What Doesn’t: An Insider’s Guide to Usage Reports

  MESUR Project

  Counter

Please send responses to: EditorsUpdate@elsevier.com