“Over the past five years, there has been an incredible increase in the number of e-journals on people’s desktops. The main aim of our study, which is only at the half-way stage, is to ask what this is achieving,” says Dr. Ian Rowlands, Reader in Scholarly Communication at University College London’s Department of Information Studies.

Rowlands, and the other members of the Centre for Information Behaviour and the Evaluation of Research (CIBER) study group, recently published a preliminary report entitled Electronic journals: modeling journal spend, use and research outcomes.

Gathering data on publishing innovation
The preliminary report is based on data collected from 112 UK universities between September 2006 and August 2007. “It was basically a quantitative exercise,” Rowlands explains. “We had no preconceptions, but simply gathered data and looked at whether any patterns emerged. This is often how we work: first gathering and examining data, looking for trends, then asking informed questions and performing further qualitative analysis to arrive at conclusions.”

“When we initiated this study, we simply didn’t know what questions to ask, because the whole field of electronic journals (e-journals) has been changing so quickly, both in terms of technological advances and growth. Digital libraries have only existed for 15 years, at the most, and represent the first major change in the library environment since the invention of the printing press. In historical terms, we’re only just getting used to this new environment.”

Various public agencies collect statistics on library investment in the UK, such as how much each university spends on resources like e-journal subscriptions and, more recently, full-text downloads, and how these resources are used. “The rise of electronic publishing and electronic access to resources means that it is now possible to collect vast amounts of information about how universities, departments and individuals use these resources,” Rowlands says. “We were also able to collect data on PhD awards, research grants and papers published at various UK universities and in different subjects.”

Initial findings
Allthough Rowlands and his team are only halfway through the study, some interesting – and strong – correlations have already been identified.

Spend/usage
“One of the first things we noticed was a strong correlation between spending on e-journals and their usage. This correlation is not as obvious as it might first appear,” he says. “Worldwide, approximately US $8 billion is spent annually on scholarly journals. The strong correlation between expenditure and usage demonstrated in our study seems to suggest that e-journals are not simply being subscribed to for the sake of completeness of the library. This may lend strength to librarians in their efforts to negotiate increases in their budgets each year.”

Usage/research outcomes
Another striking pattern suggested by the preliminary study concerns the relationship between the number of downloads from e-journals and research outcomes at various UK universities. “It seems that the universities with higher download figures also award more PhDs and publish more papers, even when we take the size of the institution into account,” Rowlands reveals. “Although whether this is merely coincidental remains to be seen.”

Usage/research grants
Initial analysis of the data revealed a significant pattern in the relationship between the number of downloads and the value of research grants awarded to different universities. “Again, this apparent correlation, which is also scale-independent, needs further investigation, but it is certainly intriguing,” Rowlands comments.

Search times
Having been granted access to ScienceDirect and Oxford Journals logs at UK universities, CIBER performed deep-log analyses to ascertain how people were accessing information, from where, for how long, etc. “What struck us most was that people tend to initiate searches outside ScienceDirect, via gateways like Google (Scholar) or PubMed,” says Rowlands.

“We also discovered a statistical relationship between session length and (Hirsch index) research rating. It seems that the higher-rated researchers spend less time obtaining information. They tend to use third-party services and are much more targeted than some of their colleagues. In addition, researchers of the same subject at different universities displayed similar behavior patterns during research sessions, but noticeable differences in behavior patterns were apparent between researchers of different subjects.”

The next step
The second stage of the study began in April, and is due to be completed early in 2010. “During the second stage, we will take a much closer look at these apparent relationships and patterns, and we will also extend the time frame of the study to cover the previous four or five years,” Rowlands says.

“We already know that there is a statistical correlation between usage, expenditure and research outcomes, but we don’t know whether this is simply a coincidence. In addition, if we do prove that these relationships are real, we will also try to answer the question ‘In what direction are they headed?’ For example, do successful researchers create demand for library services, or does library investment lead to success?”

The broader picture
“So far, we have only performed the preliminary study and we need to be careful not to draw conclusions in advance of the second stage,” Rowlands cautions. “In general, however, I feel that publishers could make a stronger case for the added value they provide, in performing peer reviews, for example. They should be more transparent about the processes and costs involved in publishing scientific journals, where demand is more finite and less elastic than in other forms of publishing. In fact, I’d say that the entire scientific community (scholars, publishers and librarians) could improve its promotion of the added value it provides.”

“Our preliminary study provides good ammunition for this, and will be of particular interest to scholarly publishers and librarians, since it offers fresh insights into the behavior of their users, as well as the relevance and usability of journals and services. During the period covered by the preliminary report, 102 million articles were downloaded at the UK universities surveyed, which proves that the material is being used. The second stage of the study should provide more compelling evidence upon which to base more far-reaching conclusions. Watch this space.”

To cite this article, please use: Gary Rudland, “The true value of e-journals”, Elsevier Editors’ Update, Issue 26, May 2009

Useful Links

  Ian Rowlands’ Biography

  CIBER News

  UCL Centre for Publishing