5 ways usage metrics can help you see the bigger picture
Download a free guide, sign up for a webinar – and tell us how you use usage metrics
By Lisa Colledge, DPhil, and Chris James Posted on 23 March 2015
Every day we each make thousands of decisions. These range from life decisions, such as whether to get married, make a major purchase or have children, to everyday decisions like what to have for lunch. Most decisions are made by analyzing multiple types of information to come to a conclusion. In fact, researchers at Cornell found that people make an average of 226.7 decisions a day about food alone.
Webinar on Usage Metrics
A webinar will introduce the Usage Guidebook. The 40-minute session covers the five reasons metrics-based on usage data should be part of your metrics portfolio, with time for questions and comments at the end.
Date: Wednesday, March 25, 2015
Time: 16:00 – 16:40 CET | 15:00 GMT | 11 am EDT | 8 am PST
Registration: The webinar is free. Register and join here.
Submitting questions: Questions for the panel can be submitted through the webinar interface or through Twitter using the hashtag #usageguidebook.
Elsevier’s Research Intelligence group has created a Usage Guidebook with practical advice on usage data and the appropriate use of multiple metrics when analyzing research. You can download it here for free.
This is also true in research and the analysis of research. Whether we are looking at the performance of a paper, person, group, institution or yourself, the total impact is always multi-dimensional. Elsevier has recently made tools available that enable the large-scale use of usage-based metrics, providing another data point that can be used to better analyze and more completely understand the impact of research.
The availability of online usage data is a relatively recent phenomenon, and research metrics derived from usage data are not yet commonplace, so we have created a practical guidebook to help you apply these metrics to your analyses. The Usage Guidebook complements the SciVal Metrics Guidebook by providing practical advice on usage data and the appropriate use of multiple metrics when analyzing research.
Here are five ways metrics based on usage data can be useful to you:
1. The best decisions recognize that researcher excellence comes in many shapes and forms.
Researchers may be seen as excellent because they are productive and well cited. But what if they are not well cited but their output is very well used? These researchers are also excellent but in a different way. There are even more ways researchers can be excellent and quantitative input into decisions will be strongest when metrics looking at different strengths, based on different types of data, are combined. Usage data and metrics are an important piece of this jigsaw.
2. Research is complex and is best quantified using multiple criteria, including usage data.
Different data sources reflect different types of behavior with different motivations. Unfortunately, usage data is sometimes only considered in terms of how well it can predict citations; citations are important, but they should not be seen as the leading research outcome against which everything else should be compared. Usage is important in its own right.
3. Viewing activity is one of the earliest indicators of interest in research.
It can occur very quickly – as soon as research is available online – even though it can continue to accumulate for years after a publication becomes available. It is one of the data types that is very well suited to spotting trends, such as an emerging new topic of research or a talented early-career researcher.
4. Usage data reflects engagement of the entire research and student community, making their interests visible even if they are not publishing.
Non-publishing – and hence non-citing or cited – users are estimated to constitute one-third of the research community (Beyond Bibliometrics, Chapter 17). The interests of this third, which includes large numbers of undergraduate and graduate students as well as researchers operating in the corporate sector, are effectively hidden from citation data. Usage data reveals what this third is doing.
5. Usage data provides metrics to evaluate the papers of researchers who publish with the expectation that their research will be used but not necessarily cited.
Elsevier research has shown that a typical researcher spends 23 percent of their work week searching, reading, reviewing and using the published literature. Typical behavior in Arts and Humanities, for instance, means that there is low potential to be cited although there may be very high interest in these outputs, which can be observed through viewing activity. And clinical research is aimed at practitioners working with patients, who tend not to publish themselves but read voraciously.
Here are two examples of how metrics based on usage data can be accessed and put to use.
Use case 1: How are my publications doing?
My Research Dashboard allows researchers to immediately see the impact of their research in terms of what is being downloaded, shared, and cited, as well as the country and discipline of viewers. Based on usage data from ScienceDirect, this multi-faceted view on their research output gives researchers a comprehensive overview of how their work is being used, and multiple options to be able to demonstrate the impact of their work on the community when they are applying for funding, or tenure.
A researcher can see, for example:
- That a very recent publication is being picked up and viewed, even though it has not been cited yet
- That an erratum, although not cited and perhaps not expected to be cited, is being actively looked at:
- That an older publication that has received citations continues to be actively viewed. Combining these 2 ways of looking at how this publication is being received gives a more complete picture than would be the case with either one alone:
- Where in the world their publication is being viewed. In this example, the publication is of great interest in the UK, Australia, the Netherlands, China and the United States, and the researcher could use this information in demonstrating international reach of their work, perhaps even beyond their collaboration network, even in the absence of citations:
Use case 2: Exploring a research topic
SciVal, which has online tools to analyze many aspects of research worldwide, displays information about the total views any set of publications in a research area or theme have received. This can be sliced by country and sector. For example, the research area “sitting” is sometimes considered to be the modern day “smoking.” Researchers have found that prolonged sitting increases the risk of developing illnesses like various types of cancer, heart disease and Type 2 diabetes and that the effects of long-term sitting are not reversible through exercise or other good habits. The keyphrase figure taken from SciVal shows that in the research area of “sitting,” terms such as posture, motor activity and exercise are increasingly common in publications in this research area.
A global view of institutional activity in “sitting” shows that the most productive institutions, as measured by Scholarly Output, are sometimes also the most viewed institutions, as shown by Views Count. But this is not always the case.
If we look a bit more closely, using the tables below, we can see examples of different ways of institutions being excellent in the area of sitting:
- Productive, well cited as measured by Citation Count, and also highly viewed – University of Queensland
- Productive, but neither highly cited nor viewed compared to other universities active in this area – Universidade de Sao Paulo
- Both highly cited and viewed, even though not producing as many publications as others – University of Sydney
- Highly cited but not highly viewed – Harvard University
- Highly viewed, but neither highly cited nor very productive – Deakin University
5 most productive institutions in “sitting”
5 most cited institutions in “sitting”
5 most viewed institutions in “sitting”
Different users may define excellence differently depending on their needs, opinions, national situation and experience – there is not a single “correct” way of being excellent, only a way that is correct for the specific situation, which is why it is so important to have multiple options available.
We want to hear from you!
The availability of online usage data is a relatively recent phenomenon, and research metrics derived from usage data are not yet commonplace. In the Usage Guidebook, we have provided a lot of information about usage data and metrics to answer some of your questions, but there is a lot still to be learned. We hope that making this kind of research intelligence available to you will stimulate questions and answers, and we are looking forward to working with you to learn about the insights you can gain from this resource.
You can give feedback in the comment section below, or join the webinar, where organizers will be asking for participants’ questions and comments.
Elsevier Connect Contributors
She started by working with editors and learned societies to develop strategies to improve journals’ standings. She then joined the Elsevier Research Intelligence product group, which most recently launched SciVal Trends. Lisa also represents Elsevier in Snowball Metrics, in which universities agree on methods to generate metrics, from all data sources available, to support international benchmarking.
Prior to joining Elsevier, she conducted postdoctoral research at the University of Edinburgh. She holds both a DPhil and an MA from the University of Oxford.
Chris James is Marketing Manager for Elsevier Research Intelligence, responsible for marketing and communication initiatives of SciVal and Analytical Services. He joined Elsevier in Amsterdam in 2004 and worked for four years as a Customer Marketing Manager, training his northern Europe customers on products such as ScienceDirect, Scopus and SciVal. Prior to joining Elsevier, he worked at an engineering consultancy in the UK.