Snowball Metrics help universities assess and compare their research
Eight universities and Elsevier develop a methodology institutions can use to make decisions on research funding and investment
By Linda Willems and Lisa Colledge, DPhil Posted on 29 January 2013
Eight leading research universities in the UK have agreed on a set of metrics that will enable institutions to better understand their research performance and compare it to that of other institutions. These experts then worked with Elsevier to test the metrics to make sure they were scalable to whole universities. Now, by using these metrics, all institutions can use a single approach, which is essential to making meaningful comparisons.
The project partners have published these methodologies in the Snowball Metrics Recipe Book, available to download free of charge. These methodologies can be used by anyone in higher education for their own purposes.
While there was already a wide variety of metrics available to those in higher education, methodologies varied and lack of consistency was a problem.
Snowball Metrics are designed to help institutions generate insights from data in a standard, commonly understood way, enabling accurate and useful benchmarking. The hope is that institutions can then use this understanding of their research strengths and weaknesses to reinforce their existing strategic decision making processes.
Dr. Guy Orpen, Professor of Chemistry and Pro Vice-Chancellor of Research at the University of Bristol, said: “There is a strong need for an agreed set of metrics across universities, that don’t rely on a particular supplier to calculate them, so that we can understand our strengths in relation to our peers.”
The Snowball Metrics initiative was launched in response to the 2010 Research Information Management survey, which identified a need for a defined and agreed-upon national framework for data and metric standards in the UK as well as increased collaboration among stakeholders. The survey, “Developing tools to inform the management of research and translating existing good practice,” was funded by JISC and jointly conducted by Imperial College London and Elsevier.
Dr. John Green, a Life Fellow of Queens’ College Cambridge, has devoted much of his career to developing research as a business within universities, and has taken on the role of Chairman of the Snowball Metrics Steering Committee. He pointed out that the idea was to work together to create a standard for the higher education sector. [pullquote align="left"]These metrics allow us to make informed decisions about the way we manage our research, and universities can now make ‘apples to apples’ comparisons among themselves.[/pullquote]
“This is not top-down or imposed by government and funders,” he said.
“These metrics allow us to make informed decisions about the way we manage our research, and universities can now make ‘apples to apples’ comparisons among themselves,” he explained. “We can understand institutional strengths and weaknesses relative to our peers, and that enables us to inform our management on where resources should become more effective and efficient.
“Our aspiration is that Snowball Metrics become global standards, and there is already a keen interest from other institutions,” he added.
In fact, the name of this project was inspired by the idea of a snowball starting small and gaining in size as it rolls through the snow — as in thesnowball effect. The plan was to start with a small group of universities to make it easier to come to agreement, and only then bring in others.
Professor Eric Thomas, Vice Chancellor at the University of Bristol, a member of the project partners, stated in the recipe book: “It’s unique for universities to join forces and take time to consider how we would like to look at our performance – and by extension how we would like our performance to be viewed. We are pleased that all the project partners have committed to openly sharing their metric definitions; we hope that the free availability of the ‘recipes’ will encourage enthusiastic support and adoption by universities across the UK, in Europe and further afield.”
More than 100 people participated in an informational webinar held by the project team in November. Although the metrics were developed in the UK, more than half the attendees were from other countries, including the US, The Netherlands, Spain, Italy and Russia. In a post-webinar survey, registrants were asked how relevant Snowball Metrics was to them and whether they would be interested in adopting them; 100 percent of respondents answered both questions positively.
What can Snowball Metrics be used for?
These standard methodologies, or “recipes,” can be used for three key areas of research activity – input, process and output. The first set of metrics is listed below, and additional metrics will continue to be shared with the sector:[note color="#f1f9fc" position="right" width=338 margin=10 align="alignright"]
- University of Oxford
- University College London
- University of Cambridge
- Imperial College London
- University of Bristol
- University of Leeds
- Queen’s University Belfast
- University of St. Andrews
The eight universities collectively account for nearly 40 percent of the UK’s competitive research funding, UK-authored articles and UK citations. They have contributed expertise and staff time to the initiative free of charge.[/note]
- Applications volume
- Awards volume
- Income volume
- Market share
- Scholarly Output
- Citation Count
- Field-Weighted Citation Impact
- Outputs in Top Percentiles
The project team plans to work with more universities while seeking feedback about how the metrics can be adapted for use in other countries and settings.
To enable as many software providers as possible to integrate the metrics into their tools, the Snowball Metrics team has partnered with euroCRIS (the European Organisation for International Research Information – a nonprofit organization for the development of research information systems and their interoperability. The result of this collaboration will be the expression of the Snowball Metrics in CERIF, a freely available global standard data format that enables different systems to communicate with each other in this common language.
For more information, visit the Snowball Metrics website.
The project team is now seeking input from universities in the UK and elsewhere on the broader applicability of the metrics. The feedback will guide the development of additional national “flavors” of existing recipes, as well as entirely new Snowball Metrics, to further support institutions in their needs to make national and global comparisons. Use the Contact form on snowballmetrics.com or email Dr. Lisa Colledge, Snowball Metrics Director, at firstname.lastname@example.org. [divider]
Dr. Lisa Colledge is Senior Manager of Strategic Alliances in Elsevier’s Global Academic Relations team and Elsevier’s main representative in the Snowball Metrics project partnership. She has also held roles in Elsevier as Publishing Editor, Data Analyst and Senior Product Manager in the SciVal team. She has a D.Phil in immunology and an M.A. in biochemistry, both from the University of Oxford.
As Academic Content & Communications Manager for Elsevier, Linda Willems oversees the Editors’ Update website, a resource center designed to keep editors in touch with the latest developments in journal publishing, policies and initiatives. The site also hosts the quarterly Editors’ Update newsletter, for which she is Editor-in-Chief. Linda, who is based in Amsterdam, is also on the team behind the Elsevier Journal Editors’ Conferences program.
By Inn Beng Lee | Posted on 16 Jun 2014
Governments urged not to neglect emerging universities and undergraduate educationBy Ryan Bailey | Posted on 27 Mar 2014
March 31 event, organized by the University of Maryland, Baltimore and Elsevier, will focus on the measurement and impact of researchBy Iris Kisjes; Lisa Colledge, DPhil; and Ikuko Oba | Posted on 08 Jan 2014
Learn how early adopters are using SciVal, and gain complimentary access for a weekBy Iris Kisjes | Posted on 10 Dec 2013
The UK 'punches above its weight,' surpassing US in research qualityBy Marilynn Larkin | Posted on 28 Oct 2013
It won’t replace NIH grants yet, but crowdfunding can help offset scientific funding cuts, recipients say