Introducing Snowball Metrics
Watch this short introduction to Snowball Metrics, recorded by Dr Malcolm Edwards, Director of Strategic Planning, Imperial College London.
Snowball metrics
Standardized research metrics – by the sector for the sector
Snowball Metrics was, crucially, a bottom-up initiative.
The outputs are owned by research-intensive universities around the globe, to ensure that the outputs are of practical use to them, and are not imposed by organizations with potentially distinct aims such as funders, agencies, or suppliers of research information.
The universities aimed to agree on methodologies that were robustly and clearly defined, so that the metrics they describe enable the confident comparison of ‘apples with apples’. The metrics which were defined are data source- and system-agnostic, meaning that they are not tied to any particular provider of data or tools. The resulting benchmarks between research-intensive universities provide reliable information to help understand research strengths, and thus to establish and monitor institutional strategies.
The aspiration is for these metrics to become global standards that enable institutional benchmarking, and cover the entire spectrum of research activities.
Snowball Metrics recipe book
The “recipes” for 32 Snowball Metrics are shared in the final, edition 3 of the Snowball Metrics Recipe Book. The feasibility of these definitions has been tested to ensure that Snowball Metrics can be readily updated to reflect the current status of an institution.
An example use of the Snowball Metric opens in new tab/window definition for measuring Success rate demonstrates the benefits of a consistent methodology being used. All recipes can be used by any organization, free-of-charge, for their own purposes and, if applicable, under their own business models.
Download Snowball Metrics recipe book opens in new tab/window
A Snowball Metric should be indicated by the use of the symbol placed after the name of the metric. Download the logo here opens in new tab/window.
In addition to the recipe book, the project partners also completed a mapping of HESA and HERD disciplines as well as a set of metric cards for all recipes from the third recipe book.
"We are taking a pragmatic approach with Snowball Metrics whereby we have identified a fit-for-purpose metrics framework, recognising that this is starting point and it will take time to develop. From there, we have begun to source data, taking that which is readily available and trying to ensure it conforms to standards..."
SR
Scott Rutherford
Director of research and enterprise at Queen’s university Belfast, United Kingdom
Why was this initiative important to the higher education sector?
A myriad of metrics is available, compounded by many similar versions of the same metric. It is difficult to know which metric will give the most useful insights, whether a metric is being calculated appropriately, or whether other institutions are looking at things in the same way. Snowball Metrics are best practices, representing a manageable set of metrics that aim eventually to inform all areas of research activity opens in new tab/window.
Agreed methodologies, which can be consistently applied to research management information, creates consistency and facilitates benchmarking between peer institutions. This helps to establish a reliable foundation for institutional strategic decision making to complement existing approaches. Snowball Metrics was a response to common frustrations voiced by universities opens in new tab/window:
Informed decisions depend on data, as well as expert opinion and peer review. Lack of an evidence-base prevents universities making the best decisions for themselves
University systems and the data that they collect are often determined in response to demands from funders and agencies, rather than what would help address their own questions
Universities are poor at collaborating with each other, and with funders and agencies
Commercial systems have not effectively addressed all the needs of a university, which has led to the proliferation of independent bespoke institutional systems and little best practice
The Snowball Metrics approach to address these problems was:
Agreeing a single method to calculate metrics that will provide input to institutional strategies and ensure the comparison of ‘apples with apples’.
Making sure Snowball Metrics are based on all the data sources available to a university, including institutional data sources, and do not depend on a particular data source or supplier
For universities to own the definition of these metrics, rather than them being imposed by funders and agencies
For institutions to collaborate with each other to agree a common solution, and to try to influence funders and agencies to adopt this as a common solution
For universities to work with a commercial supplier of research information who can learn about their needs first hand, and build systems and tools that enable institutions to effectively store their information and provide unambiguous, rigorously defined metrics based on consistent data
"Snowball Metrics is about working on and sharing a common language so that institutions are confident that they can use all of their data to compare their performance with each other in an apples-to-apples way. It’s not trying to tell anyone which of these metrics to use to answer any particular question: which sub-set of the metrics you use, whether 1-5, or 2, 9, and 13, is up to the universities and funders. It’s just like using a recipe book to cook your dinner: I don’t need to cook the entire book to find it useful; instead I choose what I like, and perhaps what I have the ingredients in the fridge for."
JJ
Jennifer Johnson
Head of performance, governance & operations, research & innovation at University of Leeds, United Kingdom