Times Higher Education approaches this by measuring universities' success in delivering the United Nations' Sustainable Development Goals (SDGs).
To do this, THE uses(opens in new tab/window) "carefully calibrated indicators to provide comprehensive and balanced comparisons across four broad areas: research, outreach, stewardship and teaching."
Those comparisons are reflected in 17 tables showing a university's progress toward delivering on each UN Sustainable Development Goal (SDG), alongside an overall Impact Ranking.
First launched in 2019 with 450 universities participating, the Impact Rankings have subsequently grown each year in number. In 2021, THE Impact Rankings included 1,118 universities representing 94 countries.
What is the Times Higher Education Impact Rankings methodology?
For each of the 17 UN Sustainable Development Goals, Times Higher Education creates different measurable objectives designed to capture a university's progress toward an individual SDG beyond teaching and research. Both quantitative and qualitative information informs your calculation, including both data provided by your university and 3rd-party data providers.
What data and metrics do Times Higher Education use for its Impact Rankings?
There are two categories that metrics fall under; research related metrics and metrics based on the university's own data and evidence supporting progress and contributions to the particular SDG outside of research-based metrics.
Research Metrics (27%)
A set of university research metrics related to the specific SDG. For example, the research metrics for SDG 5 include:
The proportion of female authors across all indexed publications (10%)
The proportion of papers on gender equality in the top 10 percent of journals as defined by Citescore (10%)
Number of publications on gender equality (7%)
Elsevier develops metric keywords, and the metrics are based on Scopus data, supplemented by additional publications identified by artificial intelligence. The dataset includes all indexed publications between a five-year period.
It is our goal to do everything we can to ensure that our unique strengths in content, data, and analytics help researchers and health professionals make the targets set by the United Nations in 2015 a reality.
Overall University SDG Specific Data & Metrics (73%)
73% of the calculation is based on university-specific data relating to the given SDG. Again, using SDG 5 as an example, in 2021, this included data and evidence relating to 5 areas:
The proportion of first-generation female students
Student access measures
The proportion of senior female academics
The proportion of women receiving degrees
Women's progress measures
To illustrate how different each methodology is, below are two visuals. The first image looks at SDG 5, Gender Equality and the second at SDG 15, Life on Land.
Proportion of female authors across all indexed publications (10%)
Proportion of papers on gender dquality in the top 10% of journals as defined by Citescore (10%)
Number of publications on gender equality (7%)
Proportion of papers in the top 10% of journals as defined by Citescore (10%)
Field-weighted citation index of papers produced by the university (10%)
Number of publications (7%)
In each SDG, you can see 27% of the calculation is based on Scopus data, and the remaining 73% consists of metrics specific to the SDG goal.
What is the Times Higher Education Overall Impact Rank?
THE also calculates an overall rank; this rank is based on an institution's top three SDGs plus SDG 17. Specifically:
SDG 17 accounts for 22% of the overall score
The other SDGs each carry a weight of 26%
To be eligible for the overall ranking, your university must supply data for SDG 17 plus data for three other SDGs. If you have provided data for more than three other SDGs, THE will use the three areas in which you have performed the strongest.
What is Scopus's role in Times Higher Education's Impact Rankings?
The Elsevier data science team designed the original SDG queries when Times Higher Education (THE) developed their initial University Impact Rankings. Since then, the team has continued reviewing and refining the queries, including looking at the publications returned by the search queries in Scopus.
The latest 2021 SDG queries result from Elsevier data science teams taking customer feedback into account and building extensive keyword queries, supplemented with a predictive machine learning element, to map documents to SDGs with very high precision. Elsevier 2021 SDG mapping captures approximately twice as many articles as the 2020 version while keeping precision above 80%. It also has a better overlap with SDG queries from other independent projects. Times Higher Education (THE) continues to use Elsevier's SDG mapping as part of their annual Impact Rankings.
Science-Metrix query benchmark
Performance of each SDG query is assessed in terms of precision and recall where:
Precision = Are the publications returned by the query relevant to the SDG?
~ 83% average across SDGs
= What percentage of relevant publications did we capture with the query?
* Recall is difficult to assess as there is no manually produced ‘gold standard’ of a good size to measure query performance against. The approach taken was to compare the publication set returned by each query with other publication sets expected to contain publications relevant to each SDG. For example, in SDG1 – No Poverty, one comparison set used was publications in the Journal of Poverty. For each SDG, around 50 different comparison sets, which we also assessed for quality, were used in order to provide a robust estimate of recall for each SDG query.
How does SDG Mapping in Scopus help my institution?
Each of the search queries are pre-generated queries in Scopus and provide a starting point for you to help understand the research related to a given societal impact research area.
For example, if you wanted to identify the most productive researchers in the area of SDG 7, Affordable and Clean Energy, here is one approach you could take. Run the pre-generated SDG 7 query in Scopus, limit your results to the past five years, and then use the analyze the results feature to see who is producing the most research related to this area.
You can also look at this information from different angles and parameters, like for a particular region, institution, funding sponsor, and more.
You can learn more about Scopus and SDG research here.
Powerful search, profiles, metrics and APIs and structured data to help you progress, evaluate and reflect your institution’s research activity
If you would like to learn more about Scopus or information for your institution, please get in touch with us.
What can SciVal tell me about SDG research & the Times Higher Education Impact Rankings?
Like Scopus, SciVal also includes the same SDG search strings in its pre-defined Research Areas. You can use the pre-defined Research Areas to go deeper into analysis and learn more about your SDG research. This could include:
Benchmarking your SDG research against other institutions
Understanding the top researchers in each SDG
Discovering the research topics that are most prominent for SDG researchers
You can follow examples of how this works in our "Analyze & understand societal impact research with SciVal."
In addition to SDG research, there are multiple ways to analyze research for your institution using SciVal. If you would like a deeper look or information specific to your institution:
Entities available to analyze
+20,000 Institutions from over 230 nations
~ 96,000 Topics
Research Areas*Publication Sets
Over 300 trillion metric values Data updated weekly
How can I rank in the Times Higher Education Impact Rankings?
According to THE(opens in new tab/window):
The Impact ranking is open to any higher education institution in the world. THE states that they want this ranking to be as inclusive as possible.
It is different from the THE World University Rankings, which includes a minimum publication volume as part of the eligibility criteria. There is no minimum here.
If an institution does not provide any data, it will not be ranked.
If you would like to take part in the impact rankings email
There is no participation fee.
A university can participate in as many or as few of the SDGs as they want. If they submit only one (or two), THE will evaluate them for a position within that SDG, but not the overall rank.
For eligibility for the overall rank, a university must submit data on at least three SDGs plus SDG 17. If you provide data on more than three SDGs, THE will take the top three best scores for consideration in the overall rank.
Data collection on the SDGs is challenging for an institution. While Elsevier, via Scopus data, provides the research component for each SDG, it is up to the institution to provide the rest.
In "Societal Impact, SDG Research & Universities: A conversation with Professor Aluísio Segurado of University of São Paulo(opens in new tab/window)" an episode of the Research 2030 podcast, Fernanda Gusmao of Elsevier asked Professor Aluisio Segurado, Head of Research at the University of São Paulo in Brazil:
Institutions need to provide evidence of the activities they developed that relate to SDGs. How did you gather such evidence? How many people were involved, and from which departments?