1. What are University Rankings?
University rankings are diverse, imperfect — and influential.
Your university has a unique mission and works hard to attract the best students, talent, and funding to support that mission. On the flip side, potential students, researchers, faculty members, (and, in some countries, even policymakers and funding agencies) look for help to in deciding why your university best fits their interests. Enter the world of University Rankings.
- What are University Rankings?
- Why do rankings matter?
- Who publishes University Rankings?
- How are rankings calculated?
- Can I influence university rankings?
Rankings are not perfect, but useful
Although not perfect, and certainly not the only input needed, university rankings and league tables provide a way to compare higher education institutions based on a set of similar criteria and are readily available.
“While university rankings are one of the essential ways of measuring the quality of higher education, quality measurement in higher education is a multi-dimensional problem that cannot be based solely on rankings.” (Gokcen Arkali Olcay, Melih Bulu, Is measuring the knowledge creation of universities possible?: A review of university rankings, Technological Forecasting and Social Change, Volume 123, 2017, Pages 153-160, https://doi.org/10.1016/j.techfore.2016.03.029)
Rankings are often met with criticism, especially when conflated with reputation. It is important to keep in mind that league tables do have their limitations and only provide one indication of university performance. Rankings, therefore, should be used as supportive tools, not as a stand-alone measurement.
Rankings are many, and not one size fits all
Just as each university is unique in its mission and purpose, each ranking and league table has its niche and focus. There are over 20 global university ranking reports or organizations alone, and each has its own methodology, data sources and set of indicators.
The table below provides a quick glance into the proliferation of ranking organizations over time, including various league tables added along the way to address the different types of universities and their missions.
International rankings organizations
According to the IREG Observatory on Academic Ranking and Excellence, rankings organizations need to
“Recognize the diversity of institutions and take the different missions and goals of institutions into account. Quality measures for research-oriented institutions, for example, are quite different from those that are appropriate for institutions that provide broad access to underserved communities.”
Rankings are influential
Whether from the perspective of a student, parent, researcher, potential faculty, funder, or another entity, rankings help build one perspective on a university's success or fit, objectively.
2. Why do rankings matter?
In the past 18 years, international university rankings have grown in visibility and prominence and, in some countries, can influence:
- How governments measure research excellence for your institution
- The criteria used by undergraduate students, parents and graduate students when choosing a university
- Why a company selects you as a partner
- A funding body’s decision to invest in research at your university
Although rankings are not the sole indicator of an institution's reputation and academic excellence, they do provide a quantitative and popular way to benchmark universities nationally, regionally and globally.
Why rankings can play a role in attracting international students
“In the global marketplace, rankings bring visibility. This is essential because of the growing percentage of undergraduates and postgraduates who have a high interest in the rankings. High-achieving and wealthier students are most likely to make choices based on them. Likewise, international students continue to rate reputation and ranking positions as key determinants in their choice of institution, programme and country, more than, say, institutional websites.”
The international student has economic significance, to both the university and its country or region. In the United States alone, the Association of International Educators (NAFSA) recently reported that
international students studying at U.S. colleges and universities contributed $41 billion and supported 458,290 jobs to the U.S. economy during the 2018-2019 academic year.” Source accessed on July 28, 2020: https://www.nafsa.org
Source: BPC Calculations from the International Institute of Education and The National Center for Education Statistics, US Department of Education
Looking at the Universities UK’s website, they report that
“As of 2018 the U.K. remained the second most popular destination for international students to study higher education globally with only the USA ahead of it. This position is however increasingly coming under threat from Australia which has experienced strong growth in recent years. Between 2011/12 and 2017/18 the U.K. saw a 5% growth in international student numbers compared to 73% in Australia. Despite recent decreases since 2016 the USA also experienced 31% growth over the same time period.” Source accessed August 6, 2020
How does the number of international students studying in the U.K. compare to other countries?
In November 2019, the Australian Minister of Education, The Hon Dan Tehan MP, shared that
“International education has experienced its fifth year of consecutive double digit growth, highlighting the strength of Australia’s higher education system” and that the “International education contributed $37.6 billion to the Australian economy last financial year, which was a $5 billion increase.” Source accessed August 6, 2020.
Rankings increase your university’s visibility
For the international student, rankings and league tables often weigh into their decision-making process when considering return on investment; therefore, being ranked, even regardless of position, can help you to be considered.
3. Who publishes University Rankings?
Rankings are developed and published by a range of entities, including magazines, newspapers, websites, academics, and governments. Some ranking organizations specialize in global rankings, others in national or regional, and a few do both. This page primarily discusses ranking organizations producing world university rankings and who focus on research output.
A brief history of global rankings
The rise of global rankings (as shared in our earlier chart) is frequently marked by Shanghai Jiao Tong’s introduction of the “Academic Ranking of World Universities (ARWU)” in 2003, and attributed to an increasingly globalized economy and competitiveness. The growing attention toward higher education and university-based research, led to a need for a comparative framework. Source: Hazelkorn, E., Gibson, A. Global science, national research, and the question of university rankings. Palgrave Commun 3, 21 (2017). https://doi.org
From 2003 on, multiple ranking organizations and reports have entered the arena, expanding options into how Higher Education Institutions can be compared and with greater variety in niche reports.
Two examples of this are:
- QS World Rankings' ‘University Rankings by Subject,’ created in 2011, which accounts for the varying research cultures and publication rates across academic disciplines
- Times Higher Education’s (THE) 2019 launch of the THE Impact Rankings, “a new global university ranking that aims to measure institutions’ success in delivering the United Nations’ Sustainable Development Goals”
From The Economist:
“The rankings race is thus marked by a happy irony. Driven in part by nationalistic urges, it has fostered the growth of a community that knows no borders. Critics are right that governments and universities obsess too much about rankings. Yet the world benefits from the growth of this productive, international body of scholars." Source accessed July 28, 2020: https://www.economist.com
What are some of the key global ranking organizations and how do they differ?
Quacquarelli Symonds (QS), Times Higher Education (THE) and Shanghai Ranking (the Academic Ranking of World Universities; ARWU) are considered among most established and prominent global ranking bodies.
Seven key global ranking reports to know are (in alphabetical order):
- CWTS Leiden Rankings
- Shanghai Rankings (ARWU)
- Times Higher Education World University Rankings
- Times Higher Education Impact Rankings
- QS World University Rankings
- QS World University Rankings by Subject
- US News & World Report: Best University Rankings
4. How are rankings calculated?
Before diving into how rankings are calculated, keep in mind there are vast differences and limitations to rankings, and that they are not intended to be the sole indication of reputation or excellence.
Lydia Snover, director of institutional research at the Massachusetts Institute of Technology in a Times Higher Education (THE) blog post
“Let me be clear: there is no such thing as a perfect university ranking. There is no ‘correct’ outcome as there is no single model of excellence in higher education, and every ranking is based on the available, comparable data, and is built on the subjective judgement (over indicators and weightings) of its compilers.”
Why is it important to understand methodologies
By understanding the inner workings of rankings, universities gain insights into how their practices and data can ultimately influence a rankings outcome.
The focus area(s), algorithms and methodologies vary across the growing number of ranking tables and reports produced by ranking organizations worldwide.
Where do ranking organizations get their data and information?
All ranking methodologies rely on data inputs from a range of external resources. These can often include the three examples below, but varies based on the niche and focus of each league table:
- Your university’s institutional data and researcher data (based on research output) from bibliometric or citation indexing databases, such as Scopus
- Relevant data on human resources, student administration, finances
- Reputation surveys from faculty, students, alumni and employers
Similarly, the weighting and calculations of the above, and other factors, range greatly based on the specific focus of the ranking report.
How can I find out what weighting and calculation ranking organizations apply?
Most ranking organizations offer some level of information on their methodology on their websites. This can help you to better understand which ranking focuses on and what data and information informs the results. For research-intensive universities, you can get a quick overview here (or download the pdf below).
You can get a quick glance about seven major ranking reports in this guide
This quick reference table provides a look into seven major and influential ranking reports. Use it to quickly compare the different focuses methodologies between the repots.
Note: This information provides a snapshot view of these rankings sourced from their own publicly available websites. Visit their websites to get more details directly from the ranking organizations.
CWTS Leiden Rankings
Focus: Research-intensive universities
Scope: 1,000 institutions
Timing: Annually (June)
- Stated goal: The Leiden Ranking stands for a multidimensional perspective on university performance.
- Data sources: Web of Science data from the Science Citation Index Expanded, the Social Sciences Citation Index, and the Arts & Humanities Citation Index, which is then enriched by CWTS. Excludes conference proceeding publications and book publications
- Methodology: The Leiden Ranking provides information exclusively about the research done at universities. They offer important insights into the scientific performance of nearly 1000 major universities worldwide. Research is represented in publications, and the collected data about these publications forms the basis for the Leiden Ranking. A set of bibliometric indicators are used to provide statistics on scientific impact, collaboration, open access publishing, and gender diversity.
Identification of universities: Typically, a university is characterized by a combination of education and research tasks in conjunction with a doctorate-granting authority. However, these characteristics do not mean that universities are particularly homogeneous entities that allow for international comparison on every aspect. As a result of its focus on scientific research, the Leiden Ranking presents a list of institutions that have a high degree of research intensity in common.
These indicators include:
Source: Accessed May 19, 2020: www.leidenranking.com
World University Rankings
Scope: 1,000 institutions
Timing: Annually (Spring)
- Stated goal: For students seeking to understand how their prospective university choices are perceived by the global academic community, and by potential employers across the world.
- Data sources: Elsevier's Scopus database
- Methodology QS uses a consistent methodological framework, compiled from six simple metrics to capture university performance. Faculty area normalization was introduced in 2015 to ensure that institutions specializing in Life Sciences and Natural Sciences were not unduly advantaged, QS has avoided fundamental changes, with the aim to provide a consistent year-on-year comparison.
QS World University Rankings evaluates universities according to six metrics:
Source: Website accessed on: 19 May 2020: www.topuniversities.com
WUR by Subject
Focus: Individual subject areas (48)
Scope: 1,00 institutions
Timing: Annually (Spring)
- Stated goal: Help prospective students identify the world’s leading schools in their chosen field in response to high demand for subject-level comparisons.
- International Reputation:
- QS global survey of academics
- QS global survey of employers
- Research impact: Elsevier's Scopus database
- Research citations per paper
- h-index in relevant subject
Methodology: Four components are combined to produce the results for each of the subject rankings, with weightings adapted for each discipline:
- Academic reputation
- Employer reputation
- Research citations per paper
As research cultures and publication rates vary significantly across academic disciplines, the QS World University Rankings by Subject applies a different weighting of the four indicators in each subject.
For example, in medicine, where publication rates are very high, research citations and the h-index account for 25% of each university’s total score. On the other hand, in areas with much lower publication rates such as history, these research-related indicators only account for 15% of the total ranking score. Meanwhile, in subjects such as art and design, where there are too few papers published to be statistically significant, the ranking is based solely on the employer and academic surveys.
Source: Website accessed on: 19 May 2020 www.topuniversities.com
Scope: 1800+ institutions are ranked annually, top 1000 are published
Timing: Annually (August)
- Stated goal: Provide a starting point for identifying national strengths and weaknesses as well as facilitating reform and setting new initiatives
- Nobel Prize
- Fields Medals (www.mathunion.org)
- HiCi: Clarivate
- N&S: Web of Science
- Bibliometrics: Web of Science
- Number of academic staff: National agencies such as National Ministry of Education, National Bureau of Statistics, National Association of Universities and Colleges, National Rector's Conference.
Methodology: The highest scoring institution is assigned a score of 100, and other institutions are calculated as a percentage of the top score. An institution's rank reflects the number of institutions that sit above it.
- 10% Quality of Education: Alumni of an institution winning Nobel Prizes and Fields Medals
- 20% Quality of Faculty: Staff of an institution winning Nobel Prizes and Fields Medals
- 20% Quality of Faculty: Highly Cited Researchers
- 20% Research Output: Papers published in Nature and Science*
- 20% Research Output: Papers indexed in Science Citation Index-Expanded and Social Science Citation Index
- 10% Per Capita Performance: Per capita academic performance of an institution
* For institutions specialized in humanities and social sciences such as London School of Economics, N&S is not considered, and the weight of N&S is relocated to other indicators.
Source: Website accessed on: 19 May 2020 www.shanghairanking.com
World University Rankings
Scope: 1,400+ institutions
Timing: Annually (September)
- Evaluate research-intensive universities across all their core missions: teaching, research, knowledge transfer and international outlook.
- Provide trusted performance data on universities for students and their families, university academics, university leaders, governments and industry
- Data sources: Academic Reputation Survey | Elsevier's Scopus database
Methodology: THE uses 13 performance indicators to position more than 1,400+ institutions worldwide. These performance indicators are grouped into five areas (as shown to the right).
30% Teaching (the learning environment):
- 15.0% Reputation survey
- 4.50% Staff-to-student ratio
- 2.25% Doctorate-to-bachelor’s ratio
- 6.00% Doctorates-awarded-to-academic-staff ratio
- 2.25% Institutional income
30% Research (volume, income and reputation):
- 18% Reputation survey
- 6.0% Research income
- 6.0% Research productivity
- 30% Citations (research influence)
7.5% International outlook (staff, students and research)
- 2.5% Proportion of international students
- 2.5% Proportion of international staff
- 2.5% International collaboration
- 2.5% Industry income (knowledge transfer)
- 30% Teaching (the learning environment):
Source: Website accessed on: 19 May 2020 www.timeshighereducation.com
Focus: United Nations’ Sustainable Development Goals (SDGs)
Scope: 1,250 institutions (changes annually)
- Stated goal: To provide a showcase for the work being delivered by universities in our communities, and an opportunity to shine a light on institutional activities and efforts not covered in other rankings and demonstrate the differences a university is making to the world we livein.
- Universities can submit data on as many of the 17 SDGs as they are able
- Elsevier's Scopus database
How universities are ranked:
THE uses indicators to provide comparisons across three broad areas: research, outreach, and stewardship, across all of the SDGs. Any university that provides data on SDG 17 and at least three other SDGs is included in the overall ranking. The methodology was developed in conjunction with THE’s partners Vertigo Ventures and Elsevier, and after consultation and input from individual universities, academics, and sector groups. Universities can submit data on as many of the SDGs as they are able. Each SDG has a series of metrics that are used to evaluate the performance of the university in that SDG. As well as the overall ranking, THE also publishes the results of each individual SDG. This rewards any university that has participated with a ranking position, even if they are not eligible to be in the overall table.
A university’s final score in the overall table is calculated by combining its score in SDG 17 with its top three scores out of the remaining 16 SDGs. SDG 17 accounts for 22% of the overall score, while the other SDGs each carry a weight of 26%. This means that different universities are scored based on a different set of SDGs, depending on their focus.
The score from each SDG is scaled so that the highest score in each SDG in the overall calculation is 100. This is to adjust for minor differences in the scoring range in each SDG and to ensure that universities are treated equitably whichever SDGs they have provided data for. It is these scaled scores that we use to determine which SDGs a university has performed most strongly in; they may not be the SDGs in which the university is ranked highest or has scored highest based on unscaled scores
Source: Website accessed on: 19 May 2020 www.timeshighereducation.com
Best University Rankings
Scope: 1,500 institutions across more than 80 countries
Timing: Annually (October)
- Stated goal:
- For potential students: used to explore higher education options beyond their own countries' borders and to compare key aspects of schools' research missions.
- For universities: provide a way to benchmark themselves against other schools, become more visible globally, and find top schools in other countries to collaborate with.
- Data sources:
- Bibliometric data and indicators: Web of Science and InCites
- Reputation Indicators: Clarivate Analytics’ Academic Reputation Survey
- Methodology: The rankings focus specifically on schools' academic research and reputation overall and not on their separate undergraduate or graduate programs. To arrive at a school's rank, the overall global scores are calculated using a combination of the weights and z-scores for each of the 13 indicators used in the rankings.
Source: Website accessed on: 19 May 2020 www.usnews.com
5. Can I influence university rankings?
If you seek to establish, maintain or improve your institution's reputation and standing, understanding the impact of your university's data and information in the calculation of international university rankings is essential. These data points can include:
- Your research output
- Citation data
- Institutional data
Further, gaining insight into what that data includes and where it can be improved puts you in a better position to manage your university’s rankings and reputation.
Rankings evolve, staying on top of changes is key
Even if your university has placed high in its valued rankings, algorithms and methodologies change, as do institutional priorities and research outputs. By understanding the inner workings of rankings, and monitoring them on an ongoing basis, both large and small universities can improve their practices in ways that ultimately will influence their ranking.
What are some steps I can take to influence ranking outcomes?
When you begin to investigate how you can influence rankings, the questions below can help you put you on a good path forward.
Which ranking should I focus on first?
What is the methodology applied by the ranking organization?
What is the source of citation and publication information and data?
Do I have a data-hygiene plan and schedule?
NOTE: THE and QS both use data from Scopus, which can be pro-actively optimized through the Institution Profile Wizard to better reflect your institution’s output.
See the Elsevier ebook for a deep-dive into the data behind THE and QS rankings, and also learn from Liz Bober, Library Assessment Officer at Case Western Reserve University, on how she did this for her university.
Webinar: “Optimizing Scopus research data points for Times Higher Education rankings”.
Which tools can give me views into my university and researcher performance?
Invest in and leverage tools like SciVal that give you insightful views into your university and researcher performance, allowing you to:
Proportion of total publications with at least one international co-author
The bottom line: Will I rise in the rankings?
Although a proactive data hygiene practice cannot guarantee a rise in rankings, it can lead to a more accurate reflection of your university in ranking outcomes through the validation of research output and citation data attributed to your institution. Further, deep analyses of the same data can yield actionable insights that drive your institutional strategy forward and help you manage your reputation.