Elsevier Connect

Research Evaluation & Funding

The Impact of Science: how research can be measured and spending maximized

Policymakers, academics and industry professionals gather to discuss challenges and trends in Europe at The Impact of Science conference

[caption id="attachment_23371" align="alignnone" width="800"]Presenters at Impact of Science conference Presenters at The Impact of Science conference: Professor Shirley Pearce, Board Member of the Higher Education Funding Council for England (HEFCE); Erkki Ormala, PhD, Professor of Innovation Management at Aalto University in Finland; Nick Fowler, PhD, Managing Director of Academic and Government Institutions at Elsevier;  and Professor Donald Dingwell, Secretary General of the European Research Council.[/caption] [divider] [caption id="attachment_23369" align="alignleft" width="128"]Iris Kisjes Iris Kisjes[/caption]

The Author

Iris Kisjes has worked at Elsevier for seven years, the past four in communications. She is currently Senior Corporate Relations Manager, based in Amsterdam. She has a keen interest in the knowledge economy, especially in relation to the valorization of science and the longevity of the higher education system. [divider] National austerity measures and the economic downturn have put pressure on government budgets across Europe resulting in an increased competition for the public purse. As a result, academic institutions find themselves competing for public funding not only with one another but with other industries, augmenting the need to demonstrate their return on investment. In addition, taxpayers globally want to know how their investment will benefit the society they live in, and demand greater transparency.

These issues were addressed at The Impact of Science conference in Amsterdam April 25. It brought together policymakers, academic research managers and industry professionals from Europe to discuss how research can be measured and how research spending can be optimized. The conference was organized by the Science Alliance in collaboration with Elsevier, which also sponsored the event.

The need to show social benefits

[caption id="attachment_23374" align="alignleft" width="194"]Douwe Breimer, PhDDouwe Breimer, PhD[/caption] As chair of the meeting, Dr. Douwe Breimer, Professor of Pharmacology and previously Rector Magnificus and President at Leiden University, led the audience in a discussion with his reference to the Nature article"A back seat for basic science."

The article outlines the trend of many governments to provide funding only for applied research that can show near-term economic and social benefits. The stage was thereby set, and the debate opened on whether these policies will shape the future that many of us hope for: the valorization of science.


In this context, valorization refers to the use of publicly funded research for socio-economic purposes. It represents society's direct and indirect return on the public sector's investment. The approaches to valorization and its potential benefits for the different stakeholders have improved significantly over the past 20 years, notably due to the digitization of research, patents and the online sharing culture. [caption id="attachment_23379" align="alignright" width="185"]Karl Dittrich, PhD Karl Dittrich, PhD[/caption]

The paradox of funding

Dr. Karl Dittrich, President of theAssociation of Universities in the Netherlands, emphasized the important role government plays in providing the right impulse to ensure that research drives national economic growth. He illustrated the paradox the government finds itself in in the Netherlands: government officials want to stimulate private investment in research while a rating agency reports that the returns on this type of investment are poor (in contrast to the advice of

OECDand the World Bank, which say this is a worthy investment).


This article in GlobalHigherEd elucidates the role of credit rating agencies on global higher education: "'Passing judgment': the role of credit rating agencies in the global governance of UK universities."

Dr. Dittrich explained that studies have demonstrated the significant impact of universities nationally because universities drive economic growth, with Leiden University providing a 250 percent return on every euro spent though in areas such as knowledge transfer and infrastructural returns. He showed that there is a shift in the balance of power such that smaller universities can compete with larger institutions, such as Harvard, by focusing on specialized niche areas.

This phenomenon emphasizes the need for smaller institutions to be strategic in their approach and to understand their core strengths and their competitive position within the knowledge economy, as well as how they can best influence their standing while serving the greater good of society. Universities will need to continue to prove their relevance to society to be successful in acquiring large investments.

Peer review and forecasting ROI

[caption id="attachment_23394" align="alignleft" width="166"]Professor Donald DingwellProfessor Donald Dingwell[/caption] Professor Donald Dingwell, Secretary General of the

European Research Council, addressed the central importance of peer review in evaluating applications for research grants.  He explained that the review boards, which consist of subject matter experts, need to estimate the potential value of a research proposal because the true value of the outcome is hard to quantify prior to performing the research.


It was clear from his presentation that despite the desire and need to measure return on investment, there is no uniform way to institutionalize such practice across all disciplines, or even within disciplines. He said the evaluation of impact is hard work and more easily proven retrospectively rather than upfront. The impact is made and felt beyond the duration of elected officials' terms of office, so the responsibility lies with those who are apply for funding to state as clearly as possible what the desired or foreseeable impact could be, and with members of the expert review panel to interpret and forecast the impact from their subjective standpoints.

Ultimately, he said, it is the review board that will shape the innovations of tomorrow; it is these specially screened boards we have to trust to ensure innovation excellence.

He also explained that it is difficult to predict the impact one's work will have on society. For example, the winners of the 2010 Physics Nobel prize for groundbreaking experiments regarding the two-dimensional material graphene did not include in their proposal that their research would end up being award winning, create billions of dollars of invested into this research area and generate thousands of patents globally. [caption id="attachment_22464" align="alignright" width="168"]Nick Fowler, PhD Nick Fowler, PhD[/caption]

Despite the difficulty, Professor Dingell concluded, it is essential it is for researchers to present the potential impact of their work for the purpose of a fruitful evaluation process: "Find it, measure it, present it."

Why the Netherlands is a top research nation

Dr. Nick Fowler, Managing Director of Academic and Government Institutions at Elsevier, presented the global research position of the Netherlands, showing the wealth and breadth of data available to help inform governments, funding bodies and research institutions in their quest to establish, execute and evaluate their strategies. He discusses his presentation in theElsevier Connectarticle "

The Netherlands — a Top research nation," illustrating a number of different citation- and publication-based parameters that one can utilize to get an over-all sense of a nation's research capabilities.

An example I found particularly compelling was the analysis of citations per gross expenditure on R&D as it provides an indication of a nation's efficiency in utilizing public money: [caption id="attachment_23367" align="alignnone" width="800"]Citations per Gross Expenditure on R&D Citations per Gross Expenditure on R&D[/caption] [caption id="attachment_23391" align="alignright" width="162"]Erkki Ormala, PhD Erkki Ormala, PhD[/caption]

Industry's role in driving research innovation

Dr. Erkki Ormala, Professor of Innovation Management atAalto Universityin Finland, presented the role industry plays in driving innovation within research. The essence lies in creating strategic partnerships between industry and academia and in ensuring that the transfer of knowledge is flowing both ways since industry offers just as much knowledge as they gain from academia, he said.

He went on to discuss the "democratization of innovation" through open innovation, where co-creation and crowdsourcing play an ever increasing role. He said it's an exciting time, as we see an increase in the contribution from geographical areas which were previously hard to collaborate with, namely Asia and Africa.

Dr. Ormala left us with two interesting questions. First, why is it that industry perceives academic research and the scientific process as a key driver of innovation, economic growth and social development yet lacks investment in this area? It widely recognizes the necessity of academic research but in the end does not seem to accept it.

Second, why do governments change their commitment to research according to electoral terms, when support for research would be more effective as a long-term commitment that transfers from one government to the next? [caption id="attachment_23388" align="alignright" width="189"]Professor Shirley Pearce Professor Shirley Pearce[/caption]

Making the most of public sector R&D spending

Professor Shirley Pearce, Board Member of theHigher Education Funding Council for England (HEFCE), presented the audience with a number of unique UK initiatives to enhance the value of public sector R&D spending. HEFCE had found there was a need to provide a guide to industry and academia on how to have profitable partnerships, as governance and communication flows often proved to be hurdles to success. HEPCE's report

Enhancing Collaboration Creating Value has more information on how to enhance industry-academic partnerships.

Europe lags behind US and China in research investment

[caption id="attachment_23385" align="alignleft" width="148"]Reinhilde Veugelers, PhDReinhilde Veugelers, PhD[/caption] Dr. Reinhilde Veugelers, a professor in the Department of Management, Strategy and Innovation of

Katholieke Universiteit Leuven,  painted a bleak picture for Europe, with the investment and performance indicators lagging far behind those of the US and China. One reason for this, he said, was that the "yollies" (young innovative companies) and the "ollies" (open-innovation companies that combine internal R&D with external sourcing from academia) seemed to contribute well to the value of research in the US, while the centrally governed research machine is working well for China.

Conclusion

It is clear that there is a need to evaluate science but no perfect or simple way to do so. In the end, it is up to all involved to establish models and standards that can help with the process, while personal interpretation will be needed to turn the data in to valuable information.


On the one hand, there needs to be trust — in the people who review research proposals and their ability to recognize good research ideas, and in the systems and experts that will evaluate the outcomes of the research, recognize its success and help to inform others where to allocate further funding. On the other hand, there needs to be balance — between our eagerness to quickly prove the return on investment of our financial contributions and the need to give research enough time to show its benefit to society.



comments powered by Disqus