“Nobody agrees on what AI is” – How Elsevier’s report used AI to define the undefinable

Our colleagues came up with a new way to analyze the complex research field of artificial intelligence

AI Report keyword chart
Figure 2.1 in Elsevier's AI report shows keyword mapping (the number and percentage of keywords) between AI sectors and perspectives. We find little overlap between the way AI is spoken about in teaching, research, industry, and the media. The four perspectives share just six broad and general keywords, most of which relate to learning: "Artificial Intelligence," "Deep Learning," "Machine Learning," "Neural Network," "Reinforcement Learning," and "Speech Recognition." A Scopus search for these six keywords (as OR statements) for the years 1998-2017 yielded a corpus of comparable size. (Source: Artificial Intelligence: How knowledge is created, transferred, and used, 2018)

Few innovations are discussed in the same way as artificial intelligence. Governments see it as vital to the future of their security and economy. The media sees it as both savior and existential threat. And academia sees it as something in between: a tool with some very useful applications.

Overall, none of these groups seems to agree on what it is. When a team of data analysts at Elsevier set out to analyze the state of play and map the trends of AI research for an in-depth report , they had significant challenges to overcome – not least of which was to decide what AI is.

“Whenever you read a policy document about AI, almost every introduction says there’s no common or agreed definition,” said Dan Olley, Elsevier’s Chief Information Officer. “But they also follow a line of reasoning that says ‘it’s a transformational power in society, we as a country need to have a share in that, and to set ourselves up for this AI future, we need to educate students to create an AI workforce and ensure that people in other professions are AI aware.

“If you ask policymakers why, it’s because it leads to innovation and growth. Running across all four of those, you have this ethics in AI stream, because there’s a concern that when you’re talking about something so powerful, it has to be applied to the benefit of society.”

Everybody wants to have a slice of AI and to control AI, but nobody agrees on what AI is. As Olley explained:

The confusion of terms in a field with such potential to transform lives needs to be addressed to ensure that policy objectives are correctly translated into research priorities, student education matches job market needs, and media can compare the knowledge being developed in various countries and regions around the world.

The report team identified four perspectives of AI: Media, Research, Industry and Teaching (Source: <em>Artificial Intelligence: How knowledge is created, transferred, and used, 2018</em>)Addressing that confusion was the challenge the report team set themselves. Using the knowledge transfer sequence Olley outlined above, the team identified four perspectives on AI: Media, Research, Industry and Teaching.

For Elsevier’s previous reports – for example, on the state of brain science and gender in research – the approach was to identify a set of approximately 60 keywords and use those to analyze a huge corpus of research. However, it quickly became clear that this approach wouldn’t fly for AI.

Maria de KleijnAs Maria de Kleijn, Senior VP of Analytical Services elaborated:

We had a hypothesis that we’d find a reasonable congruence of keywords between those four domains. What we found is that there was very little intersection at all. That’s an important message. Consider that every politician reads media. The way that AI is reported in media is almost a world of its own. Two-thirds of the AI keywords for media only come up in media; they don’t feature in Education or Patents or Research at all. How can you map that chain – from media, to research, to education to innovation – if all four of them speak a different language?

Between the four domains, the team found that there were overall 797 keywords, too great a number on which to search: applying them in Scopus generated 6 million results. “You get a lot of false positives,” Maria said. “For example, one of the keywords is ‘learning’ – if you type that into Scopus, you get back results on ‘how do children learn to write,’ or ‘how do babies find their feet?’”

Mark SiebertAgain, the methodology of Elsevier’s previous results didn’t solve the issue. For a report on Brain Science, for example, the team could limit results to journals in a specific field to reduce the number of false positives. Mark Siebert, Elsevier’s Director Partnering and Analytics, explained why that wouldn’t work for AI.

As some of the experts quoted in the final report point out, AI is a general-purpose technology – a bit like electricity or nanotechnology. It comes up inmultiple disciplines. So you might have someone in the health sector researching statistical algorithms they don’t refer to as AI but which are generally considered an AI technology. And their results might be published in a medical journal. Elsewhere, someone might label something AI that’s not really AI, such as processes like data mining.

While the 797 keywords across Media, Research, Industry and Teaching yielded too many results, looking at the words common to all domains was equally unsatisfactory – there were just six keywords applicable to all four. So, the team turned to AI itself.

Siebert explained:

Our Elsevier Labs team built a supervised learning tool, which itself is an AI technology. They used their expert viewpoint to identify 1,500 papers they viewed as absolutely AI research papers and used those as a training set for the learning tool: if these 1,500 papers are definitely AI, which of the 6 million results from our 797 keywords is also AI? That process kicked out about 90 percent of the results, giving us a meaningful set of 600,000 papers published in the past 20 years, which we could analyze to establish the trends.

That was used as the basis for the trend analytics and further definition of the AI field and the specific research fields within AI.

One of the findings the analysis turned up was that globally, AI research has accelerated to grow by over 12 percent annually in the past five years (2013-17) compared to an annual growth rate of less than 5 percent for the previous 5 years (2008-12). By contrast, research output overall, globally across all subject areas, has grown by 0.8 percent annually over the past five years (2013-17).

Prof. Enrico Motta, PhDCommenting in the foreword of the report, Dr. Enrico Motta, Professor of Knowledge Technologies at The Open University in the UK, said:

To my knowledge, the report provides the most comprehensive characterization of AI outputs produced so far. Crucially, in contrast with manually developed taxonomies of research areas, which inevitably end up reflecting the specific viewpoints of the experts involved in the process, this characterization is data-driven, using machine learning and text-mining techniques to classify documents and identify the keywords. Thus, in my view, the report enjoys greater validity, providing a more objective reflection of the variety of existing contributions to the AI field.

The report also found that industry in the United States attracts the most AI talent from both local and international academia. In Europe, there’s a stronger move of academic talent moving to non-European industry. Over the last three years, the data shows Chinese academia attracting more AI talent than it is losing, confirming that the country is on track to establish a leading position in AI research. Having overtaken the United States in AI research output in 2004, China is set to overtake Europe and become the biggest source of AI research globally in five years, if the pace of current trends continues.

You can read more about these findings in Elsevier’s Artificial Intelligence Resource Center.

Download the AI reportArtificial Intelligence report

Artificial Intelligence: How knowledge is created, transferred, and used

Download full report

Download the executive summary



Written by

Ian Evans

Written by

Ian Evans

Ian Evans is Content Director for Global Communications at Elsevier. Previously, he was Editor-in-Chief of Elsevier’s Global Communications Newsroom. Based in Oxford, he joined Elsevier six years ago from a small trade publisher specializing in popular science and literary fiction.

Related stories


comments powered by Disqus