How does Elsevier’s text mining policy work with new UK TDM law?

With its policy and technology, Elsevier aims to support the research community and respond to changing needs

In January, Elsevier announced a new text and data mining policy, which allows academic researchers at subscribing institutions to text mine subscribed content for non-commercial research purposes.

Last week, a new UK text and data mining copyright exception came into force which allows researchers with lawful access to works to make copies of these for the purposes of non-commercial text and data mining. Accordingly, it's is a good opportunity to reflect on how our policy and the exception work together.

Elsevier and the UK TDM copyright exception

A new UK text and data mining copyright exception came into force on June 1st. What is it and how do Elsevier's systems accommodate this requirement?

  • An exception to copyright is when someone is allowed to copy a work without seeking the permission of the rights holder. In this instance, researchers with lawful access to works published by Elsevier can copy these without asking, using tools we have provided for this purpose, provided they are doing the copying to carry out non-commercial text and data mining.
  • Elsevier offers an Application Programming Interface (API) to facilitate text and data mining of content held on Science Direct. This API makes the process easier and more efficient for researchers compared to manual downloading and mining of articles. It also helps us to provide a good experience to human readers and to miners at the same time.
  • Under the UK legislation, publishers can use "reasonable measures to maintain the stability and security" of their networks, and so the requirement to use this API is fully compatible with the copyright exception.
  • Our approach to TDM remains under review and continual refinement. We have already made changes based on researcher feedback during our pilot and will continue to do so in order to support researchers.
  • We believe text and data mining is important for advancing science, and we are keen to provide tools to support researchers who wish to mine no matter where they are located.


Related resources

Elsevier has provided text and data mining support for researchers since 2006. We designed our policy framework to span across all legal environments as research is global, and this framework complements the UK exception. Since the beginning of the year, in accordance with our policy, we have started to include text and data mining rights for non-commercial purposes in all new ScienceDirect subscription agreements and upon renewal for existing academic customers. The UK law adds weight to our position; we are ensuring that those with "lawful access" (in UK legislation speak) have the right to mine our works.

Contrary to what some have suggested, our policy was not designed to undermine library lobbying for copyright exceptions for text and data mining, but rather to position us to continue to offer flexible and scalable solutions to support researchers no matter where they are based.

What the law alone cannot do – in the UK or elsewhere – is resolve some of the technical sticking points that often frustrate a researcher's mining experience. That's why our policy facilitates text mining via an Application Programming Interface (API).

The advantages of using APIs for text mining

As users of many popular websites will know, it is standard best practice for users (well, their machines) to be asked to use APIs or other download mechanisms when the website in question holds a lot of content. That's the case with ScienceDirect, which holds over 12.5 million articles and almost 20,000 books, and we are among many other large platforms, including Wikipedia PubMed Central and Twitter, in asking for our API to be used for downloading and mining content. We do this to provide researchers with an optimum text mining experience.

For starters, access via the API provides full-text content of ScienceDirect in XML and plaintext formats, which researchers tell us they prefer to HTML for mining. Similarly, experience in our pilots has indicated that text miners prefer API access for automated text mining for several other reasons, one being that content is available from our APIs without all of the extraneous information that is added to web pages intended for human consumption but which make text mining more difficult (e.g., presentational JavaScript, navigational controls and images, website branding, advertisements). Access via our API also provides content to researchers in stable, well-documented formats; by contrast, HTML coding can change at any time, making it arduous to keep "screen-scraping" scripts up to date.

It's not just text miners who benefit from our API, but users of ScienceDirect who are there to read content rather than download and mine it. Their user experience of ScienceDirect can be maintained at the highest level, as bulk downloading needed for mining is done elsewhere, via our API. If bulk downloading over a short period of time took place on the ScienceDirect site, the system's stability would be compromised, affecting researchers of every hue. By contrast, our API is designed to cope with high-frequency requests from automated bots and crawlers in a very efficient manner which enables us to scale our systems to meet demand.

The Explanatory Notes published alongside the UK legislation make clear that publishers are able to impose "reasonable measures to maintain the stability and security" of their networks, as long as researchers are able to benefit from the exception to carry out non-commercial research. In other words, researchers with lawful access to works can copy these for the purposes of non-commercial text and data mining, and publishers have a role to play in managing this process. The "reasonable measures" include requesting that miners to carry out text mining via a separate API, in line with Elsevier's existing policy, and we have received numerous reassurances from the UK Government that use of our API will be in compliance with the law.

Text Mining Primer

API (application programming interface) – An interface for a software program that enables interaction with other software, similar to the way a user interface facilitates interaction between humans and computers.

Entity – In text mining, an entity may refer to a group of words, code, statistics or anything else in the document that can provide information. For Elsevier's customers, entities of interest often include such things as chemical names, genes, proteins or sequences.

Text mining – The process of deriving information from articles by extracting word patterns and other relationships that could lead to new discoveries.

We will continue to monitor how our API is used and to make tweaks and changes to our policy in response to community feedback. We have already made several adjustments. For example, we no longer request a project description as part of the API registration process, and we now allow TDM output to be hosted in an institutional repository. We also know, for example, that researchers would like to mine third-party images and graphics that they cannot currently download automatically via our API. We of course make this content available to researchers on request, but we are looking at how we might ensure that the rights of third-party content owners are respected whilst at the same time providing researchers with all of the content they want immediately via our API. And we are a signatory to the new CrossRef Prospect text and data mining service, which aims to allow researchers to mine content from a range of publishers through one single portal.

Further, we're looking at how we ensure that researchers know what they can and cannot do with content, or where to go for further information, without giving the impression that we are claiming ownership over non-copyrightable facts and data. We've already altered our output terms, so that researchers can redistribute 200 characters in addition to text entity matches; researchers told us that our previous inclusion of text entity matches within that 200 character limit sometimes caused problems when displaying lengthy chemical formulas.

In short, we will continue to do what we have always done: work with the research community to support their research, listen to feedback and respond to changing needs. Our text and data mining policy is a reflection of this and will continue to evolve accordingly.


Elsevier Connect Contributor

Gemma HershAs Policy Director on the Policy and Access team at Elsevier, Gemma Hersh (@gemmahersh) is responsible for developing policy for open access, copyright and other areas that impact the scholarly research and publishing communities. Her current work includes looking at open access and copyright developments globally and emerging areas such as Massively Open Online Courses (MOOCs) and Open Education Resources.

Before joining Elsevier two months ago, Hersh was Head of Public Affairs for the UK Publishers Association and has worked in the creative industries both in government and in industry for the last six years. She holds an MPhil in Politics and Comparative Government from Oxford University, but her real love is History, in which she holds a First Class Degree from Kings College, London.

comments powered by Disqus

Related Stories