The guardians of Scopus

Here’s how independent subject experts monitor the titles in Scopus to uncover predatory journals

By Wim Meester, PhD - February 26, 2021
detective image
@ istock.com/South_agency

Whether you do research or use research to make critical decisions, you need access to the highest quality content. That’s why we make it a priority to maintain the integrity of Scopus through expert curation and continual monitoring.

As the largest abstract and citation database of peer-reviewed literature, Scopus features the content of more than 7,000 publishers, with tools that allow you to track, analyze and visualize this research. Scopus is curated by a Content Selection and Advisory Board (CSAB), comprised of editorially independent subject experts who are vigilant in identifying and discontinuing journals that are — or have become — predatory or of poor quality. Using strict title selection criteria, these experts determine whether journals are eligible for indexing in Scopus, and they continually re-evaluate these titles to make sure quality is maintained.

Prof Jörg-Rüdiger Sack, PhD“Maintaining the integrity of Scopus and its high-quality, curated content is of paramount importance to me, the board and Scopus,” said Prof Jörg-Rüdiger Sack, Chair of CSAB and Subject Chair for Computer Sciences. “Poor-quality and predatory journals are also a threat to the integrity of science as well as to the credibility in the general population.”

A recent article in Nature focused on the presence of predatory journals in Scopus based on a study in Scientometrics describing analysis of a report from 2017, using “Beall’s List” to define predatory journals. This list, which was compiled by librarian Jeffrey Beall, has not been maintained since 2017. We were aware of it and took action at the time. As part of our journal re-evaluation program, the board re-evaluated all the Beall’s List journals in Scopus and discontinued those they determined to be predatory.

Of course, we welcome any research that sheds light on predatory journals. At the same time, determining whether a journal is predatory is a complex process that requires a detailed review based on various considerations. There is no universally agreed upon definition of a predatory journal or publisher. And to complicate matters, a journal can become predatory over time. As Prof Sack explained:

Whether or not a journal is predatory is not a binary decision; there is a broad spectrum of predatory journal behaviors. To assess whether a journal should be considered predatory, a number of parameters need to be considered, often in combination. Analogously for publishers, while some publishers clearly exhibit general predatory behavior, others may publish a mix of journals – some definitely predatory, some less so, yet others not. This often depends on the strength and independence of editorial boards. This makes publisher-level black-listing difficult and possibly even flawed.

Also, predatory publishers are always adapting to detection measures put into place. But we also adapt, coming up with novel strategies to monitor and identify these journals.

Furthermore, we have witnessed a time-variant factor. Several journals, correctly admitted into Scopus based on their quality, changed over time. Measurable changes included dramatic increases in volume and modification of editorial direction, topics, even entire fields, editorial boards and publishers. Here, the benefit of being listed in Scopus is predatorially exploited over time. This makes a process of re-evaluation essential.

Scopus has several drivers for re-evaluation, he said. Journals may be flagged by an automatic tool that regularly surveys the Scopus database, or they may be targeted for re-evaluation when concerns are raised.

Of the titles that were flagged for re-evaluation in 2017 because of concerns over publication practices (including those journals listed by Beall), the board decided to stop covering 65 percent of them.

When the decision is made to discontinue covering a journal, the content that is already in Scopus remains, but no new articles are included. Crucially, content already indexed remains as a matter of scientific record and to ensure the stability and consistency of research trend analytics — a core value of Scopus.

Prof Peter Brimblecombe, PhDAs Prof Peter Brimblecombe, CSAB Subject Chair for Environmental Science, explained:

It is clearly important to remove journals that are predatory or adopt poor publishing practices from the Scopus database. It is less clear what to do with earlier content in the database, particularly as it may have arisen in periods when the journal was well managed and represents output from well conducted research. Currently when a journal is removed from Scopus, it maintains previously indexed material, thus retaining work that is often already cited, built into H-indices or is linked to personal and institutional profiles. This ensures that metrics remain stable and don't change after re-evaluation.

You can find a complete overview of which titles have been discontinued, and the last content indexed on the platform from each, on the Scopus info site in the Title Re-evaluation section.

Karen HollandKaren Holland, CSAB Subject Chair for Nursing and Health Professions, said board members and the Scopus team use a variety of methods and indicators to assess whether journals are predatory:

There are fundamental rubrics that we have collectively agreed upon, underpinned by Principles of Transparency and Best Practice in Scholarly Publishing (COPE et al, 2018). These need to be present in a non-predatory journal.

The volume of journals submitted for consideration has added to the understanding we have as to what can be clearly considered to be a predatory or unethical publication. This can be clearly differentiated from a poor-quality journal where it is evident that there is no intention to mislead and which provides clarity on how the journal is run. The unethical journal does the opposite and has clearly visible evidence, when assimilated, that can indicate unethical policies and misrepresentation of journal content.

The Scopus re-evaluation process

These are the three metrics and benchmarks used to identify underperforming journals for re-evaluation on Scopus. (Source: Scopus Content Policy and Selection)

Metric

Benchmarks and Explanation

Self-citation rate

The journal has a substantially higher self-citation rate when compared to peer journals in its subject field.

Total citation rate

The journal received a substantially lower number of citations when compared to peer journals in its subject field.

CiteScore

The journal has a substantially lower CiteScore when compared to peer journals in its subject field.

There are four ways a journal can be flagged for re-evaluation in Scopus:

  1. The journal is underperforming as it does not meet any of the three metrics and benchmarks (above) that have been developed and agreed by the CSAB in partnership with the Scopus team.
  2. Concerns about the publication standards of the journal or publisher have been raised by formal complaints.
  3. The journal shows outlier behavior based on its publishing performance in Scopus.
  4. Collection and analyses of previous CSAB title evaluation feedback can result in re-checking the journal for Scopus coverage

This article details how the Scopus board identifies and re-evaluates predatory journals.

Contributors


Wim Meester, PhD
Written by

Wim Meester, PhD

Written by

Wim Meester, PhD

As Head of Product Management for Scopus, Dr Wim Meester is responsible for Scopus content strategy and how it meets the needs of customers in the academic, government and corporate market segments. In addition, he manages the independent, international Scopus Content Selection & Advisory Board (CSAB) and strategic partnerships with relevant research organizations and third-party publishers.

Wim received his MSc and PhD in chemistry from the University of Amsterdam. He spent two years at Harvard Medical School as a postdoc, before making the change from being an active researcher to a program manager for the Netherlands Organisation for Scientific Research (NWO). In 2006, Wim joined Elsevier as a Publisher for the Animal Science and Forensic Science journal programs. In 2010, he joined the Scopus team.

Related stories


Advancing responsible research assessment
At Harvard, developing software to spot misused images in science

Comments


comments powered by Disqus