The guardians of Scopus
February 10, 2021 | 11 min read
By Wim Meester, PhD
Here’s how independent subject experts monitor the titles in Scopus to uncover predatory journals
Editor's note: The research(opens in new tab/window) referred to here on alleged predatory journals being indexed in Scopus was retracted on September 6, 2021.
Whether you do research or use research to make critical decisions, you need access to the highest quality content. That’s why we make it a priority to maintain the integrity of Scopus through expert curation and continual monitoring.
As the largest abstract and citation database of peer-reviewed literature, Scopus features the content of more than 7,000 publishers, with tools that allow you to track, analyze and visualize this research. Scopus is curated by a Content Selection and Advisory Board (CSAB), comprised of editorially independent subject experts who are vigilant in identifying and discontinuing journals that are — or have become — predatory or of poor quality. Using strict title selection criteria, these experts determine whether journals are eligible for indexing in Scopus, and they continually re-evaluate these titles to make sure quality is maintained.
“Maintaining the integrity of Scopus and its high-quality, curated content is of paramount importance to me, the board and Scopus,” said Prof Jörg-Rüdiger Sack(opens in new tab/window), Chair of CSAB and Subject Chair for Computer Sciences. “Poor-quality and predatory journals are also a threat to the integrity of science as well as to the credibility in the general population.”
A recent article in Nature(opens in new tab/window) focused on the presence of predatory journals in Scopus based on a study in Scientometrics(opens in new tab/window) describing analysis of a report from 2017, using “Beall’s List” to define predatory journals. This list, which was compiled by librarian Jeffrey Beall, has not been maintained since 2017. We were aware of it and took action at the time. As part of our journal re-evaluation program, the board re-evaluated all the Beall’s List journals in Scopus and discontinued those they determined to be predatory.
Prof Jörg-Rüdiger Sack, PhD
Of course, we welcome any research that sheds light on predatory journals. At the same time, determining whether a journal is predatory is a complex process that requires a detailed review based on various considerations. There is no universally agreed upon definition of a predatory journal or publisher. And to complicate matters, a journal can become predatory over time.
As Prof Sack explained:
Whether or not a journal is predatory is not a binary decision; there is a broad spectrum of predatory journal behaviors. To assess whether a journal should be considered predatory, a number of parameters need to be considered, often in combination. Analogously for publishers, while some publishers clearly exhibit general predatory behavior, others may publish a mix of journals – some definitely predatory, some less so, yet others not. This often depends on the strength and independence of editorial boards. This makes publisher-level black-listing difficult and possibly even flawed.
Also, predatory publishers are always adapting to detection measures put into place. But we also adapt, coming up with novel strategies to monitor and identify these journals.
Furthermore, we have witnessed a time-variant factor. Several journals, correctly admitted into Scopus based on their quality, changed over time. Measurable changes included dramatic increases in volume and modification of editorial direction, topics, even entire fields, editorial boards and publishers. Here, the benefit of being listed in Scopus is predatorially exploited over time. This makes a process of re-evaluation essential.
Scopus has several drivers for re-evaluation, he said. Journals may be flagged by an automatic tool that regularly surveys the Scopus database, or they may be targeted for re-evaluation when concerns are raised.
Of the titles that were flagged for re-evaluation in 2017 because of concerns over publication practices (including those journals listed by Beall), the board decided to stop covering 65 percent of them.
When the decision is made to discontinue covering a journal, the content that is already in Scopus remains, but no new articles are included. Crucially, content already indexed remains as a matter of scientific record and to ensure the stability and consistency of research trend analytics — a core value of Scopus.
As Prof Peter Brimblecombe, CSAB Subject Chair for Environmental Science, explained:
It is clearly important to remove journals that are predatory or adopt poor publishing practices from the Scopus database. It is less clear what to do with earlier content in the database, particularly as it may have arisen in periods when the journal was well managed and represents output from well conducted research. Currently when a journal is removed from Scopus, it maintains previously indexed material, thus retaining work that is often already cited, built into H-indices or is linked to personal and institutional profiles. This ensures that metrics remain stable and don't change after re-evaluation.
Prof Peter Brimblecombe, PhD
You can find a complete overview of which titles have been discontinued, and the last content indexed on the platform from each, on the Scopus info site (navigate to the section Title Re-evaluation).
Karen Holland(opens in new tab/window), CSAB Subject Chair for Nursing and Health Professions, said board members and the Scopus team use a variety of methods and indicators to assess whether journals are predatory:
There are fundamental rubrics that we have collectively agreed upon, underpinned by Principles of Transparency and Best Practice in Scholarly Publishing(opens in new tab/window) (COPE et al, 2018). These need to be present in a non-predatory journal.
The volume of journals submitted for consideration has added to the understanding we have as to what can be clearly considered to be a predatory or unethical publication. This can be clearly differentiated from a poor-quality journal where it is evident that there is no intention to mislead and which provides clarity on how the journal is run. The unethical journal does the opposite and has clearly visible evidence, when assimilated, that can indicate unethical policies and misrepresentation of journal content.
Karen Holland
The Scopus re-evaluation process
Metric | Benchmarks and Explanation |
Self-citation rate | The journal has a substantially higher self-citation rate when compared to peer journals in its subject field. |
Total citation rate | The journal received a substantially lower number of citations when compared to peer journals in its subject field. |
CiteScore | The journal has a substantially lower CiteScore when compared to peer journals in its subject field. |
The three metrics and benchmarks used to identify underperforming journals for re-evaluation on Scopus. (Source: Scopus Content Policy and Selection)
There are four ways a journal can be flagged for re-evaluation in Scopus:
The journal is underperforming as it does not meet any of the three metrics and benchmarks (above) that have been developed and agreed by the CSAB in partnership with the Scopus team.
Concerns about the publication standards of the journal or publisher have been raised by formal complaints.
The journal shows outlier behavior based on its publishing performance in Scopus.
Collection and analyses of previous CSAB title evaluation feedback can result in re-checking the journal for Scopus coverage
This article details how the Scopus board identifies and re-evaluates predatory journals.
Contributor
WMP