Lessons from a large-scale audit: strengthening trust through rigour, transparency and collaboration

Safeguarding the scholarly record can be challenging. When the scale of a research integrity journal audit extends to tens of thousands of articles, it becomes a broader test of editorial processes, infrastructure, collaboration, and institutional oversight.

This was the context for Elsevier’s recent assessment of Heliyonopens in new tab/window; an extensive review undertaken to support re-evaluation by Web of Science and, just as importantly, to reinforce confidence in the integrity of its content. The corrections made affect approximately 3% of content published in Heliyon across 12 years of publication.

The 18-month process delivered not only the removal of the hold on Heliyon, but a set of lessons with relevance beyond a single journal.

Due process at scale: why time matters

From the outset, a deliberate decision was made to prioritise a comprehensive and methodical approach. The review extended back to the earliest volumes of the journal, resulting in an 18-month process - longer than initially anticipated, but necessary to ensure consistency and fairness.

“Ensuring we took this time was essential to ensure that every decision, especially retractions, was grounded in robust evidence and fair process,” explains Sarah Jenkins, Director, Research Integrity & Publishing Ethics at Elsevier.

Reviewing such a large body of content required more than additional resource. It necessitated the development of new infrastructure to track article-level decisions, manage correspondence with authors, address legal considerations, and integrate with publication workflows.

This combination of technology and editorial expertise was essential to maintaining a consistent and rigorous approach, reflecting Elsevier’s ongoing commitment to safeguarding research integrity. Importantly, this process confirmed the integrity of the vast majority of Heliyon content, enabling researchers to continue using the journal’s published articles with confidence.

Beyond correction: strengthening prevention

The audit demonstrated quite how much the landscape has evolved. Systematic manipulation, whether through paper mills or other coordinated behaviours, has become more sophisticated, challenging traditional editorial safeguards.

In response, Elsevier has continued to strengthen its focus on pre-publication checks. Enhanced screening processes now combine automated tools such as Check Integrity with expert human oversight and manual checks, allowing potential issues to be identified earlier in the workflow. Signals from manuscript screening systems are assessed alongside editorial judgement, supporting more informed decision-making. This shift reflects a broader objective: not only addressing historical issues but strengthening processes to prevent future occurrences.

Not perfect, but controlled

“This isn’t a case of achieving perfection; the challenges will persist and evolve. It’s about having the systems in place to detect and act on issues quickly and transparently,” explains Sarah.

The systems in place strengthen Elsevier’s ability to detect concerns, conduct thorough investigations, and act transparently, with the support of a dedicated Research Integrity and Publishing Ethics team which has grown significantly over the last three years. Due process includes engaging with authors, and ensuring they have the opportunity to respond, even when reviews are conducted at scale. This is one of the reasons a thorough investigation can take longer than people might expect.

Our research integrity work across our journals is ongoing and reflects a continuous process of monitoring and improvement to ensure adherence to policies, which are developed alongside COPE guidelinesopens in new tab/window.

Rethinking scale: when established approaches aren’t quite sufficient

Reviewing over 40,000 articles isn’t just a bigger version of business as usual; it required a different way of working.

This was the first time an investigation had been conducted at this scale, requiring a fundamental shift in approach. Alongside editorial expertise, teams deployed technologies such as image analysis and duplication detection tools to inform the final decisions made by humans. These are embedded in standard integrity workflows, but a new part of the process was to develop a whole system to monitor progress and ensure consistency. This can now be redeployed for other investigations.

Identifying patterns

As the review progressed, it became clear that isolated cases were only part of the picture. Patterns across submissions, authorship, and content provided important insights into broader risks. Recognising these patterns informed both the audit process and the subsequent strengthening of editorial controls, reinforcing the importance of a systematic and data-informed approach.

Collaboration across the research ecosystem

Safeguarding the scholarly record is a shared responsibility, and this review highlighted the importance of collaboration. This reflects a broader principle; maintaining trust in the scholarly record requires coordination across publishers, indexers, editors, and the research community. Signals from the community, including those raised through platforms such as PubPeer, continue to play an important role in identifying potential concerns.

“We’re operating in an environment where concerns are increasingly visible to the learned eye. That scrutiny is valuable, as it helps surface potential issues within a large and growing body of content,” says Sarah Jenkins.

A changing risk landscape

She adds that “the reality is that the risk landscape has evolved faster than traditional editorial systems.” The factors underlying this audit are not unique to a single journal. Growth in submissions, increased publication pressure, and more sophisticated forms of manipulation have reshaped the publishing environment. In some cases, existing controls required strengthening, and in others, new data driven insights empowered editorial teams to make fully informed decisions. This review provided an opportunity to address those challenges in a structured and systematic way.

Looking ahead: implications for the wider industry

Many of the lessons from this process have broader application. Strengthening pre-publication checks, investing in more detection technologies, and increasing collaboration across organisations are all part of a wider shift in how research integrity is supported.

This work will continue. While issues may still arise, the systems, processes, and partnerships now in place provide a stronger foundation for identifying and addressing them.

Building trust through transparency

Sarah makes an important distinction that “trust in science depends not only on rigorous efforts to avoid problems, but also on how transparently and effectively they are addressed when they occur.” The audit, which resulted in corrections affecting approximately 3% of content published in Heliyon across 12 years of publication, demonstrates that maintaining trust in the scholarly record depends on rigour, transparency, and a willingness to address challenges directly.

At scale, this requires not only sustained effort, but ongoing evolution.