Seeking certainty: why the results of 50 landmark cancer papers are being examined

The Reproducibility Initiative gets $1.3 million for a project that could enable the development of future therapies

In the scientific community, there's a growing concern that many basic research findings can't be independently replicated. These concerns grew when scientists at Bayer and Amgen reported that they could not replicate many landmark findings in biology, and were magnified by the companies' refusal to disclose which papers weren't replicable.

In response, the Center for Open Science is designating $1.3 million from the Laura and John Arnold Foundation to the Reproducibility Initiative to independently validate 50 landmark cancer biology studies. The 50 chosen studies are among the highest impact studies in the field from 2010 to 2012, and systematic validation could be crucial to developing future cancer drugs. Without this validation, many labs will spend time and money in vain, trying to extend the findings or apply them to their problem.

Elizabeth Iorns, PhD"The lack of reproducibility in cancer studies is a major obstacle in the development of viable therapies to cure cancer," said Dr. Elizabeth Iorns, Co-Director of the Reproducibility Initiative. "The Reproducibility Initiative hopes to transform the scientific process by enabling researchers to verify key scientific findings and incentivizing scientific replication. The funding will be instrumental in not only verifying landmark cancer studies, but also helping to institutionalize scientific replication. "

[pullquote align="right"]The lack of reproducibility in cancer studies is a major obstacle in the development of viable therapies to cure cancer.[/pullquote]

The Reproducibility Initiative was launched by several prominent scientific journals and organizations (Science Exchange, Mendeley, PLOS and figshare) last year in response to revelations from the pharmaceutical industry that more than 70 percent of published cancer research cannot be reproduced, thus stifling the development of effective new therapies.

How the initiative will work

The Reproducibility Initiative intends to identify and reward high-quality reproducible research through independent validation of key experimental results. To that end, it has partnered with antibodies-online to independently validate thousands of commercial antibodies.

With the funding, the Reproducibility Initiative is will collaborate with the similarly named Reproducibility Project, already supported by the Center for Open Science.

Researchers interested in having a study replicated or validating their own work may submit a study to the initiative.The Reproducibility Project is a crowd-sourced effort by researchers to identify the predictors of reproducibility in a large sample of published studies in psychological science.  "The integration of these two projects is an opportunity to understand and address reproducibility challenges that are shared across scientific disciplines," said Dr. Brian Nosek, director of the Center for Open Science.

The key experimental findings from each cancer study will be replicated by experts from the Science Exchange network according to best practices for replication established by the Center for Open Science through the Center's Open Science Framework.

The number of researchers reading the replication papers – as well as the effect of the replications on the readership of the original publications – will be tracked on Mendeley's research analytics platform.

All of the ultimate publications and data will be freely available online, providing the first publicly available complete dataset of replicated biomedical research and representing a major advancement in the study of reproducibility of research.

Stuart Buck, Director of Research at the Laura and John Arnold Foundation, said the Reproducibility Initiative "may eventually serve as a model for other funding agencies and patient groups, with the ultimate goal of improving cancer treatment through more rigorous and reliable science."

The Center for Open Science

The Center for Open Science was founded by Dr. Nosek, a professor in the Department of Psychology at the University of Virginia, and Jeffrey Spies, who recently finished his PhD there, to increase openness, integrity and reproducibility of scientific research. The Center for Open Science will administer the funds as part of their mission to incentivize the replication of important scientific studies. Core funding for the Center for Open Science and its replication initiatives comes from the Laura and John Arnold Foundation (, which funds projects to promote transformational change.[divider]

The Author

William Gunn, PhDDr. William Gunn (@mrgunn) is Head of Academic Outreach for Mendeley, a research management tool for collaboration and discovery.

He is Co-Director of the Reproducibility Initiative, which he writes about here.

Dr. Gunn attended Tulane University as a Louisiana Board of Regents Fellow, receiving his PhD in biomedical science from the Center for Gene Therapy in 2008. His research involved dissecting the molecular mechanism of bone metastasis in multiple myeloma and resulted in a novel treatment approach employing mesenchymal stem cells, the body's own reparative forces.

Frustrated with the inefficiencies of the modern research process, he left academia and established the biology program at Genalyte, a novel diagnostics startup.

At Mendeley, he works to make research more impactful and reproducible and is an expert on altmetrics, reproducibility and open access.

comments powered by Disqus

1 Archived Comment

Jessie Goodpasture November 6, 2013 at 11:45 pm

It's about time!


Related Stories