Executable papers in computer science go live on ScienceDirect
Team seeks feedback on pilot of Collage Authoring Environment
By Hylke Koers, PhD, Ann Gabriel and Rebecca Capone Posted on 13 May 2013
Research articles have many purposes, but arguably the main motivation is to share important research results and the experimental parameters that produced these results.
Readers, however, are limited in the extent to which they can replicate these results in their own labs, relying not only on the article but also on any supplementary material the author may provide.
This situation highlights a limitation in the scholarly communication process: that most experiments are difficult or impossible to reproduce, and that the archival research article doesn’t actually archive how the results came to be. Authors might supply their underlying data and computer code, but they usually don’t. And if they do, it is often hosted on personal or institutional homepages, easily disconnected from the article itself.
With the Article of the Future, Elsevier created a more dynamic online article format that enables researchers to capture and disseminate their work more fully. It includes domain-specific types of data and other research output that help readers gain deeper insight. This article concept provides a natural home for a solution to improve the reproducibility of computational work.
In 2011, Elsevier launched the Executable Papers Grand Challenge, looking to the computer science community to address the question of how to reproduce computational results within the confines of the research article. The first-place winner, the Collage Authoring Environment, is now launching a pilot special issue on 3D Object Retrieval with the journal Computers & Graphics, showcasing executable research results in articles published on ScienceDirect.
In close consultation with the winning Collage team, in particular Eryk Ciepiela, Daniel Harezlak, and Piotr Nowakowski (ACC Cyfronet AGH, Poland), Elsevier’s Journal and Content Technology group mapped content synergies in order to select an appropriate journal to take a pilot project forward. Computers and Graphics was identified as a particularly viable journal due to prior work in transforming visual images through computer code, which could potentially result in a compelling outcome.
Editor-in-Chief Joaquim Jorge and his guest editors took up the challenge. Dr. Jorge is Professor at Instituto Superior Técnico, Departamento de Engenharia Informática, Portugal, and guest editors were Dr. Michela Spagnuolo, Senior Researcher at Istituto di Matematica Applicata e Tecnologie Informatiche, Italy, and Dr. Remco Veltkamp, Professor at Universiteit Utrecht in the Netherlands.
Vetting articles authored in this new format was time-intensive and required close collaboration with authors, reviewers and Collage developers to ensure full utilization of the Collage system during the submission and review process. Meanwhile, Gail Rodney and Lisa Gordon, the publisher and journal manager for Computers and Graphics, made sure any necessary exceptions to the workflow were tracked closely and that the editors and developers were able to work within the submission and publication deadlines.Last week, teams from Collage and Elsevier demonstrated their system at Eurographics 2013, the 34th annual conference of the European Association for Computer Graphics, in Girona, Spain.
How does the Collage Executable Paper work?
Collage allows authors to create a numerical experiment that belongs with their article. This experiment contains one or more pieces of computer code, together with all the data sets that may be necessary to execute the code.But the Collage system offers more than just a way to collect code and data; it lets authors describe the entire computational workflow from input to output by specifying how all the elements depend on each other. For example, one data set may be the input for a piece of computer code, which then outputs another data set. By capturing the full methodology, the Collage system ensures that results are fully reproducible.
Collage supports several leading computer languages, including C, C++, Perl, Python, and R. Authors can also include compiled code that is executed through a script. This flexibility allows author to fully tailor their experiments for optimal presentation, and also lets authors decide which parameters and pieces of code may be changed by readers during their session.
When the article is published on ScienceDirect, the numerical experiment is displayed alongside the text, in the right-hand pane. This provides for a fully integrated, seamless user experience that allows readers to explore the computational elements while reading the article. Readers can re-run code, change parameters, and even apply computer code to their own data – all from within the Collage application on ScienceDirect. It even includes built-in tools to visualize special kinds of data such as 3D models.
[note color="#f1f9fc" position="center" width=800 margin=10]
Tell them what you think
The team behind the Executable Paper Grand Challenge invites you to check out the Executable Paper website and let them what you think in the comment section below. More details on the Collage Authoring Environment are available on the Collage website and the Computers and Graphics journal website.[/note]
Collage is managed by Eryk Ciepiela with the support of Daniel Harężlak and the assistance of the DICE team: Jan Meizner, Marek Kasztelnik and Grzegorz Dyk, as well as Piotr Nowakowski and the leader of the DICE group, Dr. Marian Bubak, a professor at AGH University of Science and Technology in Krakow and the Informatics Institute at the University of Amsterdam.
Ann Gabriel is the Publishing Director for Elsevier's computer science journals. She joined Elsevier from Cambridge University Press as part of the ScienceDirect product team and later managed Competitive Intelligence for the strategy group; she became part of STM Journals in 2009. She is based in New York.
Rebecca Capone is a journal publisher Elsevier's Computer Science group, responsible for the Theoretical Computer Science portfolio. She joined Elsevier in 2008, previously managing Biotechnology journals at Wiley-Blackwell. She is based in New York.
Dr. Hylke Koers is a Content Innovation Manager at Elsevier, where he is responsible for a range of projects to enhance the online article format. Part of Elsevier’s Article of the Future program, this includes improved online presentation, better support and visualization of digital content, and contextualization of the article by linking with data repositories and other sources of trusted scientific content on the web. Before joining Elsevier in 2010, Hylke received a PhD in theoretical astrophysics from the University of Amsterdam and served as a postdoctoral research associate at the Université Libre de Bruxelles. He is based in Amsterdam.