How a software journal is boosting scientific discovery
SoftwareX helps developers improve their processes and programs – and get credit for their code
By Tobias Wesselius Posted on 22 February 2016
But it does much more than that. Researchers use software across all academic disciplines to do everything from model the brain to discover new sub-atomic particles.
To acknowledge the impact software has on today’s research and highlight the importance of software developers in science, Elsevier launched a new open access journal last year called SoftwareX. It was first of its kind, with short articles that describe software and link to the code hosted on GitHub. This means that the article and the code itself are both citable.
Since then, the journal has received great feedback and more than 100 submissions, and so far three volumes have been published on ScienceDirect. It has also attracted attention from the publishing industry. This month, SoftwareX was one of eight Elsevier books and journals to win PROSE Awards. It received the PROSE Award for Innovation in Journal Publishing – given at the discretion of the Professional & Scholarly Publishing Association of American Publishers only when the judges “overwhelmingly feel a journal is very cutting edge.” This is the first time it’s been awarded in seven years.
“We’re really excited to receive this award as it highlights the innovative approach we’re taking to give software and the developers who create it the recognition they deserve,” said Dr. Eleonora Presani, the journal’s publisher. Dr. Presani is familiar with their plight, having spent much of her time coding in her previous role as particle physicist at CERN.
Software engineer’s eye view
It’s often the behind-the-scenes work of software engineers that makes the big difference to a research study. One such example is SAGA – a community standard, developed by a community of engineers and scientists around the world that simplifies access to supercomputing applications.
Dr. Shantenu Jha, Associate Professor in the Department of Electrical and Computer Engineering at Rutgers University, and his colleagues started developing SAGA a decade ago and have been its custodians ever since.
The community took three or four years to develop the standard, which involved gathering feedback, making changes and getting formal approval. There are various ways to put a program to use once it’s ready, and SAGA also provides instructions on how to implement the software. Dr. Jha explained:
Software engineers and computer scientists think of the world in layers: end user, middle layer, system. When it comes to high-performance and distributed computing systems, there’s a huge diversity in how things are done. That introduces complexity at the systems level, and all the tools at the upper ‘user’ level suffer from this diversity. SAGA addresses this issue and provides a simpler interface for users to interact with the systems.
All sorts of researchers use SAGA in their work. Scientists who study particle physics, particularly at the KEK laboratory in Japan, and the ATLAS experiment at CERN that’s investigating the Higgs boson and the physics beyond it, need to use high performance computing to carry out the calculations needed for their work. Individual researchers also use SAGA.
“It always feels good to have people use your product – it validates your effort and intellectual enterprise,” said Prof. Jha. “It’s good when large projects build their needs around your product, but it’s not always obvious why. At times, individual users may have a more rational approach to it. Why do people use the tools that they use?”
This is a conceptual question that SoftwareX may go some way to answering, Prof. Jha said.
Understanding why people use the tools they use is fundamental to the health of software sustainability. If we don’t know why people use X vs. Y we can’t cater to those needs. By publishing in SoftwareX, researchers can share with us how they’re applying the software they use.
The publishing experience
Prof. Jha and his colleagues published one of the first articles in SoftwareX in September. It describes the implementation of SAGA and how they have employed good software engineering and community practices to provide the community with a sustainable and open source product. Publishing in SoftwareX helped the team share their approach, Prof. Jha said:
We think of ourselves as custodians of the standard. We’re constantly addressing feedback and user requirements; we resolve more than 100 issues a year from members of the community. Our article is about how we’ve designed the code and the processes we used to achieve this.
It’s important to share the details of the processes, in part because they contributed to the sustainability of the program. SAGA, as a concept and a product, has survived a decade, but there’s no single funding mechanism that can be sustained for so long. Because Prof. Jha and his colleagues adopted a community approach to the development of SAGA, it has had a significant impact on research over the years.
So finding a place to share that story was important. For Prof. Jha, SoftwareX provides a fresh perspective on the importance of software to the scientific endeavor, making it an easy decision to submit. He explained:
Once I understood what the journal was trying to achieve it was a no-brainer. We’d been wanting to write about the processes we’d employed for a long time and hadn’t gotten around to it. We wanted to formally document how we’ve developed SAGA; the richness is in the detail.
The article was published in the very first volume of the journal, so there were a few bumps in the road, but this didn’t have a negative impact – it was to be expected, Prof. Jha said:
The first edition of any journal has some issues. Despite that it wasn’t a pain at all – the staff and editors were very supportive. By formalizing our approach in this way, we hope that we can reach the community, get good feedback, and encourage users to come back and help us improve the program. It will have a positive effect on SAGA.
This is just the beginning for SoftwareX. As more researchers and software engineers share their code and the processes they went through to develop their programs, the knowledge base will be built up and this in turn will contribute to lifting the quality of processes and the resulting software. Prof. Jha concluded:
SoftwareX emphasizes the importance of quality engineering and careful consideration in the software development process, which is so central to the scientific enterprise. I could wax lyrical about the goals and objectives that have not been met because of limitations of the software on which they were predicated. Inverting that, why don’t we ask what’s the problem? We don’t appreciate the importance of the process. By trying to do that, and eventually lift the tide of quality as a whole, the journal is providing a service to the community.
Read the study
The following article is open access:
Andre Merzky et al: “SAGA: A standardized access layer to heterogeneous Distributed Computing Infrastructure,” SoftwareX (September 2015)
Dr. Shantenu Jha is an Associate Professor of Computer Engineering at Rutgers University. His research interests lie at the triple point of high-performance distributed computing, computational science and cyberinfrastructure. Shantenu leads the RADICAL-Cybertools project, which is a suite of standards-driven and abstractions-based tools used to support large-scale science and engineering applications. He has won several awards at leading supercomputing conferences for the application of high-performance distributed computing to computational science. His research is supported by the US DOE and NSF, including the NSF CAREER Award. He was appointed a Rutgers Chancellor Scholar in 2015. Away from work, Jha indulges in middle-distance running and “the quest for the perfect bad joke!”
SoftwareX is an open access multidisciplinary journal. It aims to acknowledge the impact of software on today's research practice and on new scientific discoveries in almost all research domains. The journal supports the peer review and publication of research software, making it citable and available for reuse and supporting the careers of software developers. Submissions consist of two parts: a short descriptive paper and an open source software distribution with support material.
SoftwareX aims to promote, share and validate all software that has an impact on research. Large comprehensive programs like SAGA that are part of a collaboration effort represent the tip of the iceberg; it is also important to recognize the value of smaller individual programs, such as anyFish 2.0 – an open-source software platform for generating and sharing animated fish models to study behavior.
Elsevier Connect Contributor
Tobias Wesselius is the marketing communications manager for SoftwareX and is also responsible for marketing a portfolio of journals in the environmental sciences. He has a background in psychology and communications, and is planning a cycling trip through England this summer (tips are always welcome).