Bringing 3D visualization to online research articles
Various journals on ScienceDirect now have tools to view scientific models in 3D
By Elena Zudilova-Seinstra, PhD, and Julien Jomier Posted on 12 December 2013
With researchers increasingly generating and referencing 3D data in their papers, the ability to accurately visualize that data in online publications has become extremely important.
In many scientific publications, 3D models are "flattened" into static 2D images, significantly reducing the value of the author's analysis and the reader's ability to interact with the content.
With new software Elsevier has developed with Kitware SAS, a company known for creating advanced visualization technologies, readers can examine 3D objects as part of the reading process, and authors can display and share their 3D data.
Four types of 3D visualization viewers are available in various Elsevier journals on ScienceDirect. The functionality of each viewer has been carefully thought out to support domain-specific needs.
- The 3D molecular viewer
- The 3D archaeological viewer
- The 3D neuroimaging viewer
- The generic U3D viewer
The overarching goal of the project is to enable Elsevier authors to showcase their 3D data, and to provide ScienceDirect users with a means to view and interact with these author-provided small to massive 3D datasets on a large number of devices with no additional plug-in required. These devices include smartphones, tablets, laptops and desktop computers.
How 3D visualization works
The underlying 3D visualization infrastructure (Figure 1) combines local (WebGL) and remote (ParaViewWeb) rendering techniques. The web browser and the size of the 3D data is analyzed to ensure that users are given optimal support and are able to interact with very big 3D models in real time.
All viewers are displayed next to the relevant articles, in the right-hand pane. Viewers can also be opened in a full-screen mode.
Authors upload the 3D models and data as supplementary files when submitting their paper; each 3D data file is of a format that is commonly used in the specific discipline (no additional work is needed by the author). After the article is accepted, each 3D data file is pre-processed to ensure the most optimal default view of the model inside the viewer.
3D molecular viewer
The 3D molecular viewer shown in Figure 2 visualizes proteins, ribosomes, chemical compounds, crystallographic data and related structures and supports very popular PDB, PSE and MOL/MOL2 data formats. It allows the 3D molecular models to be explored using the two most common visualization techniques: "ribbons" and "balls-and-sticks."
Readers can select different models via the thumbnail menu at the top of the viewer, zoom in on, rotate and pan the model, change display settings, and download original data files. It is also possible to view molecular models in the 3D stereo mode using anaglyph glasses. This functionality is currently available for 20 Elsevier journals in Life Sciences.
3D archeological viewer
The 3D archaeological viewer visualizes models submitted in widely applicable PLY and OBJ data formats. The surface rendering technique is applied to display 3D data (including texture and material properties). The reader can scale a 3D model of the archaeological artifact described in the article, rotate it and pan using a mouse; use the auto-rotate option; change display settings; and download original data files. Two examples of 3D archaeological models visualized with the viewer can be found in Figure 3.
This viewer was developed to support the new Elsevier Journal Digital Applications in Archaeology and Cultural Heritage (DAACH). DAACH gives scientists the opportunity to publish their models online with full interactivity so that readers can explore them at will. The journal is unique in that its focus is on the application of 3D modeling to cultural heritage.
3D neuroimaging viewer
The 3D neuroimaging viewer shown in Figure 4 visualizes analyzed neuroimaging data in NIfTI format, widely used in neuroscience. This viewer enables the reader to browse through available neuroimaging datasets; zoom in on, rotate and pan the 3D brain reconstruction; cut through the volume; change opacity and color mapping; switch between 3D and 2D projected views and download the data. It has been developed in close collaboration with neuroscientists and supports the most important capabilities as selected in an online survey of researchers with more than 200 respondents. This functionality is currently available for 26 Elsevier journals in neuroscience, psychiatry, neurology and psychology.
The generic U3D viewer shown in Figure 4 is the latest development related to this project. It enables the visualization of U3D models, which are traditionally embedded in 3D PDF files. Instead of creating huge PDFs, authors can now upload their U3D models for display alongside their online articles on ScienceDirect. The reader can browse through all available U3D models; zoom in on, rotate and pan them, hide/unhide model elements, view the name of each model element one-by-one, change display settings, switch between the "Surface" and "Wireframe" display modes, explore 3D models in stereo with anaglyph glasses, and download original data files. This content innovation is broadly applicable across various journals.
The hybrid visualization platform and domain-specific modules developed in the joint Elsevier-Kitware project provide the basis for ongoing enrichment of content in Elsevier's journals. Offering interactive visualization tools and techniques that allow for better communication and more accurate interpretation and analysis of research data is one of the pillars of Article of the Future, Elsevier's ongoing program to improve the format of the scientific article.[divider]
3D visualization in online articles
- 3D molecular visualization: elsevier.com/3DMolecularModels
- 3D archaeological visualization: elsevier.com/3DArchaeologicalModels
- 3D neuroimaging: elsevier.com/3DNeuroimaging
- U3D models: elsevier.com/about/content-innovation/u3d-models
Examples of 3D viewers
- Neuroimaging viewer: J.H. Cole, C.P. Boyle, A. Simmons, S. Cohen-Woods, M. Rivera, P. McGuffin, P.M. Thompson, C.H.Y. Fu, "Body mass index, but not FTO genotype or major depressive disorder, influences brain structure," Neuroscience, Volume 252, 12 November 2013, pp. 109-117.
- Archeology viewer: Gerhard W. Weber, "Another link between archaeology and anthropology: Virtual anthropology," Digital Applications in Archaeology and Cultural Heritage, Available online 13 April 2013.
- Molecular viewer: Stephen S. Chen, James R. Williamson, "Characterization of the Ribosome Biogenesis Landscape in E. coli Using Quantitative Mass Spectrometry," Journal of Molecular Biology, Volume 425, Issue 4, 22 February 2013, pp. 767-779.
- U3D viewer: B. J. Klußmann-Fricke, L. Prendini, C.S. Wirkner, "Evolutionary morphology of the hemolymph vascular system in scorpions: A character analysis," Arthropod Structure & Development, Volume 41, Issue 6, November 2012, Pages 545-560, ISSN 1467-8039.
Article of the Future
Kitware's mission is to develop and deliver cutting-edge software products and services using advanced software quality methods and technologies. Since its founding in 1998, Kitware has combined an open-source business model with state-of-the-art software engineering and world-class imaging expertise to become a leader in the creation and support of open-source software, data visualization and analysis technology. By fostering extended, collaborative communities, Kitware provides high-performance visualization, computer vision, medical imaging, data publishing and quality software process solutions to a variety of academic and government institutions and private corporations worldwide.
Here is the joint press release about the collaboration between Kitware and Elsevier.
Elsevier Connect Contributors
Dr. Elena Zudilova-Seinstra is a Content Innovation Manager for Journal & Data Solutions at Elsevier Academic and Government Research Markets. She joined Elsevier in 2010 as a Senior User Experience Specialist for the User Centered Design group. She holds a PhD in Computer Science and an MSc degree in technical engineering from the St. Petersburg State Technical University. Before joining Elsevier, she worked at the University of Amsterdam, SARA Computing and Networking Services and Corning Inc. Elena leads the 3D visualization program for Content Innovation projects at Elsevier.
Julien Jomier directs Kitware's European subsidiary in Lyon, France, (Kitware SAS), where he focuses on European business development. He joined Kitware in 2005 as technical leader and led the MIDAS project, a system for collecting, processing and distributing massive collections of data.
Julien received his BS and MS in Electrical Engineering and Information Processing in 2002 from the ESCPE-Lyon in France and an MS in Computer Science from The University of North Carolina at Chapel Hill (UNC). He worked on a variety of projects in the areas of parallel and distributed computing, mobile computing, image processing and visualization. He is responsible for collaboration with Elsevier at Kitware.