How your feedback makes for better research tools

As they use ScienceDirect, researchers tell us what they need; here’s how we turn feedback into action

feedback image

When you use ScienceDirect, you can let us know which features are essential, what works and what doesn’t. You may even fill out a survey or take part in a user workshop. So what happens to all the information and feedback we receive from you and other users?

Bastiaan de Jong“User feedback is the starting point for continuous improvements,” says Bastiaan de Jong, Director of Product Management at Elsevier. “ScienceDirect is one of our flagship offerings for researchers, and we simply can’t afford to miss the features researchers want and need to do their important work.”

For example, a recent survey among German researchers revealed a great appreciation for the value of ScienceDirect. While it’s tempting to celebrate this as a direct result of the quality of the platform, the truth is that it’s the researchers themselves that have helped ScienceDirect reach that value level, De Jong explained:

We need to give researchers the best possible platform, which is why we have a massive team of product experts and technicians in place who constantly interact with ScienceDirect users to get an in-depth understanding of how they use the platform, how their needs are evolving and what we need to develop to cater for those needs.

Combining qualitative and quantitative feedback

Rose L’HuillierUser feedback is gathered both in quantitative and qualitative ways. As Director of Product Management Rose L’ Huillier explains:

The tracking of online user behavior brings us valuable quantitative data that tells us which features they frequently use, which less so, and what their online journey looks like. We also make use of A/B testing to test the value of new product features, by making two versions of a page available to a small percentage of users and track which they find more valuable. We combine these insights with the results of qualitative user research, conducted via one-on-one conversations with users, online user commentary, and a tool called UserZoom, which gives researchers an ‘assignment’ without explanation, and we then measure how successful they are in completing the assignment.

Value beyond content: leveraging smart technology

Over 2 million researchers visit ScienceDirect daily, attracted by access to about 16 percent of the world’s highest quality scientific content. But content is just part of the story. ScienceDirect is a technology platform designed to improve and accelerate the way researchers search, discover, read, understand and share scholarly information, making their workflow more efficient and saving them much time. It enables smarter research.

“That’s where our focus lies,” says De Jong. “We are continuously applying the latest technology to enable researchers to become ‘engagers’ with the platform rather than readers of information.”

Building on our article recommender service, for instance, his team created a personalized version of this feature – an email-based tool that recommends articles to researchers based on their reading behavior. Their data show that since its launch this summer, the service has already had a measurable impact on the time researchers take to find the information they need.

“Over the past months, we’ve been analyzing user feedback to further improve the recommendations and how the user interacts with them,” De Jong says. “As of mid-November, users also have the opportunity to see personalized recommendations on the ScienceDirect site itself.”

More is not always better

While technological developments are well-appreciated and used, sometimes things just need to be kept simple. “ScienceDirect needs to be intuitive to use,” L’Huillier explains. “As soon as we need to clarify or explain how something works, we’re on the wrong path. In product development, it’s easy to get carried away with the almost unlimited ways in which we can make our product more exciting. The power of researcher feedback is that it forces us to keep both feet firmly on the ground and focus on what they need, not what we think they may need.”

A surprising revelation about the impact of ScienceDirect

Having a structured feedback mechanism in place is essential to keeping ScienceDirect at the level researchers need it to be. Sometimes, however, the most impactful feedback comes through unexpected channels, as De Jong relates:

About a year ago, my plane was delayed at Johannesburg’s international airport, so I had some time to kill. I sat down and struck up a conversation with the lady sitting next to me, who was travelling with her young son. She asked me what I did for a living, and I told her about my involvement with ScienceDirect. Then she thanked me. I didn’t understand why until she told me that her boy had been diagnosed with a chronic disease and his doctors had decided to give up on further treatment. Unable to accept this, she told me how she had turned to ScienceDirect to find the latest research papers about her son’s disease, and to identify the world’s top medical professionals in that field.

De Jong said the woman managed to get in contact with a leading medical professional in the United States:

They agreed to look into ways to help the young boy, with amazing life-improving results. This story made me realize that we’re not just working on a product; we’re working on sharing information that improves and saves lives.

German researchers rate ScienceDirect #1

Earlier this year, Elsevier’s research team conducted a survey* of German researchers, inquiring about their perception of the value ScienceDirect in their work. Here are the main findings:

  • ScienceDirect was rated as the most used and important source of full-text services for scientific information.
  • 85% said ScienceDirect helps them discover new research so that they are fully informed.
  • 72% of the ScienceDirect users who responded would recommend ScienceDirect to a friend or a colleague.
  • 92% agreed that the scientific quality of the ScienceDirect content is good.
  • 93% agreed that ScienceDirect is easy to use.

*Survey of researchers in Germany ending January 2017, conducted by Elsevier’s research team. 266 responses, including 127 ScienceDirect users. Overall this represents a margin of error of ±5.0% at 90% confidence levels. For ScienceDirect scores this represents a margin of error of ±7.3% at 90% confidence levels. This is considering the total population of the German research community (~586,000 individuals / ~388,000 FTE, according to latest available OECD figures (2015)). To reduce error margin to ±2%, ~2,000 responses needed. This study was carried out under the terms of the Market Research Society Code of Conduct.

Enriching content

Over the years, ScienceDirect has been proactive in developing new features in the areas of article promotion and data visualization. For example, AudioSlides allow researchers to briefly describe their article in their own words and promote it through social media. With the drive towards reproducible research and better data availability, it’s becoming increasingly important for researchers to visualize data and make it interactive in the article. Our integrated research data management functionalities support our commitment to transparency and reproducibility in science, facilitating easy access to the data underpinning research.


Written by

Harald Boersma

Written by

Harald Boersma

Harald Boersma is Director of Global Corporate Relations at Elsevier. He is based in Amsterdam.


comments powered by Disqus