Journals, books, libraries – these were sources not only of targeted, specific research, but also of surprise discoveries: ‘eureka moments’, or instances of serendipity. With the recent boom in online information, are search engines removing serendipitous moments from the information discovery and research process? We spoke to Kevin Cohn, Director of Product Management for Atypon, about ways to ensure the serendipity of information is not lost to the Internet ether.

“To stay in business, companies like Amazon need to ensure that consumers continue to ‘discover’ new books or music. They employ ’recommendations’ and ‘related items’ functionality to keep customers aware of the newest products, in the same way that they used to with displays, bookshelves and CD racks,” Cohn explains.

If online ‘shops’ do not provide consumers with the capacity to browse and search for items that are related to their areas of interest, they simply will not buy. While scholars and researchers are likely to make the effort to search for information – after all, it is part of their job – tools that help lead them to ‘related items’ can be useful to them, too. Amazon is in the business of product discovery, while publishers are in the business of information discovery. Is there any real difference between the two?

According to Cohn, the difference is negligible. “Scholarly publishing has a lot to learn from consumer publishing,” he says. “The most competitive publishers should treat scholars and researchers as ‘consumers’ if they want to retain their business – especially in the current economic climate. And that means making information discovery easier for researchers.
Serendipity is one form of information discovery in print, but it can be greatly enhanced in online media by employing better technology.”

Print vs. online
In Cohn’s opinion, the number of serendipitous moments that actually occur when people use print media is far lower than is perceived. Books and journals require the reader to flick through thousands of pages, which means there’s a low ‘hit rate’ when it comes to accidental discoveries. While there may be serendipitous moments, these are few and far between. “As a medium, print is very inefficient,” claims Cohn. “Online media can work far more efficiently because the response can differ based on the user’s actions. That means you don’t see everything out there on your computer screen – you just see the ‘cloud’ of information that your actions have generated.”

One of the problems with searching and browsing online is that it is undirected. It is impossible to uncover the rich and varied ‘random’ content available without some specific, directed input from the user. And once this limitation has been imposed, does online media still deliver the possibility for creative thinking? Or does it instead limit the potential for fortuitous accidental discoveries?

While on the one hand, technology narrows down the information you receive, it may, on the other hand, be deployed to provide other resources that do not specifically match search criteria, but that are related or useful in some way. Cohn explains that, by employing advanced algorithms, technology can ‘predict’ what information users may find interesting and guide them to it, without them having to search for it specifically.

Collaborative filtering
Technologies that make use of user data to determine what information they return are known as collaborative filtering features. For example, Atypon’s collaborative filtering implementation checks the content users have viewed, and in which order, so as to predict other content that users may be interested in. Cohn claims that for Atypon, “algorithms can make this prediction process up to 40% successful. This reduces the time researchers need to spend searching for content and increases the time they can spend actually using it.”

There are many actions that can be fed into the collaborative filtering algorithm to improve a service’s recommendation functionality. A user’s past searches, their current search criteria, whether a user tends to look at abstracts or full-text articles – all these provide additional filtering possibilities.

“Google is not the be-all and end-all of search tools,” says Cohn. “And we shouldn’t leave information discovery in the hands of search engines alone – it doesn’t solve all problems. But collaborative filtering is one way of supporting information discovery.”

Return on investment
In the current economic climate, however, does it make sense to spend money on new technologies to improve these functionalities? Scholarly publishing tends to be more resilient to downward economic trends than other businesses, but the market for investment is still competitive. Sales of scholarly publishing materials are increasingly based on researchers’ usage rates. Librarians are assessing online information on a cost-per-download basis, wanting to be sure of the potential return on investment before buying subscriptions to online media resources.

Cohn explains: “This means that online publishers need to demonstrate that their content has a high value-to-cost ratio. To do this, they can either reduce their prices or increase the value of their offering – and by extension their usage – by guiding users to interesting, new and relevant content.”

While the scholarly publishing industry can benefit hugely from the technologies that consumer companies like Amazon have put in place, scholarly publishers may take longer to adapt. If Cohn’s argument is anything to go by, the technology is there: and the most forward-thinking online publishers should be investing in it if they want to stay ahead in difficult economic times.

Useful links:

  http://www.nytimes.com

  www.guardian.co.uk

  www.info.scopus.com

  www.atypon.com

The following links require subscriptions:

  IEEE Xplore

  Using collaborative filtering

  Evaluating collaborative filtering

  ScienceDirect: Algorithms & Art