DA-DA-DA DAAAAAA! There’s a tradition in the world of classical music that, from Beethoven onwards, a composer’s fifth symphony represents a sort of coming of age, a distillation of all that is most unique in his or her work. After some appropriately portentous fanfares – the development and consultation process has taken over two years – the usage standards body COUNTER officially launched its fifth Code of Practice on January 1, 2019.
As with any mature work, COP5, as it is known, builds on years of experience. COUNTER has been providing librarians and vendors with shared protocols for electronic resource usage reporting since 2003, and the consistency and clarity of the new Code of Practice reflects this long history as much as the changing information environment in which we now find ourselves. It is also the product of collaboration on a grand scale. As Lorraine Estelle, COUNTER’s Project Director, explained to us, "The development of the new COUNTER Code of Practice was undertaken by expert volunteers, who formed the Technical Sub-Group. Members of this group are librarians, publishers, vendors and other service providers in the area of scholarly communication. The group’s objective was to seek the balance between addressing changing needs and reducing the complexity of the Code of Practice to ensure that all publishers and content providers can achieve compliance. The Technical Sub-Group devoted hundreds of hours to the design and development of the new release."
The resulting release embodies COUNTER’s hallmark drive towards inclusiveness and objectivity. To take one prominent example, the additional full text usage metric, Unique Item Requests, facilitates comparison between vendors and makes it easier for smaller publishers to comply with COUNTER standards. It does this by offering a simplified counting mechanism that consolidates HTML and PDF usage. Currently, when a user lands on the HTML full text version of an article and also downloads the PDF version of the same article, two article requests are recorded. While this will continue to count as two Total Item Requests, the new metric will now also record this activity as one Unique Item Request.
The importance of inclusiveness
As Jean Shipman, Elsevier’s Vice President, Global Library Relations notes, this is both a simplification and a sophistication. “It’s a simplification because it conflates both formats into a single number, but it’s also a sophistication because it provides libraries with another core usage metric – two different ways of looking at the same thing – enriching the types of analysis they can perform.” I wanted to talk to Jean about COP5 because, within Elsevier at least, she seems uniquely well qualified to assess its possible impact, having served with distinction as both a librarian and an information provider. She highlights inclusiveness as a key theme of the release, not just because of the drive to increase participation among vendors, but because of the coverage of aspects of journal and book value that haven’t previously been tracked. For example, the Code includes a new Item Investigations metric that measures the value of content interactions beyond the full text (e.g. use of abstracts, videos, research data) – while Distributed Usage Logging (DUL) provides a framework for publishers to capture the usage of DOI-identified content items that occurs on other websites. “You can get better idea of the use of the article with DUL, because it’s more than just one platform count, it’s every platform count,” Jean enthuses.
A new perspective on your collections
At this point it’s worth mentioning that Elsevier is a founding member of COUNTER and with the other key stakeholders in the Technical Sub-Group helped to develop and define COP5. “This doesn’t mean that Elsevier and other vendors have dominated the process,” says Jean. “What you still see is a group of stakeholders – librarians and publishers – working together to serve the best interests of the community as a whole. It’s all about reconciling different points of view to find the best available solution”, she adds, echoing Lorraine Estelle’s comments.
This seems like a good cue to talk about the way the new Unique Item Requests metric will complement the current Total Item Requests, which is also being retained. Jean tells me that “librarians are basically getting a new perspective on their collections”, but I wonder how much they actually need this. Couldn’t it be argued that one simplified content usage metric is enough and that anything else just risks creating confusion? Jean smiles and reflects that some of her former colleagues might once have felt the same way about the Impact Factor as a measurement of journal prestige until, useful as it is, new metrics became available that did a better job of addressing the disparities between different scientific disciplines, or that covered areas beyond citations, like shares or media mentions.
“Something similar is true here,” she avers. “It’s still important to look at HTML and PDF downloads separately because they represent distinct use cases”. What might those use cases be?, I wonder. “Well,” says Jean, “a lot of publishers, Elsevier included, are enriching their HTML pages with new features – recommender systems, interactive maps, 3D viewers, etc. – that are designed to help researchers work smarter, so when you look at your Total Item counts you’re also looking at a proxy for the value of the platform your articles are based on. You don’t get that with Unique Item counting, but both approaches are useful if you’re evaluating collections, or if you’re reporting upwards about the value the library provides.”
Driven by the continuous need to assess, report and act on both collection and library value, librarians have been employing a growing range of metrics, not all of them usage-based. In a recent Library Connect article Leo Appleton, author of Libraries and Key Performance Indicators, noted that while “data about usage quantity can show, to some extent, how much a library’s services and resources have been drawn upon ... usage is not synonymous with value or benefits to users, even though that is a common perception.” Indeed no single metric, however ingenious, can be wholly synonymous with “value,” which is a necessarily diffuse idea based around the benefits derived by different stakeholder groups. With this in mind, Appleton goes on to propose a suite of performance indicators that address specific performance outcomes. For example, for the outcome “Percentage of an institution’s research outputs regarded as world class,” he proposes examining the percentage increase of articles and research papers submitted to an institutional repository, as well as the number and percentage increase of citations of items contained within that repository.
Information overload via the back door?
Perceptively, Appleton observes that “there is no internationally agreed upon or tested method for assessing the different aspects of library outcomes”. In the wake of COP5 it would be easy to imagine COUNTER, or possibly an offshoot initiative, moving towards the development of such a standard. The challenge would be reconciling the need for clarity and consensus with the sheer range of metrics now on offer. Elsevier, for example, provides a “basket” of citation, readership and media impact metrics, as well as free tools like Elsevier Product Insight for Customers (E-PIC), designed to give institutional customers an informed overview of their engagement with its content. With other information providers rapidly following suit, it can sometimes feel that this is merely information overload via the back door. Are there simply too many metrics?
Jean tells me that there are always going to be those who pine for a return to a simpler life, in librarianship or any other profession (I’m beginning to think that I might be one of them), but that the growth in the number of indicators now available to the library really represents an enormous opportunity. “In the context of budget management, choosing the right metrics can help you evaluate the performance of your holdings more accurately than ever before. They can also provide insights into the library’s own ROI that were not previously available – and give you a way of talking to managers in a language they readily understand.”
Custodians of the metrics
All of which has significant implications for the way academic librarianship is evolving. Jean sees librarians becoming “custodians of the metrics,” which basically means “making informed choices about the indicators being used, overseeing the quality and consistency of the data that underpins them, and reporting clearly to the organization as a whole”. “Just as researchers have had to become more metrics savvy,” she adds, “so those who support them need to learn how to use these tools more effectively.”
This is also true for areas that fall outside the librarian’s traditional collection management remit. The open access attribute in COP5 allows librarians to more easily evaluate the role played by freely accessible content in the mix of resources offered or endorsed by the library. At the same time, COUNTER is already working on a separate Code of Practice for research data, taking COP5 as a starting point, although it is unclear whether the two approaches will be integrated in the longer term.
By the throat
There now is quite an extensive literature on the changing role of the academic librarian, much of it dating from the last decade. While commentators agree about the process of change, there is surprisingly little consensus about the end result, with the librarians of the future presented variously as information managers, teachers, de facto researchers, de facto publishers and bibliometricians. Through all of these persona, it’s hard not to discern a certain quantification of the job, mirroring trends across the university as a whole. While it’s tempting to see the love of knowledge and learning – the inclinations that have steered so many along a library-based career path – vanishing in a blizzard of statistics, this is an unduly submissive point of view. As Jean notes “that will only be the case if libraries meet these changes passively, allowing others to call the metrics shots.” Proactivity is the order of the day.
From this perspective, the long-awaited arrival of COP5 in January represents both a safeguard and an opportunity. Or, as Lorraine Estelle described it to us, "The new Code of Practice is consistent, unambiguous, and flexible. Flexibility is important because it means that the Code of Practice can be adapted and extended as digital publishing changes over the years. The future-proofing built into Release 5 means that it can be subject to a continuous maintenance process, changing over time to stay relevant, instead of being replaced by Release 6."
Like a good fifth symphony, COP5 is built to last – drawing on past experience and a growing frame of reference to deliver a technical and strategic leap forward. Beethoven once remarked that he wanted to “seize fate by the throat” and there are many in the library community who would now share these sentiments.
In this recorded webinar, presenters including COUNTER's Project Director Lorraine Estelle, highlighted key points in the development of the COUNTER Code of Practice Release 5, touching on the development of the new release, introduction of new metric types and a deep dive into the new reports. Step-by-step instructions of the new reports were presented to the audience along with common use cases and scenarios. Watch the recorded webinar and access the slides, Q&A from the audience, and more.
comments powered by Disqus