Connect

Measuring what matters: library-led insight at a National Laboratory

August 8, 2025

By Susan Jenkins

How librarians at the Pacific Northwest National Laboratory redesigned publication reports to improve decision-making and strategic alignment.

Understanding research impact has become a top priority at research institutions in recent years. This is especially true for institutions with government mandates to explore solutions on behalf of national interests.

At the Pacific Northwest National Laboratory (PNNL), a U.S. Department of Energy (DOE) national laboratory focused on chemistry, earth science, and innovations that protect critical infrastructure, understanding research performance can inform major decisions about resource allocation and future scientific direction.

But translating research performance into impact isn’t easy. Mary Frances Lembo, a research librarian at PNNL with a flair for bibliometrics, frames it this way: “We had some discussion about the idea that we don’t determine our impact, impact is an external thing that others are assigning to us.” However, analyzing citation metrics and benchmarking this data against similar research institutions is a useful indicator of the reach and impact of their contributions to the scientific field. Combining bibliometric impact factors with seasoned interpretation, two library-led initiatives at PNNL have reshaped the way management engages with research impact data, enabling their decision-making and strategic planning. In both cases, the work has involved technical skill, collaboration, and a commitment to making metrics tell a clear story.

Redesigning how publication metrics are reported

The publication metrics report Mary Frances had been producing prior to 2024 was a comprehensive document that included a lot of context but was long and underused. “That report was very thorough and very detailed, but stakeholders weren’t reading it,” Mary Frances explained. “Meanwhile, some groups were going off and doing their own publication metrics without the appropriate context and guidance – that also leads to inconsistency.”

At the same time as the library was starting to consider a redesign, PNNL’s Deputy Director for Science and Technology asked for a new kind of publication reporting. “We were charged by the research office to develop a more modern and simplified metrics report that reflected performance characteristics that were of most interest to our stakeholders,” said Mary Frances.

Illustration – Side by side comparison, pages from the old and new reports

The challenge, then, was not just to modernize the report’s format but to ensure the data would tie closely to strategic thinking across the lab, which could also support a shared understanding to the concept of “impact.” “Maybe we have more publications, we have more citations, but going the step beyond, what is the impact? That’s difficult to determine,” notes Mary Frances. She knew they had to choose which metrics packed the most meaning and show them with relevant context so that those numbers bear a relationship to the visibility and uptake of the research they represent.

Working in collaboration with her manager, Miriam Blake, Mary Frances led the redesign of PNNL’s publication metrics report. Instead of a dense PDF, the new version is a compact, two-slide visual summary that’s presented quarterly to the oversight board.

Mary Frances’ long-standing interest in data visualization techniques was a great asset in the redesign. “Visualizations help make connections without having to use so many words.”

The simplified format also stirs more discussion. By sharing the report in a presentation, Miriam can offer background on the data that directly addresses the questions in the room. “It’s easier to talk about two slides rather than this long document,” Mary Frances said. “When you’re at this presentation, you can quickly be up to speed.”

By helping stakeholders develop a critical understanding of publication data, the library is actively shaping institutional insight and improving how research performance is discussed.

Going deeper with topic-based analysis

A second, more experimental initiative piloted a new approach to analyzing impact in relation to PNNL’s scientific objectives. They wanted to explore how to gain valuable insight into specific research topics that were within the larger directorate areas of the laboratory but also crossed those boundaries. They selected one key research area, chemical catalysis, to develop the approach.

The project presented technical and semantic challenges. PNNL’s internal objectives in the topic area didn’t map neatly to existing bibliometric topic structures. “Our lab objectives are really detailed and even AI was having trouble identifying topics, so we changed the approach and went with a known factor.” Rather than searching predefined topics, the team searched authors from their laboratory they knew were publishing on chemical catalysis. Using SciVal, they developed topic clusters based on the work of these researchers, then expanded the dataset to incorporate global research output in those clusters across all institutions. “Then we could use that to benchmark against other laboratories working on those topics,” explains Mary Frances. It also allowed them to capture output from collaborations across directorates in their laboratory that otherwise would not have been included.

Map of 11 topic clusters — Illustration – slides from Chemical Catalysis report

While still in a pilot phase, this deeper analysis highlights how the laboratory is contributing to this specialized research space within the boundaries of the field. [SJ1] Feedback from stakeholders is forthcoming, but for Mary Frances, the value lies in being able to test more granular methods to see if topic-level analysis reveals more actionable insights.

She’s also seeking ways to include research outputs that are frequently cited, but often invisible in citation databases. “Besides journal articles, we publish conference papers, books, data sets, and technical reports. Those show up in the Office of Scientific and Technical Information database (OSTI) but they don’t show up in other bibliographic databases, so getting citation metrics on those is very difficult” she explains.

Eventually Mary Frances hopes to expand the topic-based approach piloted in this analysis to inform other key laboratory objectives. “We know how the labs are doing as a whole, and to some extent different directorates, but regarding these specialized topic areas, can we better understand how they are moving the science forward?"

As PNNL seeks to understand impact better, the library is building internal capacity to support it. Mary Frances is training other librarians to work with bibliometric tools and contribute to the reporting process. “I’m helping our team of research librarians who have not worked as much in this space become familiar with the tools, so I’m not the only one doing it.”

She also offers practical advice for other libraries that want to strengthen their role in institutional reporting. “Start with your customer’sask, what’s important to them,” she said, and then broaden their horizons a bit – “introduce what they may not have realized they need. They may say, ‘yeah, we hadn’t thought of that.’”

At PNNL, the library’s commitment to starting with what matters to stakeholders and making it clear and useful is helping research value to be better understood across the organization. And it’s proving that librarians with the right tools and skills can become trusted collaborators and key players in research strategy.

If it wasn’t for my librarian...

I’d have flunked my first year. From lending an ear to offering words of advice, we’ve heard so many stories about how librarians have supported students in overcoming academic challenges.

To the librarians of the world, thank you.

Contributor