How to improve reviewer diversity: a toolkit for editors (part I)

© istockphoto.com/PeterPencil

In our miniseries so far, we have discussed the importance of inclusion & diversity (I&D), highlighted the work that still needs to be done to diversify the peer reviewer pool, and shared examples of the actions being taken by publishers, editors and reviewers themselves. Yesterday we shared some examples of how Elsevier is promoting inclusion & diversity in peer review. Today and tomorrow, we will take a deep dive into the role editors play and offer best practice principles to follow when inviting reviewers. Editors, we recognize that finding willing peer reviewers is a non-trivial task, and we are loath to add more to your burden, but please hear us out. Yes, considering I&D attributes, in addition to an individual’s qualifications as a referee, is of course an additional ask. But, as explained in our earlier article, by diversifying the pool and increasing the number of available reviewers, you will save time down the road by minimizing the fatigue that results from too many demands being placed on too few reviewers, and you’ll be promoting scientific rigor and quality to boot.

We believe passionately that it is both worthwhile and imperative that editors employ best working practices that promote inclusion & diversity in their peer reviewer pool and invitations. It’s not as hard as you might think! Read on to hear what you can do as an editor, beginning here with some simple principles to apply day-to-day as you go about inviting reviewers.

First acknowledge your (unconscious) bias (spoiler alert: we all have it!)

While doing so can be uncomfortable, it is important to acknowledge that each and every one of us has bias, and this unconscious bias can affect our decision making without us even realizing. With regards to peer review, it can impact both reviewers invited, and influence the weight given to the reports returned (confirmation bias). With this understanding we can move forward with the knowledge that we must each actively challenge our inherent bias and self-check as we go about our activities, especially those we perform frequently, such as selecting and inviting reviewers, where it can be easy to slip into “auto-pilot”.

If you only do one thing: self-check as you issue invites

If you can only commit to one thing, make it this: to take a minute to self-check and strive for balance as you issue your reviewer invites. For example, if you approach a field leader with a high h-index who is likely to provide an established “big-picture” view, always pair them with an early career reviewer. Consider diversity attributes such as: gender, career stage, affiliation & country, race & ethnicity, first language, etc. We have a collective responsibility to no longer find it acceptable to invite reviewers all of the same gender, all from the same region, all with similar experience and so on.

Of course, rarely does reviewing involve issuing two invites for two reviews; usually more invites will need to be issued. With this in mind, you can use Editorial Manager’s alternate reviewer feature opens in new tab/window to link reviewers with similar diversity attributes to ensure diversity in the final selection.

Widen your search sources

You should seek candidates from beyond the existing journal database and your own personal network, and ideally use a variety of sources opens in new tab/window to generate your candidate list rather than relying on the same one time and time again. Options include Elsevier’s Find Reviewers reference tool opens in new tab/window (Reviewer Recommender and Keyword Search tabs), Scopus, PubMed, and more. By so doing you have the opportunity to expand the referee pool beyond the journal’s existing network.

Be aware, however, of the limitations of any reference tools. For example, any tool based on academic records will be biased towards more senior researchers, and those with the greatest opportunities to participate in the research-publication cycle, who tend to have larger academic records. Mindful of this we are working hard to eliminate any historical bias in the tools and signals we provide you.

Exercise scepticism over suggestions and caution over conflicts (and not just for the obvious reasons!)

Author-suggested reviewers are not the best way to increase diversity. Since authors are not provided with enhanced search tools to identify relevant referees, they usually suggest individuals from their own network, who might have potential conflicts of interest and/or confirmational biases. Several studies  show that author-suggested reviewers are more likely to recommend “minor revision” or “accept”, for example. You should therefore use author-suggested reviewers with caution, always independently validating the email addresses of any suggestions and using at least one more independent referee who is ideally also diverse from the submission author(s).​

For the same reason, you should avoid inviting candidates to review who have a potential conflict of interest with the authors. Elsevier's Find Reviewers reference tool supports conflict of interest checks and either excludes candidates (for example author-suggested reviewers who were also recent co-authors of the submission authors) or flags potential conflicts such as recent co-authorship or being based at the same institution or in the same country as any of the submission authors. Editorial Manager itself flags candidates who are submission authors, author-suggested reviewers or those based at the same institution as the submission author(s). These are helpful tools, but be mindful that they don’t capture all possible conflicts of interest. Therefore, it is important to ask reviewers to raise any potential conflict of interest with you in advance of reviewing a manuscript, something that is prompted in Elsevier’s standard reviewer invitation letters.

We hope this provides you with core principles which we strongly encourage you to embed in your daily editorial work as you select reviewer candidates and issue invitations. Check back tomorrow for the final article in the series in which we’ll share further recommendations to help editors promote I&D in journal reviewer pools, and some broader considerations.