The peer-review process has been part of academic publishing since the 18th century, although the systemized and institutionalized form familiar today is a more recent development, since the Second World War. Yet for such a long-standing institution, peer review can provoke heated debate within the scientific community. It is a sensitive issue and people tend to have very personal and strongly held views.

Throwing a little water on the flames is Mark Ware, a private consultant commissioned by the Publishing Research Consortium to investigate attitudes about the peer-review process. Peer review: benefits, perceptions and alternatives (1), the first large investigation into attitudes towards peer review, has confirmed that almost all academics are satisfied with current system, considering it an integral part of science. In addition, the study has identified the contentious elements in the process.

Mixed feelings

The study has found that while most academics are committed to peer review, they do have some misgivings. Most notably, according to Ware, they are concerned that reviewers are overloaded, wait times between submission and final publication are too long and there is suspicion of bias and abuse of power by anonymous reviewers.

Those concerned about any or all of these issues often express their dissatisfaction loudly, yet Ware’s study failed to identify any serious objection to the process among the more than 3,000 academics questioned.

It is an emotional issue. Academic careers are made or lost in the publishing world. Careers are measured by successful publications, and this is linked to the positions and funding available. “Of course people get upset if they suspect bias – this is their career we are talking about and if you feel you are being discriminated against, you are likely to speak out,” says Ware.

In fact, the most remarkable result of the study is the mixed feelings it has teased out. For instance, 90% of authors reported that peer review improved their last paper, 85% agreed that it greatly improves scientific communication and 83% believe there would be no control without peer review, while only 3% think it is completely unnecessary. Yet, with such support, the statement “peer review needs a complete overhaul” received a surprisingly high level of support, with opinion split between 35% disagreeing and 32% agreeing. A similar division was seen in the response to the statement “the current system is the best achievable” (36% against and 32% in favor).

Meeting specific concerns

Some 90% of authors questioned in the study are also reviewers. While the average number of reviews was eight per year, 44% of the reviewers questioned actually carried out 79% of reviews, and this group reported doing more reviews each year than they were comfortable with, suggesting serious overload in a core group.

The study also investigated what would tempt more people to review. Free subscription to the journal came out as a clear favorite (56%), closely followed by acknowledgement. Although reviewers are less likely to review if their name is revealed to authors (47%) or if their reports are published with the paper (49%), many would like to be acknowledged (44%). Ware suggests that journals print a list of all reviewers once a year, thanking them for their contribution.

Long wait times are also a major cause of dissatisfaction. According to the study, review times of up to 30 days are acceptable by about two thirds of respondents. However, for those waiting three to six months, only 19% were satisfied, and beyond that, the satisfaction rate drops to just 9%. Ware says, “This is obvious. As the world gets used to instant results, people expect things to move faster, not slower.”

Justice is blind

Although single-blind peer review, where the reviewer’s identity is hidden, is the most commonly used and experienced form of peer review, it is not the only option available (see sidebar). The practice of hiding the reviewer’s identity goes back to the 18th century and is considered an essential component. Without it, few would be prepared to review. Yet many authors claim that this encourages bias and other dubious activities behind the veil of secrecy.

This helps explain the overwhelming support for double-blind peer review, which also masks the authors’ identities, with 56% citing it as their preferred option compared to 25% for single-blind.

According to Ware, this can be partly explained by the fact that it seems more scientific. “Since science is supposed to be unbiased, many academics, especially those in scientific fields, tend to believe that a double-blind system would eliminate any perceived bias.”

Ware adds that a lot of the suspicion of bias comes from non-Anglophone regions, which generally experience higher rejection rates. However, there can be many reasons for this, including inferior English language skills and less well developed research infrastructure, rather than any specific bias. Women also claim discrimination, although there is little hard evidence for this view (Link to: last issue), and young authors say that their papers are being rejected unfairly in favor of more established names.

Double-blind reviews would go some way towards dispelling these concerns; however, as many commentators have pointed out, in small academic circles it is not difficult to guess who the author is, so would hardly be worth the effort. There is also some support for open peer-review, with 27% considering it effective, although only 13% prefer it. Post-publication was considered effective by 37% of respondents, although only 5% said it was their preferred option. Ware puts this discrepancy down to the fact that it can be used as a supplementary review.

Seal of quality

Returning to support for the system as a whole, most academics see the peer-review process as part of the general procedure for maintaining and improving quality. The simple fact of having your work reviewed by your peers confers a sense of having met some standard. It also helps support the system that routes better papers to better journals.

According to Ware, “Most academics see peer review as part of a wider system that improves both individual papers and the literature in general. It is good for readers and is an accepted part of the system for assessing research.”

Especially for readers outside a specific community, peer review helps everyone identify leading research and understand what fits with the general consensus on a particular topic (see Sense About Science).

Yet, for a system to be truly accepted, it not only has to be fair, it has to be percieved to be fair, and in this respect single-blind peer review is failing. While no one is calling for the system to be scrapped, the community as a whole has certain misgivings that need to be addressed.

According to Ware, “this topic is very broad, and the next step is for the community to try out different peer-review models. There is overwhelming support for peer review, but what we have found is that certain reviewers are carrying the bulk of the load, so journal editors need to find ways of easing their role. They also need to think about review times, and look into ways of bringing the wait times to acceptable norms for their communities.”

The problem of perceived bias is probably the hardest to dispel; for authors of substandard papers, it is easy to accuse the reviewer of prejudice. However, it is also impossible to guarantee that it never happens, and the majority of authors would certainly welcome more transparency.

To cite this article, please use: Michelle Pirotta, “Reporting back… Attitudes about peer review”, Elsevier Editors’ Update, Issue 24, November 2008

References:
(1) Ware, M (2008) Peer review: benefits, perceptions and alternatives. London: Publishing Research Consortium.

Useful links
  Peer review: benefits, perceptions and alternatives

  Nature discussion forum on peer review and the Publishing Research Consortium study