Reporting back... peer review conference

When a conference is devoted to peer review and biomedical publication, one hopes that the aim is to look for improvements and solutions to peer review issues and problems; to encourage new practices and better ways of doing things. At this conference, however, most of the papers focused on problems rather than offered solutions. So, if there was a flaw to this busy conference, attended by hundreds of delegates, it was this. Of course, there were some great topics of interest to editors, and I will outline those here.

Although it’s not needed to follow the points mentioned in this article, you might want to see the abstracts that are discussed. To do so, go to  www.jama-peer.org and download the PDF of the final program with abstracts. In case it is helpful, I have included page references to both the PDF and the program (since they differ).

Funded positivity?
Many would argue that pharmaceutical companies for reasons of potential conflict of interest should not sponsor research, particularly medical research. This is a problem often seen in peer review when, for example, selecting reviewers for an article that is tied financially to a reviewer’s affiliated funding source (or not).

Recent data suggests that there may be a positive bias for a drug study when the research has financial ties to a pharmaceutical company. In their paper, Are Author's Financial Ties With Pharmaceutical Companies Associated With Positive Results or Conclusions in Meta-analyses on Antihypertensive Medications? (PDF p. 20; program p. 16), authors Yank, Rennie, and Bero suggest that there is such a link, although more research needs to be done.

Similarly, in another paper Sponsorship, Bias, and Methodology: Cochrane Reviews Compared With Industry-Sponsored Meta-analyses of the Same Drug (PDF p. 20; program p. 16), authors Joergensen and Gotzsche find that drug reviews sponsored by industry should not be trusted. Although, perhaps, an impractical bit of advice, this paper still offers a solution rather than focusing on the problem without offering a way out of the research dilemma.

Moreover, it should be borne in mind that irrespective of whether there is a relationship between pharmaceutical companies and positive results, the problems outlined in the studies presented affect a specific subject field, they cover a relatively small number of studies and are not representative of peer review process across the whole of scholarly publishing.

Remember, too, that those funding the research often push positive results to the forefront. Peer reviewers and editors are not at fault if negative results of research are never submitted for publication (ergo they are never peer reviewed).

Danger in the popular press
In Reading Between the Ads: Assessing the Quality of Health Articles in Top Magazines, (PDF p. 25; program p. 21), authors Hussey, Miller, and Jadad argue that misinformation in the popular press is exacerbating the knowledge gap between the public and the scientific community. The fact is, the untrained reader cannot distinguish between science and bunkum. The authors of this paper conclude that medical articles in the popular press fail to meet even basic criteria for quality and can mislead the public. The recommendation is for peer-reviewed biomedical publications to support and define standards for the popular press so that the public is better protected. In fact there is a very good article that covers these issues at http://www.senseaboutscience.org.uk/peerreview/

Blind peer review
Finding in favor of the adoption of blind peer review for all scientific meetings, the paper from Ross, et al, Assessment of Blind Peer Review on Abstract Acceptance for Scientific Meetings, found bias favoring prestigious American institutions when an open review model was used.

The recommendation to use double-blind review, where neither the author nor the reviewer is known, should be treated with caution and one might consider that, even when the author’s identity is masked, his or her identity will still be known in a niche research area or more likely through self-citation.

In addition, when an author has recommended a reviewer, there is likelihood that the review will be favorable, blind or not.  Asking an author to recommend a reviewer is a good way of building your reviewer database. However, in order to mediate against any bias, we suggest that editors use only one recommended reviewer out of the three normally reviewing a given paper for a more unbiased review. It is also worth being aware that the paper Comparison of Author and Editor Suggested Reviewers in Terms of Review Quality, Timeliness, and Recommendations for Publication (PDF p. 16; program p. 12), by Schroter, Tite, Hutchings, and Black found no difference in the review quality and timeliness of the different reviewers although author-suggested reviewers tended to recommend more papers for publication.

Open access
Open access is something of a hot topic, and is even categorized as such in the conference program. In the paper by Schroter, Tite, and Kassem: Authors' Access to Financial Support at the Time of Paper Acceptance: A Survey of Biomedical Journals (PDF p. 32; program p. 28), the authors found that most published research is not externally funded and that, even when there is external funding, the funds are not available to support publication. So an author-pays open access model was not found to be currently viable and might, in fact, hinder the publication of some research.

As with the possibility of drugs research with negative results being ignored by a pharmaceutical company, the idea of papers not being submitted due to the author-pays open access model raises the concern that papers of outstanding merit may never be peer reviewed and ultimately published.

Another paper, this one by Sox and Bylsma, asks Would You Drop Your Membership? Professional Organization Members' Reaction to Open Access (PDF p. 32; program p. 28). The paper says that a significant minority of members would drop their membership but I believe the research disguises the true figure the research didn’t take into account the peripheral benefits and other factors involved in membership in professional associations. I think membership is about more than the publication received as part of the membership fee and I believe that further research would support that.

Ethics
With the proliferation of papers critical of peer review throughout this conference, and the recent high profile new stories in the media, editors might start to think that the peer review process and publishing are in disarray. But the  Committee of Publication Ethics (COPE) continues to provide excellent support to editors in their day-to-day moments of doubt. The paper, For Which Cases of Suspected Misconduct Do Editors Seek Advice? An Observational Study of All Cases Submitted to COPE (PDF p. 19; program p. 15), Kleinert, Theobald, Wager, and Godlee find that the single most common problem faced by editors is redundant or duplicate publication.

Of course, redundant or duplicate publication is a pressing problem for editors, but it is a relief to be able to say that the most difficult, damaging, and embarrassing problems facing editors, such as fraud and falsified results, are far less prevalent than the sensational papers focusing on them would have us believe. The paper Retractions in the Research Literature: Misconduct or Mistakes? (PDF p. 18; program p. 14), by Druss, Bressi, and Marcus substantiates this viewpoint, suggesting that unintentional mistakes are far more common than scientific misconduct.

Editorial changes
To reiterate that it is not all gloom and doom, the last paper I will tell you about here recognizes the value of peer review and describes its important place in scientific publishing. By Lee, Boyd, and Bero, the paper Editorial Changes to Manuscripts Published in Major Biomedical Journals (PDF p. 15; program p. 11), assesses the added value of peer review and concludes that the current editorial process improves the clarity, accuracy, and reporting of results in papers. What’s more, it indicates that reviewers’ immersion in the paper’s area of research helps to tease out potential conflicts of interest including the role of the funding source in the research.

So, my message to editors is not to believe the hype and sensationalism of the most damaging issues that arise and the headlines they grab. Peer review is alive and well in scientific publishing, and quietly adding value every day.

****

Please send responses to this article to EditorsUpdate@elsevier.com