Opportunities to enhance peer review

The topic of peer review was central to the plot of the 2021 movie Don’t Look Up, wherein a tech company undertakes a mission to disintegrate a comet heading for Earth and harvest its minerals. The conflict? Research backing the approach had not been peer reviewed.

Peer review is the academic evaluation of scholarship by others in the same field, aiming to ensure research is scientifically valid.¹ Canadian Family Physician (CFP)’s peer review process is outlined on page 628, and many medical journals follow a similar process. An activity dating back centuries,² peer review is often considered to be the criterion standard when validating scientific research. However, it has problems. I have seen them first-hand.

For starters—bias. Peer review is often double blinded, but research communities are small. Even when names and institutions are not visible, researchers can still be identified (eg, by content or references cited, since researchers often cite themselves). One study found that blinding failed about one-third of the time, and it questioned the effectiveness of double-blinded reviews.³ When reviewers are anonymous, there are more “rude reviews,” with problematic consequences of their own.⁴

Delay in time to publication is another concern. It takes time to find reviewers, and completing a thorough peer review can take a full day. It is typically an unpaid activity, consuming time that academics and clinicians could spend advancing their own research or treating patients. In this issue, Dr Nicholas Pimlott discusses steps CFP will take to shorten time to publication (page 639)⁵.

Peer review can be inconsistent. While some reviews are detailed, others are short. The decisions of reviewers are also variable; sometimes one reviewer rejects a manuscript while another accepts it with minor revisions. Such discrepancies can be difficult to resolve. In such a case CFP would reach out to a third reviewer, but this causes further delays. In an article, former BMJ editor Dr Richard Smith said, “If [peer review were] a drug … it would never be approved.”⁶

Even with these pitfalls, eliminating peer review would be unwise because we need a system in which scholarly work can be interrogated and validated. How can the process be improved? Some journals, such as those the European Geosciences Union (EGU) publishes, have adopted an interactive model. In addition to articles being peer reviewed, any EGU member or scientist can comment on the preprints online while the reviews proceed.⁷ In turn, authors must respond to every comment. If a decision is made to publish a paper, the original manuscript, comments, and responses remain visible online, along with the final version. In this way readers can see the evolution of the paper. However, I could not find evidence that major English-speaking medical journals use this system.

With the interactive model, the turnaround time is faster⁸ and may improve quality since more reviewers are involved. Drawbacks include the cost and administrative support required to operate the platform. Some journals with this model charge publication fees, whereas CFP does not charge fees under its existing peer review model.

Journal finances are important to consider; both CFP and the EGU publications are open access, meaning they can be read for free. Reforming the peer review process at the expense of adding a paywall would be detrimental, especially for researchers in low-income countries.

Peer review does not end with publication, as emphasized and encouraged by CFP Managing Editor Kathryn Taylor. Postpublication peer review can take the form of responses, external blogs, or media releases or can involve engagement through social media networks. Though less formal, it is another crucial part of the process.

While we do not have an impending collision with a comet threatening our existence, Earth has plenty of emergencies, from the climate crisis to overburdened health care systems. Though there must be political and social will, robust research can help solve these problems. Engaging with and improving the peer review process are key to ensuring research is of high quality.

• Copyright © 2022 the College of Family Physicians of Canada