Research ethics boards (REBs) are perhaps the most unloved component of the research endeavour. In this issue of Canadian Family Physician, Kotecha and colleagues (page 1165)1 join a chorus of voices decrying current governance of research by institutional REBs. In their article they describe the experiences of the Canadian Primary Care Sentinel Surveillance Network in securing ethics approval. The authors document inconsistent interpretation and application of privacy laws and handling of ethical issues. They contend that REB delays impede research programs that could better inform us about chronic disease. They recommend a more specialized national centralized REB responsible for multisite studies related to population health.
So what’s new?
There is not much new in the experiences related by Kotecha and colleagues.1 Some of the problems with REB delay and inconsistent application of multisite reviews have been noted in many studies. For example, Goodyear-Smith and colleagues noted the international variation in ethics committee requirements for the same protocol across 5 Western nations in 2001.2 Similarly, varied experiences in local ethics approval for multicentre studies within one country have been exhaustively described by Maskell et al.3 In fact, an editorial on the study by Maskell et al noted that investigators required 62 hours to photocopy 25 396 pieces of paper to satisfy 51 local research ethics committees.4 Whitney and Schneider5 recently tried to calculate the amount of time and the actual harm to patients associated with REB delay.
These accounts come from researchers. People involved in research ethics itself have similar criticisms. For example, Jerry Menikoff, Director of the Office of Human Research Protections in the United States, offers an interesting perspective in the New England Journal of Medicine in an editorial entitled “The paradoxical problem with multi-IRB review,” in which he puts forth the rationale for centralization of ethics review.6 In a commentary published in the Journal of the American Medical Association (JAMA), Christine Grady, who is from the Department of Bioethics at the National Institutes of Health, asks whether institutional review boards (IRBs) actually protect human participants. She states, “To date, no published study of which I am aware has evaluated the effectiveness of IRBs in protecting research participants, and few have investigated the nature, quality, or thoroughness of IRB deliberations.”7 One scholar said, “IRBs have disrupted student careers, set back tenure clocks, and blunted the essence of many intellectual traditions. Facing demands that spiral to the level of sheer impracticality, faculty and students … face a stark choice: to conduct innovative research in their fields or to meet the requirements of their IRB.”7
In 2007 Norman Fost and Robert J. Levine,8 noted experts in research ethics, argued the following: “Inflexible requirements for adherence to narrow interpretations of every word in regulations and other policies have led to a system that is more concerned with protection of the institution than protection of human research participants.” Further, they argued that the cost of the system is increasing with scant evidence of a return on investment with regard to protection of patients and other research participants.
Responsibility for delay
As a Canada Research Chair, I have sent many protocols to REBs, so I am aware of the difficulties and problems associated with REBs. However, I am not entirely certain that the REB is the most capricious and most time-wasting segment of the research enterprise. Indeed, although I agree with many of the complaints lodged against REBs, we really do need to step back and look at improving the entire process of research review. The way I see it, there are 3 principal elements of research review:
-
grant submission, review, and approval;
-
REB submission, review, and approval; and
-
manuscript submission and acceptance for publication.
First, it is instructive to note that, at least from the published data available, the least time-consuming of these steps is, in fact, the REB. Second, it is notable that claims and arguments about the inconsistency of ethics review boards are paralleled by similar concerns about the arbitrariness and lack of standardization of both grant and publication review. In essence, every aspect of review has its difficulties. Focusing as much as we do on REBs without considering the entire ecology of the research enterprise and arguing for similar improvements to each element is largely unfair to the least understood, supported, and valued element.
I wonder, for example, how long it took Kotecha and colleagues1 to get their paper published? To how many journals did they submit and how long was the process between initial submission to final publication? It is interesting that Richard Smith, former Editor of the British Medical Journal, recently argued that “[p]republication peer review is faith based not evidence based.” He explained, “[There is a] mountain of evidence of the failures of peer-review: it is slow, expensive, largely a lottery, poor at detecting errors and fraud, anti-innovatory, biased, and prone to abuse.”9 Hopewell et al10 reviewed the time to publication of the results of 196 clinical trials and found that just over half of all trials were published in full. Trials with positive results were published in approximately 4 to 5 years and trials with null or negative results were published after about 6 to 8 years.
Similarly there is evidence that the granting cycle takes a long time. The average National Institutes of Health funding cycle is 27 months and success rates are low.11 A Cochrane review that looked at peer review for improving the quality of grants concluded that little empirical evidence showed that peer review affected grant giving.12 In other words, across the board no good evidence indicates benefit of review, from grant writing through ethics review to publication.
In the current study (though no tabular data simply show the average time and range for approval from each site), the longest delay was close to a year.1 Similar studies of REB performance indicate average response in months.13 Many REBs do not routinely publish their time-to-approval statistics, though there is a move to make this a standard practice. Even then, it is not necessarily the REB that is responsible for delays. Researchers sometimes fail to provide required documentation and often do not respond promptly to queries raised by the REBs.
No blanket solution
In my professional life, I have participated in all elements of research review. As a researcher and as a Chair and member of REBs, I am often uncertain who is guilty of greater malfeasance: the researcher or the REB. I think there are good arguments against centralization, and arguing for this as a blanket solution to all problems does not solve the issues of interpretation. While it might reduce the number of possible interpretations, it does nothing to guarantee any less capricious interpretation of guidance documents.
Our peers in peer review
I will end with a particular plea. I think part of the heat and venom directed at REBs originates because members are not considered peers. Whereas there is some complaint about grant and peer review, mostly investigators take these procedures as part of the process, and consider it necessary to develop the thick skin a good researcher requires. No one would claim in a prominent journal such as JAMA that prepublication review or grant peer review blunted intellectual traditions. In fact, peer review is regarded as key to the scientific method.
This is precisely the problem. Investigators do not see engaging with the REB as akin to peer review. That is, REB members are not of the same tribe, so to speak. That is where the original suggestion comes from: ask not what your REB can do for you; ask what you can do for your REB.
Many innovative solutions might improve REB performance. There is no question that it must improve. One step is for researchers themselves to participate in REBs. Researchers who have served on REBs report more positive experiences with REBs than researchers who do not.14 In fact, according to the Tri-Council Policy Statement,15 only one member of the REB is, in fact, an ethicist. Others are a lawyer, content experts, or scientists, as well as lay members. We need to reward participation and contributions to research ethics and accord them the same prestige of academic citizenship as serving on grant peer-review panels and reviewing manuscripts for high-quality journals. The data suggest that all elements of the research process require improvement. I hope we start to see some editorials soon in JAMA and the New England Journal of Medicine decrying the arbitrariness and caprice of journal reviews and grant reviews. Then perhaps we can move forward constructively without singling out one element of the research enterprise as deficient.
Footnotes
-
Cet article se trouve aussi en français à la page 1115.
-
Competing interests
None declared
-
The opinions expressed in commentaries are those of the authors. Publication does not imply endorsement by the College of Family Physicians of Canada.
- Copyright© the College of Family Physicians of Canada