Choosing guidelines to use in your practice

Describing the evidence

Every guideline expresses its decisions using a scale, and understanding the various scales is important. Table 1 shows examples of scales used by the CTFPHC and the United States Preventive Services Task Force over time. Initially, they both used a 5-point A-to-E Likert scale. Then the negative recommendations were combined into a single category, and the I recommendation was added for situations in which evidence is inadequate. The CTFPHC now uses the GRADE (Grading of Recommendations Assessment, Development and Evaluation) system, developed by an international group led by Canadians.⁶ The CTFPHC has chosen to use a version with only 4 points. Notably, there is no middle category, forcing guideline developers to decide for or against. When benefit is absent, small, uncertain, or not likely to outweigh harms, then the GRADE system recommends against the activity.

Originally, guidelines focused primarily on the type of evidence supporting a recommendation and whether that showed the intervention had any effect. Subsequently, guideline developers differentiated the quality of the evidence and the size of the effect. Higher-quality evidence with greater precision provides greater trust in the estimate of the effect. Thus, high-quality evidence might show that an intervention has a very small effect and is, therefore, not worth doing. More recently, we in the medical profession have better recognized that our actions, while intended to do good, also create harms. Therefore, high-quality guidelines now assess quality of evidence to decide the presence and strength of the benefit, and measure the severity of harms, then reach a conclusion that balances the harms and benefits. Ultimately, the information should be shared with patients to understand their perspectives, so we can help make decisions that are right for them. We have written about this process for preventive care,^7,8 but it applies at each stage of illness care as well. Different guideline developers use varying systems, and readers must understand the meaning of the systems to interpret recommendations properly.

In the rush of daily practice, we must make binary (yes or no) decisions—to do something or nothing, to screen or not screen, to treat or not treat. Yet most medical data are continuous, with variation in risk as the measurement changes. For example, the risk of disease often increases with age, as does the likelihood of finding treatable disease, but the potential benefit for extending life might be less for those with less remaining life to live. The risk of harm often increases substantially with age, so that at a certain point the potential for harm is greater than any benefit. Thus, screening for diabetes and treating it might be worthwhile in middle age. However, a 90-year-old person whose hemoglobin A_1c level is slightly above the diagnostic threshold need not worry, as the complications of diabetes take a long time to appear. Drug treatment might cause adverse effects, with no benefit. This balance must not be lost for the sake of writing simple rules.

How to evaluate guidelines

Understanding how guidelines are developed helps us critically appraise those guidelines we encounter in practice. The Appraisal of Guidelines for Research Evaluation (AGREE) instrument was developed by a Canadian-based group to assist users with assessing variability in guideline quality.⁹ Table 2 lists the headings for the AGREE II approaches.⁹ These headings appear overwhelming, but not all of them carry equal weight. Their value varies from topic to topic, and we indicate a rough measure of their importance. Rigour of development, ensuring that preconceptions and bias are minimized, and how the evidence is used to develop recommendations are critical. A guideline that is written for primary care, is focused on the components of health issues we deal with in primary care, and was created under the supervision of a group based in primary care is often more useful for our needs than one about the whole depth of a problem written by other specialists who are mostly focused on care of advanced and complex cases.

Recently, a group of family medicine–based researchers published the Guideline Trustworthiness, Relevance, and Utility Scoring Tool (G-TRUST) approach to classifying the value of guidelines (Box 2).¹⁰ This tool considers threats to relevance, to evidence, and to interpretation. If a guideline is not relevant, we need go no further. If the evidence is not considered properly or there are threats to unbiased interpretation, we can reject the guideline. This enables quick focus on a smaller number of guidelines that are likely to be good. During a resident seminar in 2015, 4 groups of residents were each asked to assess a different guideline on prostate cancer screening using the AGREE II approach. Table 3 shows their findings.^9,11–14 If the G-TRUST approach were available at the time, the residents would likely have been even more discriminating.

Conflicts of interest

Guidelines written by disease advocacy groups are common. While their concerns are understandable, too often they are sponsored in variably concealed ways by commercial interests. The evidence is strong that sponsored research studies and guidelines are nearly always biased toward the sponsor’s preferences, compared with guidelines on the same topic without such sponsorship.^15,16 Guideline groups that do not base their recommendations on a high-quality systematic review, or guidelines written by experts who already have beliefs about the topic, are at risk of allowing preconceptions to supersede the evidence.^17,18 Some guideline committees include representation from token family physicians, who might not have much skill in research and critical appraisal. Such representatives are placed in a difficult position—it is hard for them to push back against disease “experts” who are promoting their own agendas.

As an illustrative example, Box 3 shows the process used to create the recent “Men’s Health Guidelines for Family Medicine,”¹⁹ widely distributed by Acerus Pharma, which sells hormone products. The panel of 50 names listed is stated to have

[a] balanced composition ... (ie, specialists, family physicians, nurse practitioners and pharmacists) [that] is instrumental in limiting the many types of bias, including competing or conflicting interests. This approach obviates the need to have the panel members declare any existing or potential competing interests. For further information on competing interests contact cssam.ca or the individual panel members directly.¹⁹

Eleven members of the guideline committee are urologists and 20 are family physicians, but it is unclear how involved many of them were in considering the evidence. The website that is said to contain their conflict of interest statements is not accessible to non-members. While a round of discussions took place, the final document was not reviewed and approved by all members of the advisory board whose names are listed. These features make it difficult to accept their guideline recommendations.