Understanding and communicating risk, such as the magnitude or effect size of the benefits and harms associated with preventive screening and other health care management interventions, is fundamental to evidence-based decision making by physicians and shared decision making with patients. Family physicians encounter several different measures of outcome and of magnitude or effect size that are used to describe benefits and harms. Some of these measures might be inappropriate to describe the benefits of screening, and others might be difficult for patients and physicians to understand.1,2
Effective communication of harms and benefits with patients can be challenging owing to limited basic numeracy and health literacy skills among many patients. Numeracy refers to an individual’s ability to understand basic numerical concepts.2–4 For example, patients might have difficulty understanding basic probability or converting between percentage and rate.4 This issue has been described as “collective statistical illiteracy” and has been identified as an important barrier to patients’ understanding of the benefits and harms associated with health decision making.2,3 In Canada in 2014, 55% of Canadian adults were found to have inadequate numeracy skills.5 Health literacy has been defined as “the ability to access, understand, evaluate and communicate information as a way to promote, maintain and improve health in a variety of settings across the life-course.”6 Similar to numeracy, a 2008 report from the Canadian Public Health Association found 55% or 11.7 million adults were estimated to have less than adequate health literacy skills.6
Physicians might also have challenges in understanding statistical measures. A recent survey of physicians from 8 countries, with various levels of training and specialty backgrounds, found that most had difficulty understanding commonly used measures of magnitude, such as relative risk (RR).7 Physicians also had difficulty understanding basic concepts of risk and probability that are important in understanding and communicating the benefits and harms associated with preventive health care.2 A 2012 study conducted in the United States found that most primary care practitioners had difficulty correctly interpreting the results of cancer screening studies and misinterpreted the measures used to describe the benefits of screening.8 In spite of increased training in evidence-based medicine, statistics, and probabilistic reasoning in medical school curriculums, the level of physician statistical literacy does not appear to have increased during the past 40 years.9,10
This article will review and discuss the appropriateness, advantages, and disadvantages of commonly used measures of outcome (overall and disease-specific mortality, incidence or number of new cases, and 5- and 10-year survival) and magnitude or effect size (RR, absolute risk [AR], natural frequency, and number needed to screen [NNS]) that family physicians encounter when interpreting and communicating the harms and benefits of screening. The choice of outcome and magnitude or effect size measures and the format or frame used to present the information can result in different levels of understanding of the same probabilistic information by physicians and patients.1,2
In a previous article11 in this Prevention in Practice series, we introduced a clinical case on screening for lung cancer with low-dose computed tomography (CT). Let’s briefly review the case.
John is a 66-year-old man with a more than 30 pack-year history of smoking. John is a candidate for screening for lung cancer with low-dose CT based on recommendations from the Canadian Task Force on Preventive Health Care (CTFPHC). At a previous visit, John decided not to proceed with screening. After reading a news article on screening, John’s wife asked him to reconsider his decision. At this visit, John and his wife are asking for additional information about screening for lung cancer. How could you best describe the potential benefits and harms of screening to John and his wife?
Outcome measures encountered in preventive screening
Outcome measures of preventive screening that are frequently encountered by family physicians in information provided by government agencies, guideline developers, and advocacy groups on the benefits or harms of screening are outlined in Table 1. These measures include overall and disease-specific mortality, 5- or 10-year survival rates, and increased detection of disease (incidence or number of new cases).
Outcome measures encountered in preventive screening
Overall and disease-specific mortality
Reduced overall or disease-specific mortality in a randomized controlled trial provides the highest-quality evidence for estimates of the benefit of cancer screening.2,12–18 Mortality rates in randomized controlled trials are unaffected by lead-time, length-time, or overdiagnosis bias. Mortality statistics are not affected by timing of disease diagnosis because all deaths that occur in the study population would be included in the overall or disease-specific mortality.2,12–18 The main limitation of overall mortality is the requirement for very large sample sizes, while the limitations of disease-specific mortality relate to assignment of the cause of death and the potential difficulty in measuring harms related to screening.13
Five- or 10-year survival rates
Five- or 10-year survival rates are common survival statistics that consider patients diagnosed with a disease. Physicians will be aware of the frequent use of 5- and 10-year survival rates in cancer to describe the benefits of treatment and prognosis with different stages of disease.2,12,19 However, physicians must be cautious in the use of 5- or 10-year survival rates to describe the benefits of preventive screening.2,14,15,20 This occurs because of the increasing sensitivity of screening tests to identify smaller lesions and the heterogeneity of cancer progression where there is variability in the rates of cancer progression, with some cancers destined to fail to progress or grow so slowly that the patient will die of other causes.2,18,20 Under these circumstances, 5- and 10-year survival can provide exaggerated estimates of the benefits of cancer screening.2,14,15 Lead-time, length-time, and overdiagnosis bias describe how these factors could artificially increase 5- and 10-year survival rates.2,14,15
Lead-time bias is the earlier identification of patients with a disease owing to screening, but who have no change in the actual time of death owing to screening. Five- or 10-year survival rates are increased by early diagnosis because of the increased time from diagnosis to death in screened compared with unscreened patients. In both groups, there would be no change in the time of death.2,14,15
Length-time bias is the tendency for screening to identify more indolent or slower-growing cancers. Faster-growing, more aggressive tumours are more likely to become symptomatic and detected clinically rather than through screening; therefore, patients with screen-detected cancers have longer survival compared with those with clinically detected cancers. This apparent improvement in survival is incorrectly attributed to screening.2,14,15
Overdiagnosis is the detection of an “abnormality” or a “condition” that would ultimately not go on to cause symptoms or death.18,20 In overdiagnosis bias, survival rates are increased because of the detection of patients with nonprogressive cancers (overdiagnosed patients) with screening. The overall number of patients who die of cancer would be unchanged but the 5- or 10-year survival rate is increased because of the inclusion of patients with nonprogressive or overdiagnosed cancer in the estimates of screened patients.2,15,18,20
Incidence
Incidence is the number of new cases or events that develop in a population at risk during a specified time interval. In screening, incidence can be increased owing to the detection of patients with disease that could progress to cause symptoms or death, or the detection of patients who are overdiagnosed.18 Patients overdiagnosed and treated would be considered “survivors” of the disease, although if undiagnosed they would not have experienced symptoms or died of the disease.18 Incidence provides exaggerated estimates of the benefits of preventive screening owing to overdiagnosis. Thyroid cancer provides an example of a disease where an increased incidence of disease was observed from screening.21–23 In Canada21 and other countries,22,23 the increased detection or incidence of thyroid cancer was found to have almost no effect on thyroid cancer mortality owing to overdiagnosis.
Measures of magnitude or effect size encountered in preventive screening
Table 2 outlines the commonly used measures of magnitude or effect size that are used to describe outcome measures of screening and highlights the advantages and disadvantages of each measure. All examples in this table are taken from the National Lung Screening Trial.24–26
Measures of magnitude or effect size encountered in preventive screening
Natural frequencies
Natural frequencies can be defined as the number of persons with events juxtaposed with a baseline denominator of persons at risk.1,2,27 In preventive screening, it is common to present the expected probabilities of outcomes in a population of 1000 persons undergoing screening compared with an equivalent population that is not being screened. Presentation of the results of screening trials in natural frequency formats has been found to improve the understanding of magnitude or effect size on benefits and harms by patients.1,2 Patient understanding is increased with the inclusion of information on the baseline risk.1,2
Absolute risk and absolute risk reduction (ARR)
The AR is the number of events in the screened or control groups divided by the number of people in that group. The ARR is the difference in the event rates between the control and treatment groups. Physicians and patients have greater understanding of risk differences when the results are presented as ARR compared with RR.1,2
Relative risk and relative risk reduction (RRR)
The RR is the event rate in the screened group divided by the event rate in the control group. The RRR is the difference in event rates between the screened and control groups divided by the event rate in the control group. Presentation of risk in the form of RR or RRR can cause exaggerated perceived screening or treatment effects.1,2 This is particularly problematic when the base rate is very low, in which case small changes in ARR can lead to a large change in RRR. This effect is shown in the example in Table 2 of lung cancer screening.24–26
Number needed to screen
The NNS is the number of patients who must be screened in order to prevent 1 adverse event. The NNS is similar to the number needed to treat. The NNS is calculated as the reciprocal of the ARR. Although it has been advocated as a more easily understood measure of the benefits of an intervention, the presentation of results in this format resulted in lower levels of understanding by patients compared with other formats, such as natural frequencies.1
Visual aids
The addition of visual aids to numerical information on risk improves the accuracy and comprehension of numerical data on risk by patients and physicians. Visual displays are better understood when they include both the “sick” and “healthy” populations. Commonly used formats include icon arrays, such as 1000-person diagrams and bar graphs. Examples of 1000-person diagrams developed by the CTFPHC on screening for breast, prostate, and lung cancer can be found on the CTFPHC website (www.canadiantaskforce.ca) and in previous articles in this series.28
Framing of risk information
Framing is the expression of logically equivalent information (whether numerical or verbal) in different ways. Positive and negative frames refer to whether an outcome is described as a chance of survival (positive) or a chance of death (negative). Evidence suggests that positive framing is more effective than negative framing in persuading people to choose risky treatment options such as high-risk surgery.1,27,29 Presenting information as RR as opposed to ARR increased perceptions of treatment or screening benefit.1,2,29
Bottom line
Overall or disease-specific mortality from randomized clinical trials provides the highest-quality evidence for the estimates of the benefits of preventive screening, while other measures of outcomes, such as 5- or 10-year survival rates or incidence, can result in exaggerated estimates of the benefits of screening. In presenting the magnitude or effect size of benefits and harms to patients, physicians should consider using measures of magnitude or effect size that are most effective in improving the understanding of patients. The most easily understood measure of magnitude or effect size is natural frequency supported by the use of knowledge translation tools, such as decision aids that feature 1000-person diagrams. Shared decision making and communication of the magnitude of benefits and harms might be challenging with some patients owing to inadequate health literacy and numeracy skills.
Back to John
You take the opportunity to review the CTFPHC 1000-person diagram on lung cancer screening with John and his wife. This diagram is a visual representation of natural frequencies. After clarifying what the colours in the diagram indicate, John and his wife leave with a good understanding of the potential harms and benefits of screening and plan to discuss further before making a decision. John indicates that he will follow up if he wishes to proceed with the low-dose CT scan.
Notes
Key points
▸ Overall or disease-specific mortality demonstrated in a randomized clinical trial provides the highest-quality evidence for estimates of the benefits of preventive screening. Improved 5- or 10-year survival provides exaggerated estimates of the benefits of preventive screening owing to lead-time, length-time, and overdiagnosis bias. Increased disease detection (incidence or number of new cases) provides exaggerated estimates of the benefits of preventive screening owing to overdiagnosis.
▸ More than 50% of Canadians have inadequate numeracy and health literacy skills. Communicating the harms and benefits of preventive screening might be challenging with these patients.
▸ Patient and physician understanding of the magnitude or effect size of benefits and harms is improved when they are expressed as natural frequencies or in absolute terms such as absolute risk reduction with baseline risk. Visual displays of measures of magnitude or effect size such as 1000-person diagrams increase understanding for both physicians and patients.
▸ Proportionate measures of magnitude or effect size, such as relative risk, can lead patients and physicians to overestimate benefits; number needed to screen is less well understood by patients compared with other measures of magnitude or effect size, such as absolute risk reduction or natural frequencies.
Footnotes
Competing interests
All authors have completed the International Committee of Medical Journal Editors’ Unified Competing Interest form (available on request from the corresponding author). Dr Singh reports grants from Merck Canada, personal fees from Pendopharm, and personal fees from Ferring Canada, outside the submitted work. The other authors declare that they have no competing interests. For additional information on the conflict of interests of the Canadian Task Force on Preventive Health Care, please visit www.canadiantaskforce.ca.
This article is eligible for Mainpro+ certified Self-Learning credits. To earn credits, go to www.cfp.ca and click on the Mainpro+ link.
Cet article se trouve aussi en français à la page 186.
Suggested additional reading
1. Gigerenzer G, Gaissmaier W, Kurtz-Milcke E, Schwartz LM, Woloshin S. Helping doctors and patients make sense of health statistics. Psychol Sci Public Interest 2007;8(2):53-96.
2. Zipkin DA, Umscheid CA, Keating N, Allen E, Aung K, Beyth R, et al. Evidence-based risk communication: a systematic review. Ann Intern Med 2014;161(4):270-80.
- Copyright© the College of Family Physicians of Canada
References
In this issue
Jump to section
Related Articles
Cited By...
- Que devraient enseigner les educateurs pour ameliorer les soins de sante preventifs?
- What should educators teach to improve preventive health care?
- A lencontre du statu quo en matiere de depistage: Appel a laction pour ameliorer lenseignement des soins de sante preventifs
- Going against the status quo in screening: Call to action to improve teaching in preventive health care
- Les soins de sante preventifs et les medias
- Preventive health care and the media
- Clinical considerations when applying machine learning to decision-support tasks versus automation
- Age to stop?: Appropriate screening in older patients
- Est-ce lage darreter?: Depistage approprie chez les patients ages
- Practice organization for preventive screening
- Organisation de la pratique pour le depistage preventif