Research ArticleResearch

Choosing Wisely Canada campaign associated with less overuse of thyroid testing

Retrospective parallel cohort study

Kimberly Wintemute, Michelle Greiver, Warren McIsaac, M. Elisabeth Del Giudice, Frank Sullivan, Babak Aliarzadeh, Sumeet Kalia, Chris Meaney, Rahim Moineddin and Alexander Singer
Canadian Family Physician November 2019; 65 (11) e487-e496;

Abstract

Objective To assess the effectiveness of a Choosing Wisely Canada (CWC) initiative to improve thyroid-stimulating hormone (TSH) test ordering for patients with no identified indication for this test.

Design Retrospective parallel cohort study using routinely collected electronic medical record (EMR) data. The CWC initiative included supporting primary care leads in each participating family health team, providing education on better test ordering, and allowing adaptation appropriate to each local context.

Setting Toronto, Ont, and surrounding areas.

Participants Family physicians contributing EMR data to the University of Toronto Practice-Based Research Network and their patients aged 18 or older.

Main outcome measures Proportion of adult patients with a TSH test done in a 2-year period (2016 to 2017) in the absence of EMR data with an indication for TSH testing; proportion of TSH test results in the normal range for those patients; and change in the rate of TSH screening in sites participating in the CWC initiative compared with sites not participating.

Results A total of 150 944 patients (51.7% of studied adults) had no identified indication for TSH testing; 33.4% of those patients were seen by physicians in the family health teams participating in the CWC initiative. Overall, 35.1% of all patients with no identified indication had at least 1 TSH test between January 1, 2016, and December 31, 2017. The 119 physicians participating in the CWC initiative decreased their monthly rate of testing by 0.23% from 2016 to 2017, a relative reduction of 13.2%. The 233 physicians not participating decreased testing by 0.04%, a relative reduction of 1.8%. The monthly difference between the 2 groups was 0.19% (95% CI -0.02 to −0.35 P = .03), a relative difference of 11.4%. The TSH testing decreased for almost all CWC patient subgroups. More than 95% of patients tested in both groups had TSH results in the normal range.

Conclusion This study found high rates of TSH testing without identified indications in the practices studied. A CWC initiative implemented in primary care was effective in reducing TSH testing.

Population-based screening of asymptomatic adults for thyroid disorders is not recommended, as there is no evidence that this is beneficial.1,2 Through Choosing Wisely Canada (CWC), the College of Family Physicians of Canada advises against ordering thyroid function tests in asymptomatic patients.3

The recommended test for the detection of hypothyroidism or hyperthyroidism is a thyroid-stimulating hormone (TSH) assay.4 In one Ontario-based report, this test accounted for the second-highest laboratory costs after microbiology cultures.5 Owing to concerns about overuse, TSH testing was removed from the standard Ontario laboratory requisition in 2012 as a routine test request option.6

There is considerable variation in and overuse of laboratory tests including TSH testing.7–9 More recently, a study in 2 family practices in or near Toronto, Ont, found significant rates of overuse, demonstrating that 22.4% (95% CI 16.9% to 28.8%) of TSH tests did not conform with ordering guidelines.10

Six family health teams (FHTs) in Toronto recently undertook an initiative to promote the adoption of CWC recommendations in their clinical settings. Family health teams are organized interdisciplinary primary care teams; the leadership team usually includes a medical director or equivalent, and there are opportunities to communicate and promote best practices among colleagues in the FHT.11 Ordering of TSH testing was an agreed-upon target to be addressed as a component of the CWC initiative.12

The aim of this study was to assess the effectiveness of the CWC TSH test ordering improvement initiative. We estimated the proportion of patients who had a TSH test done with no identified indication for testing and the proportion of normal test results for those patients. We compared the change in the rate of TSH testing without identified indication in sites participating in the CWC initiative with the change in sites not participating.

METHODS

We conducted a retrospective parallel cohort analysis to study the effect of the initiative to reduce TSH testing for patients without identified indications. We followed the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) recommendations to report the results.13

The CWC recommendation targeted was “Don’t order thyroid function tests in asymptomatic patients.”3 Supported through the Adopting Research to Improve Care (ARTIC) program,14 this initiative included the following elements: hiring a central program manager; supporting primary care leads at each FHT; providing education on better ordering; allowing adaptation, change ideas, and implementation appropriate to each team’s context; and providing information on rates of TSH ordering over time and comparisons with peers in collaboration with a large community-based laboratory. Information on elements of the CWC ARTIC initiative and an example of a laboratory report are available from CFPlus.* The initiative was initiated in March 2016 and lasted for 2 years. Information on laboratory testing rates was provided to each consenting family physician every 3 months; the initial report was provided in April 2017.

Six FHTs in the Toronto area participated in the ARTIC CWC initiative. These were a convenience sample of FHTs associated with a 6-hospital–based collaborative (the Joint Centres for Transformative Health Care Innovations)15 that expressed interest after discussions in October 2014. Each hospital had a large FHT affiliated with it, and all FHTs joined the initiative.

We used routinely collected clinical electronic medical record (EMR) data from the University of Toronto Practice-Based Research Network (UTOPIAN) Data Safe Haven16; UTOPIAN receives regularly updated EMR records from more than 400 family physicians affiliated with the University of Toronto Department of Family and Community Medicine and contributes to the Canadian Primary Care Sentinel Surveillance Network.17 Consenting family physicians and other primary care providers contribute de-identified EMR data to the UTOPIAN repository; patients can opt out if they choose to do so.17,18

The UTOPIAN Data Safe Haven included data for 4 of the 6 FHTs participating in the CWC initiative, and these data are reported here. Data from all other sites contributing to UTOPIAN were included for comparison. The geographic distribution of sites contributing to UTOPIAN can be found online.19

We used data on TSH testing from January 1, 2016, to December 31, 2017, and data on patient characteristics dating back to January 1, 2010. Education and dissemination activities were provided as part of the ARTIC project in 2016, while information on laboratory testing was provided in 2017. There were thus a number of activities occurring during a 2-year period; we used January 1, 2017, as an estimated midpoint for the intervention. We calculated monthly rates of TSH testing (number of TSH test results present divided by number of patients meeting cohort criteria) during the 2 years of interest.

Data from EMRs are observational and therefore include information about the health of patients, the processes of care, and ways care providers enter data in the record.20 The CWC initiative might have resulted in changes in physician recording of symptoms and conditions considered appropriate for TSH testing, which would have resulted in a change in the proportion of TSH testing in the CWC cohorts unrelated to patient factors. We used a retrospective approach and excluded any patient with any TSH-related issues as of December 31, 2017. If there were any changes related to differences in recording, these would therefore not affect the analysis.

Eligible patients were 18 years of age or older as of January 1, 2016. Patients had at least 2 visits with their family physician between January 1, 2010, and December 31, 2017, with at least 1 visit during the 2 years of interest for thyroid testing (2016 to 2017).

We searched the records for indications to test for TSH at any time in the past 7 years, starting from January 1, 2010. These indications included a diagnosis of thyroid disease (hyperthyroid, hypothyroid, thyroid cancer) in the EMR or at least 1 prescription for thyroid replacement therapy. Other indications for testing included prescriptions for amiodarone or lithium and current or recent pregnancy or infertility.1 The data did not allow for differentiation between tests done for screening and tests based on clinical suspicion of thyroid dysfunction during clinical encounters. However, we attempted to further isolate tests done for clinical suspicion of thyroid dysfunction in our sample by searching for diagnoses of anxiety, tachycardia, fatigue, insomnia, abnormal weight gain, hair loss, or galactorrhea.1 Supplemental material available from CFPlus* provides information on the EMR search criteria used for exclusions.

Once all patients with an identified indication for TSH testing were excluded, the remaining patients were considered to have had no clear indication for TSH testing.

We analyzed clinical cofactors that could be associated with a higher prevalence of thyroid disorders. The prevalence of thyroid disorders is greater in the presence of autoimmune conditions; we included the most common condition, rheumatoid arthritis, as a cofactor.1 We also examined conditions for which TSH testing might be indicated such as dementia, depression, and obesity.1 We report levels of comorbidity, as they are associated with increased health care use; we included the 8 conditions validated by the Canadian Primary Care Sentinel Surveillance Network: diabetes, hypertension, depression, dementia, chronic obstructive pulmonary disease, osteoarthritis, epilepsy, and Parkinson disease.21

Statistical methods

All statistical analyses were conducted using SAS software, version 9.4. A random-effects logistic regression was fitted to compare the 2 cohorts. Population-level adjusted estimates were derived using multivariate logistic regression. An exchangeable correlation structure was used for correlation in TSH testing among patients belonging to the same FHT. We used an interrupted time-series analysis with a stepwise function22 for monthly TSH testing during the 2 years of interest; January 1, 2017, was the point of intervention, as the groups had been formed, they had met several times, and the initiative had been implemented at all sites. The monthly rate of TSH testing in the 2 cohorts was adjusted for patients’ age and sex.

We calculated differences in testing between 2016 and 2017 by patient characteristics for the 2 groups and compared the differences. We compared patient characteristics known to be associated with greater prevalence of thyroid disorders or increased health care use: age, sex, number of encounters in past 2 years, number of comorbidities, and presence of dementia, obesity, and rheumatoid arthritis.

We calculated the proportions of patients with normal and out-of-range TSH results for the most recent TSH test. We excluded TSH results that were not plausible (above 150 mIU/L).

This study was reviewed and approved by the Research Ethics Board at the North York General Hospital in Ontario. All participating primary care providers have provided written informed consent for the collection and analysis of their EMR data.

RESULTS

A flow diagram for the generation of the cohort is shown in Figure 1. A total of 150 944 patients in the practices of 352 physicians met the inclusion criteria.

Figure 1.
Figure 1.

Generation of the cohort of adult patients without an identified indication for thyroid testing

UTOPIAN—University of Toronto Practice-Based Research Network.

There were 50 351 patients (33.4% of all patients meeting inclusion criteria) in the practices of 119 physicians included in the CWC initiative. Patient and physician characteristics for the 2 groups are shown on Table 1. Physicians participating in the CWC initiative had a greater proportion of female patients, a lower proportion of patients with 10 or more clinical encounters, and a greater proportion of patients with none of the assessed comorbidities.

View this table:
Table 1.

Patient and provider characteristics for groups participating or not participating in the CWC initiative

We found 52 934 patients (35.1% of all patients) with no identified indication for TSH testing who had at least 1 test done in the 2-year period.

Characteristics of patients who had at least 1 TSH test done are shown in Table 2. The percentage of patients tested was greater in women, in patients with a condition of interest, for those with more encounters, and for those with more comorbidities.

View this table:
Table 2.

Testing of TSH levels by patient characteristics during the 2-year interval of interest for providers who did or did not participate in the CWC initiative

Figure 2 provides the adjusted monthly rate of TSH testing before and after January 1, 2017, and 95% confidence intervals. In 2016, 2.17% (95% CI 1.96% to 2.24%) of patients in the group that was not part of the CWC initiative were tested each month, and 1.74% (95% CI 1.58% to 1.89%) of patients in the CWC group were tested monthly. There was an absolute decrease of 0.04% (95% CI −0.18% to 0.11%, P = .62) in the rate of monthly testing after January 1, 2017, in the practices that were not participating in the CWC initiative, a relative reduction of 1.8%. The decrease in the practices participating in the CWC initiative was 0.23% (95% CI −0.13% to −0.23%, P = .0001), a relative reduction of 13.2%. Monthly testing in the CWC group decreased by 0.19% (95% CI −0.02 to −0.35, P = .03) more than in the non-CWC group; the relative difference between the change in the 2 groups was 11.4%.

Figure 2.
Figure 2.

Time series plot for average monthly rate of TSH testing for physicians participating in the CWC initiative and for those not participating in the initiative, adjusted for patient age and sex: The intervention midpoint was defined as January 1, 2017. Rates of TSH testing for physicians participating in the CWC initiative are in red; rates for physicians not participating are in blue.

CWC—Choosing Wisely Canada, TSH—thyroid-stimulating hormone.

Results for subgroup analyses by patient characteristics of changes in mean monthly rates of testing between 2016 and 2017 are shown in Figure 3; detailed results are shown as a supplementary table available from CFPlus.* There were statistically significant decreases in almost all categories for patients of physicians participating in the CWC initiative. This was not the case in the non-CWC group; the only statistically significant decrease was for patients with dementia.

Figure 3.
Figure 3.

Intervention effect of CWC initiative to reduce monthly rates of TSH testing

CPCSSN—Canadian Primary Care Sentinel Surveillance Network, CWC—Choosing Wisely Canada, TSH—thyroid-stimulating hormone.

Table 3 provides information on TSH ranges. More than 95% of tests were within the normal range.

View this table:
Table 3.

Ranges of TSH results for patients tested during the 2-year interval of interest

DISCUSSION

More than a third of all adults with no identified indication for testing in their charts had at least 1 TSH test during the 2 years of observation. We found a decrease in tests with no identified indication in practices participating in the CWC initiative, when compared with practices not participating. Less than 5% of TSH test results were out of the normal range in both groups.

The annual incidence of thyroid dysfunction has been found to be less than 1%.23 In a meta-analysis, this was 0.3%—0.4% for women and 0.09% for men.24 The rate of testing without an identified indication in our study was 35.1% over 2 years, or 17.5% per year. There is likely room to reduce TSH testing; this could be explored by identifying reasons testing was chosen as well as the value of testing for different indications.

Dissemination of guidelines and evidence-based recommendations (eg, mailing out information) without additional change-management activities has been found to be ineffective or of limited effectiveness in changing practice.25,26 Some of the variation in uptake might be explained by variation in the quality of evidence upon which guidelines are based.27 According to Rogers’ theory of diffusion of innovations, uptake is greater if a change is actively promoted by local opinion leaders; is compatible with values of adopters; is not overly complex; and if local adaptation is allowed.28 The ARTIC project deliberately focused on local physician leadership and allowed local customization after agreement on general aims. There was financial and operational support at each site for local leaders, as well as ongoing communication and shared learning between champions at different sites, consistent with recommendations and principles of a Learning Healthcare System.29–31 Supplementary material available from CFPlus* describes activities done for the CWC program.

Although feedback on TSH ordering compared with peers was provided as part of the project, we could not estimate the effect of this component, as the start of feedback was delayed owing to administrative and operational issues. As a result, feedback was only available more than midway through the project, with the first report sent to physicians in April 2017. Audit and feedback has been found to result in a 4.3% improvement (interquartile range 0.5% to 16%)32; it is possible that the reduction in TSH for the group exposed to the intervention would have been greater if feedback on laboratory testing had been available earlier.

Screening of TSH levels might be driven by some factors beyond a physician’s direct control, including patient demand33; thus eliminating screening entirely might be unrealistic. Nonetheless, this project successfully and rapidly lowered the rate of TSH testing for patients with no identified indication.

While we did not study the cost-effectiveness of this intervention, about 18% of all adults in this project received a TSH test that could have been unnecessarily ordered in a 2-year period. The groups implementing CWC recommendations decreased their rate of TSH screening by 11% more than groups not implementing these recommendations. Ontario’s adult population is about 10 million people; a TSH test costs about $14 in Ontario.10 The biannual costs of testing 18% of adults can be estimated as $25 million. An 11% decrease attributed to supporting locally contextualized change-management efforts would save the province approximately $3 million in costs every 2 years for this single laboratory test. This would potentially cover the cost of the initiative, which could be extended to additional CWC recommendations.

Limitations

This was a convenience sample of primary care practices that participated in the CWC initiative and of practices that contributed EMR data to UTOPIAN, rather than a random sample. These physicians and their patients might not represent the general population. Physicians were part of an FHT, where more resources could be devoted to supporting physician leadership. Implementation might be more challenging in other practice settings.

There are limitations in EMR data used for secondary purposes. Specifically, we were not certain that there was an absence of clinical indications warranting TSH testing in all cases considered, as free-text data contained in encounters were not part of our data set. In addition, physicians might not have recorded all reasons for which testing might be indicated in the coded fields that were extracted and processed.

There were differences in patient age and sex between the 2 groups; we controlled for these in the statistical analysis.

We were not able to implement laboratory-based feedback in a timely manner; this might have biased the results toward the null hypothesis.

Physicians in the CWC initiative were receiving laboratory feedback and thus will have known that they were being measured, which might have introduced a Hawthorne effect: the behaviour of the CWC cohort might have been influenced partly by the knowledge that they were being observed.34

The cost-effectiveness of this approach was not studied as part of this analysis.

However, the study also had several strengths. It reflected data from care routinely provided to patients. Family physicians were not required to collect any additional data, so there was no additional work for participants related to the measurement aspect of the project.

Conclusion

There appears to be overuse of TSH testing in primary care practices in Ontario, consistent with findings in other countries.7,8 This evaluation suggests that an initiative based on CWC recommendations and using effective change-management strategies in primary care was associated with a reduction in TSH tests.

Further evaluations of this approach might be worthwhile. These would determine conditions needed for further spread of the TSH testing improvement initiative, as well as additional initiatives that target other CWC recommendations.

Acknowledgments

We thank the primary care leads at each site and the family physicians who contributed the electronic medical record data that made this study possible. This study was supported by Health Quality Ontario through the ARTIC (Adopting Research to Improve Care) program. Dr Greiver holds an investigator award from the Department of Family and Community Medicine at the University of Toronto and was supported by a research stipend from North York General Hospital.

Notes

Editor’s key points

  • ▸ Population-based screening of asymptomatic adults for thyroid disorders is not recommended, as there is no evidence that this is beneficial. Choosing Wisely Canada (CWC) has implemented an initiative to improve thyroid-stimulating hormone (TSH) test ordering for patients with no identified indication. This study compares results from 4 family health teams participating in the initiative with those of teams not participating.

  • ▸ More than a third of all adults with no identified indication for testing in their charts had at least 1 TSH test during the 2 years of observation. The authors found a decrease in tests with no identified indication in practices participating in the CWC initiative, when compared with practices not participating. Less than 5% of TSH test results were out of the normal range in both groups.

  • ▸ There appears to be overuse of TSH testing in primary care practices in Ontario. The CWC initiative was effective in reducing apparently inappropriate TSH testing. Although a cost-effective analysis was not performed, the authors estimate that as a result of such interventions approximately $1.5 million per year could be saved on this single laboratory test in Ontario.

Points de repère du rédacteur

  • ▸ Il n’est pas recommandé de procéder au dépistage systématique de troubles thyroïdiens chez des adultes asymptomatiques, étant donné l’absence de données étayant ses bienfaits. Choisir avec soin Canada (CSC) a mis en œuvre une initiative visant à améliorer les pratiques de prescription du dosage de la thyréostimuline (TSH) pour des patients ne présentant aucune indication. Cette étude compare les résultats de 4 équipes de santé familiale participant à l’initiative avec les résultats d’équipes non participantes.

  • ▸ Plus du tiers de tous les adultes sans indication identifiée dans leur dossier avaient subi 1 dosage de la TSH durant les 2 années d’observation. Les auteurs ont observé, dans les équipes participant à l’initiative de CSC, une diminution des tests non indiqués, par rapport aux pratiques dans les équipes non participantes. Moins de 5 % des résultats du dosage de la TSH ne se situaient pas dans la normale, et ce, dans les 2 groupes.

  • ▸ Il semble qu’il y ait une surutilisation du dosage de la TSH dans les équipes de soins primaires en Ontario. L’initiative de CSC s’est révélée efficace pour réduire les tests de la TSH apparemment inappropriés. Même si une analyse de la rentabilité n’a pas été effectuée, les auteurs estiment que grâce à de telles interventions, il serait possible d’économiser environ 1,5 million de dollars par année en dépenses pour cette seule analyse en laboratoire en Ontario.

Footnotes

  • ↵* Information on elements of the Choosing Wisely Canada initiative, an example of a laboratory report, details of the electronic medical record search criteria, and subgroup analyses are available from www.cfp.ca. Go to the full text of the article online and click on the CFPlus tab.

  • Contributors

    Dr Wintemute led the Choosing Wisely Canada project implementation in the primary care sites. Drs Wintemute and Greiver were responsible for the conception of the article. Dr Aliarzadeh was responsible for acquisition of data, supporting analysis, and project design. Mr Kalia and Dr Moineddin contributed substantially to the analysis of data. Dr Greiver drafted the initial version of the article. All authors contributed to the interpretation of data. All authors reviewed and revised the article for important intellectual content and gave final approval of the version to be published.

  • Competing interests

    None declared

  • This article has been peer reviewed.

  • Cet article a fait l’objet d’une révision par des pairs.

References

  1. 1.↵
    Thyroid function tests in the diagnosis and monitoring of adults. Vancouver, BC: Province of British Columbia; 2010. Available from: www2.gov.bc.ca/gov/content/health/practitioner-professional-resources/bc-guidelines/thyroid-testing#two. Accessed 2017 Mar 25.
  2. 2.↵
    1. LeFevre ML,
    2. US Preventive Services Task Force
    . Screening for thyroid dysfunction: US Preventive Services Task Force recommendation statement. Ann Intern Med 2015;162(9):641-50.
    OpenUrlCrossRefPubMed
  3. 3.↵
    1. College of Family Physicians of Canada
    . Family medicine. Thirteen things physicians and patients should question. Toronto, ON: Choosing Wisely Canada; 2014. www.choosingwiselycanada.org/recommendations/family-medicine/. Accessed 2017 Mar 25.
  4. 4.↵
    1. Dufour DR
    . Laboratory tests of thyroid function: uses and limitations. Endocrinol Metab Clin North Am 2007;36(3):579-94.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Anderson GM,
    2. Chan B,
    3. Carter JA,
    4. Axcell T
    . An overview of trends in the use of acute care hospitals, physician and diagnostic services, and prescription drugs. In: Goel V, Williams JI, Anderson GM, Blackstien-Hirsch P, Fooks C, Naylor CD, editors. Patterns of health care in Ontario: the ICES practice atlas. 2nd ed. Ottawa, ON: Canadian Medical Association; 1996.
  6. 6.↵
    1. INFOBulletin 10030
    . Amendments to laboratory requisition form #4422-84. Toronto, ON: Ontario Ministry of Health and Long-Term Care; 2012. Available from: www.health.gov.on.ca/en/pro/programs/ohip/bulletins/10000/bul10030.pdf. Accessed 2017 Mar 25.
  7. 7.↵
    1. Vaidya B,
    2. Ukoumunne OC,
    3. Shuttleworth J,
    4. Bromley A,
    5. Lewis A,
    6. Hyde C,
    7. et al
    . Variability in thyroid function test requests across general practices in south-west England. Qual Prim Care 2013;21(3):143-8.
    OpenUrl
  8. 8.↵
    1. Werhun A,
    2. Hamilton W
    . Thyroid function testing in primary care: overused and under-evidenced? A study examining which clinical features correspond to an abnormal thyroid function result. Fam Pract 2015;32(2):187-91. Epub 2015 Mar 17.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. O’Sullivan JW,
    2. Albasri A,
    3. Nicholson BD,
    4. Perera R,
    5. Aronson JK,
    6. Roberts N,
    7. et al
    . Over-testing and undertesting in primary care: a systematic review and meta-analysis. BMJ Open 2018;8(2):e018557.
    OpenUrlAbstract/FREE Full Text
  10. 10.↵
    1. Birk-Urovitz E,
    2. Del Giudice EM,
    3. Meaney C,
    4. Grewal K
    . Use of thyroid-stimulating hormone tests for identifying primary hypothyroidism in family medicine patients. Can Fam Physician 2017;63:e389-94. Available from: www.cfp.ca/content/cfp/63/9/e389.full.pdf. Accessed 2019 Oct 15.
    OpenUrlAbstract/FREE Full Text
  11. 11.↵
    1. Rosser WW,
    2. Colwill JM,
    3. Kasperski J,
    4. Wilson L
    . Patient-centered medical homes in Ontario. N Engl J Med 2010;362(3):e7. Epub 2010 Jan 6.
    OpenUrlCrossRefPubMed
  12. 12.↵
    Choosing Wisely: an idea worth spreading. Toronto, ON: Health Quality Ontario; 2018. Available from: www.hqontario.ca/Quality-Improvement/Quality-Improvement-in-Action/ARTIC/ARTIC-Projects/Choosing-Wisely. Accessed 2018 Jul 3.
  13. 13.↵
    1. Von Elm E,
    2. Altman DG,
    3. Egger M,
    4. Pocock SJ,
    5. Gøtzsche PC,
    6. Vandenbroucke JP,
    7. et al
    . The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Int J Surg 2014;12(12):1495-9.
    OpenUrlCrossRefPubMed
  14. 14.↵
    ARTIC. Adopting Research To Improve Care. Toronto, ON: Health Quality Ontario; 2018. Available from: www.hqontario.ca/Quality-Improvement/Quality-Improvement-in-Action/ARTIC. Accessed June 17, 2018.
  15. 15.↵
    1. North York General Hospital [website]
    . Joint Centres for Transformative Healthcare Innovation. North York, ON: North York General Hospital; 2018. Available from: https://www.nygh.on.ca/education-and-research/joint-centres-transformativehealthcare-innovation. Accessed 2019 Oct 15.
  16. 16.↵
    UTOPIAN [website]. Toronto, ON: University of Toronto; 2017. Available from: www.dfcm.utoronto.ca/landing-page/utopian. Accessed 2017 Apr 25.
  17. 17.↵
    1. Birtwhistle R,
    2. Keshavjee K,
    3. Lambert-Lanning A,
    4. Godwin M,
    5. Greiver M,
    6. Manca D,
    7. et al
    . Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. J Am Board Fam Med 2009;22(4):412-22.
    OpenUrlAbstract/FREE Full Text
  18. 18.↵
    1. Birtwhistle RV
    . Canadian Primary Care Sentinel Surveillance Network. A developing resource for family medicine and public health. Can Fam Physician 2011;57:1219-20. (Eng), e401–2 (Fr).
    OpenUrlFREE Full Text
  19. 19.↵
    UTOPIAN geographical distribution. 2018. batchgeo.com.
  20. 20.↵
    1. Agniel D,
    2. Kohane IS,
    3. Weber GM
    . Biases in electronic health record data due to processes within the healthcare system: retrospective observational study. BMJ 2018;361:k1479. Erratum in: BMJ 2018;363:k4416.
    OpenUrlAbstract/FREE Full Text
  21. 21.↵
    1. Williamson T,
    2. Birtwhistle R,
    3. Green M,
    4. Khan S,
    5. Wong S,
    6. Natarajan N,
    7. et al
    . Validating the CPCSSN algorithms for eight chronic diseases. Paper presented at: Family Medicine Forum; 2013 Nov 6; Vancouver, BC.
  22. 22.↵
    1. Wagner AK,
    2. Soumerai SB,
    3. Zhang F,
    4. Ross-Degnan D
    . Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther 2002;27(4):299-309.
    OpenUrlCrossRefPubMed
  23. 23.↵
    1. Vanderpump MP,
    2. Tunbridge WM,
    3. French JM,
    4. Appleton D,
    5. Bates D,
    6. Clark F,
    7. et al
    . The incidence of thyroid disorders in the community: a twenty-year follow-up of the Whickham Survey. Clin Endocrinol (Oxf) 1995;43(1):55-68.
    OpenUrlCrossRefPubMed
  24. 24.↵
    1. Garmendia Madariaga A,
    2. Santos Palacios S,
    3. Guillen-Grima F,
    4. Galofre JC
    . The incidence and prevalence of thyroid dysfunction in Europe: a meta-analysis. J Clin Endocrinol Metab 2014;99(3):923-31. Epub 2014 Jan 1.
    OpenUrlCrossRefPubMed
  25. 25.↵
    1. Zwarenstein M,
    2. Grimshaw JM,
    3. Presseau J,
    4. Francis JJ,
    5. Godin G,
    6. Johnston M,
    7. et al
    . Printed educational messages fail to increase use of thiazides as first-line medication for hypertension in primary care: a cluster randomized controlled trial [ISRCTN72772651]. Implement Sci 2016;11(1):124.
    OpenUrl
  26. 26.↵
    1. Grimshaw JM,
    2. Thomas RE,
    3. MacLennan G,
    4. Fraser C,
    5. Ramsay CR,
    6. Vale L,
    7. et al
    . Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8(6):iii-iv, 1-72.
    OpenUrlCrossRefPubMed
  27. 27.↵
    1. O’Sullivan JW,
    2. Albasri A,
    3. Koshiaris C,
    4. Aronson J,
    5. Heneghan C,
    6. Perera R
    . Diagnostic test guidelines based on high-quality evidence had greater rates of adherence: a metaepidemiological study. J Clin Epidemiol 2018;103:40-50. Epub 2018 Jul 5.
    OpenUrlCrossRefPubMed
  28. 28.↵
    1. Rogers E
    . Diffusion of innovations. 4th ed. New York, NY: The Free Press; 1995.
  29. 29.↵
    1. Harris SB,
    2. Green ME,
    3. Brown JB,
    4. Roberts S,
    5. Russell G,
    6. Fournie M,
    7. et al
    . Impact of a quality improvement program on primary healthcare in Canada: a mixed-method evaluation. Health Policy 2015;119(4):405-16. Epub 2014 Nov 5.
    OpenUrlCrossRefPubMed
  30. 30.
    1. Wells S,
    2. Tamir O,
    3. Gray J,
    4. Naidoo D,
    5. Bekhit M,
    6. Goldmann D
    . Are quality improvement collaboratives effective? A systematic review. BMJ Qual Saf 2018;27(3):226-40. Epub 2017 Oct 21.
    OpenUrlAbstract/FREE Full Text
  31. 31.↵
    1. Institute for Healthcare Improvement
    . The Breakthrough Series: IHI’s collaborative model for achieving breakthrough improvement. Boston, MA: Institute for Healthcare Improvement; 2003.
  32. 32.↵
    1. Ivers N,
    2. Jamtvedt G,
    3. Flottorp S,
    4. Young JM,
    5. Odgaard-Jensen J,
    6. French SD,
    7. et al
    . Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;(6):CD000259.
  33. 33.↵
    1. Kravitz RL,
    2. Bell RA,
    3. Azari R,
    4. Kelly-Reif S,
    5. Krupat E,
    6. Thom DH
    . Direct observation of requests for clinical services in office practice: what do patients want and do they get it? Arch Intern Med 2003;163(14):1673-81.
    OpenUrlCrossRefPubMed
  34. 34.↵
    1. McCambridge J,
    2. Witton J,
    3. Elbourne DR
    . Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol 2014;67(3):267-77. Epub 2013 Nov 22.
    OpenUrlCrossRefPubMed
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Respond to this article
Choosing Wisely Canada campaign associated with less overuse of thyroid testing
Kimberly Wintemute, Michelle Greiver, Warren McIsaac, M. Elisabeth Del Giudice, Frank Sullivan, Babak Aliarzadeh, Sumeet Kalia, Chris Meaney, Rahim Moineddin, Alexander Singer
Canadian Family Physician Nov 2019, 65 (11) e487-e496;
Twitter logo Facebook logo Mendeley logo

Jump to section

Subjects