Research ArticleResearch

Improving prevention in primary care

Evaluating the sustainability of outreach facilitation

William Hogg, Jacques Lemelin, Isabella Moroz, Enrique Soto and Grant Russell
Canadian Family Physician May 2008; 54 (5) 712-720;

This article has a correction. Please see:

ABSTRACT

OBJECTIVE To assess the extent to which advances in preventive care delivery, achieved in primary care practices through outreach facilitation, could be sustained over time after purposefully redirecting the focus of practice physicians and staff away from prevention and toward a new content area in need of improvement—chronic illness management.

DESIGN Before-and-after study.

SETTING Primary care networks and family health networks in Ontario.

PARTICIPANTS A volunteer sample of 30 primary care practices recruited from 99 eligible sites.

INTERVENTION Outreach visits directed at modifying physician behaviour were delivered by trained nurse facilitators using practice-tailored systems strategies. For the first 12 months, the intervention focused on improving delivery of preventive care, after which facilitation of chronic illness management was introduced for another 3 to 9 months.

MAIN OUTCOME MEASURES Changes in practices’ performance rates for selected preventive maneuvers (according to recommendations of the Canadian Task Force on Preventive Health Care) between baseline and follow-up, conducted 3 to 9 months after the end of the prevention intervention, measured from chart reviews for those maneuvers likely to be recorded and from telephoneinterviews with patients for lifestyle counseling.

RESULTS Four of the 30 practices dropped out of the study. In the remaining practices, at the postintervention follow-up, there was an increase in the delivery of the appropriate grade A (19.3%, 95% confidence interval [CI] 10.4% to 28.3%) and B (9.3%, 95% CI 5.4% to 13.2%) maneuvers, accompanied by a reduction in inappropriate grade D maneuvers (−15.9%, 95% CI −22.1% to −9.6%), for an absolute improvement of 12% (P < .0001) in the overall preventive care performance, as determined by a chart audit. We found no changes in the provision of lifestyle counseling maneuvers measured from telephone interviews with patients (1.3%, 95% CI 1.0% to 3.7%).

CONCLUSION The tailored, multifaceted intervention delivered by nurse facilitators was effective in producing significant improvements in preventive care performance that extended beyond the prevention intervention period.

Outreach facilitation is one of the most effective methods of implementing best practices in delivery of preventive health services in primary care.1–3 It employs individuals with a nursing background who provide prevention performance feedback to a practice, build consensus on improvement goals, and, through regular visits, support the practice by following a systems strategy tailored to that practice. We previously demonstrated, in a randomized trial within community-based family practices in Ontario, that outreach facilitation leads to substantial improvements in physician practice patterns and preventive care performance.4 Furthermore, in a recent economic evaluation, we demonstrated substantial net savings as a result of reducing inappropriate practice behaviour and increasing appropriate preventive screening.5

The sustainability of the improved care has been pursued in several studies, but only a few investigations of office-system interventions have been subjected to long-term analysis. Varying degrees of sustainability were observed as early as 6 months and as late as 5 years after the end of the interventions6–10 and were attributed to factors such as practice-individualized approaches (allowing for integration of change within day-to-day routines of the practice), provision of feedback, and physicians’ preventive care philosophy. In many cases, sustainability of initial success depended on continued assistance.11–13 An important concern about the postintervention maintenance of preventive care, which was not addressed in these investigations, is the effect of multiple competing demands, which often divert practitioners’ time and energy away from prevention and toward other activities. The goal of the present investigation was therefore to directly examine this possibility by first implementing preventive care strategies and then engaging the practices in a distractor activity. In particular, we assessed the extent to which gains in preventive performance achieved through outreach facilitation could be maintained after a period of time when the focus and attention of the practice physicians and staff members were purposefully redirected away from preventive care and toward chronic illness care.

METHODS

Study design

We conducted a before-and-after study to gauge the sustainability of an outreach facilitation program. The progress of the study is shown in Figure 1. The unit of intervention and analysis was the primary care practice and the units of observation for outcome assessment were obtained from medical charts and telephone interviews with patients. The study was approved by the Ottawa Hospital Research Ethics Board.

Figure 1
Figure 1

Progress during study: Because practices were recruited over time, actual dates of data collection, preventive health care intervention initiation, and implementation of the chronic illness care management distractor activity differed across practices.

Selection and recruitment of practices

The intervention involved primary care network (PCN) and family health network (FHN) practices in Ontario, which are 2 of the newer models of primary health care delivery in Ontario. Primary care networks were introduced in May 1998 and served as pilot sites for the FHNs, which were introduced in March 2001. Both models are characterized by patient rostering (with an average roster size of approximately 1500 patients per physician), capitation payment structure with added incentives for prevention and other targeted services, provision of after-hours service and teletriage, and support for information technology. Of the 30 recruited practices, 25 (83%) were from established PCNs that had existed for about 4 to 5 years at the time of the study, protecting the study against the confounding effects of transitioning into a new model of care. Solo practitioners and those who practised in groups of up to 10 were considered eligible to participate in the trial. Practices with more than 10 physicians or with an academic affiliation were excluded owing to their size and resulting complexity. We recruited practices by repeated mailings followed, when necessary, by a telephone call from a physician recruiter or principal investigator until the required number of practices provided informed consent to participate in the study. The recruitment process lasted from September 2003 to May 2004.

Intervention

Three nurses with master’s degrees were employed as prevention facilitators. Following training, facilitators were each assigned up to 11 practices, according to geographic proximity to their residence. The facilitators attempted to visit the practices approximately every 3 to 4 weeks and delivered primarily 3 intervention strategies that we found to be particularly effective for improving preventive care: audit and feedback, consensus building, and reminder systems (Table 1).10

View this table:
Table 1

Outreach facilitation program activities and intervention strategies

The prevention facilitation phase lasted 12 months or until the facilitators felt that practices had maximized the improvement potential of preventive care delivery. It was followed by a distractor phase—chronic illness care management (CICM)—to redirect the focus of the physicians and staff members away from preventive care (Figure 2* presents a timeline of the intervention activities). This allowed for an assessment of the extent to which the increase in preventive care performance could be maintained over time in the face of competing demands placed on physicians. The CICM phase lasted between 3 and 9 months and was followed immediately by collection of the final outcome measures.

Collection of outcome data

Data on practice profiles and physician characteristics were collected by facilitators during the initial meetings with the practices.

Trained auditors collected indices of practices’ preventive performance from chart reviews and telephone interviews with patients conducted just before the prevention facilitation phase and immediately after the CICM distractor phase. No data of that nature were collected at the end of the prevention facilitation phase. Details about the data collection process are available.* In brief, we chose 26 preventive maneuvers from grades A, B, and D, as recommended by the Canadian Task Force on Preventive Health Care.14 We limited patient eligibility to age 17 and older. Standard instruments for both chart reviews and patient interviews were used, and confidentiality was ensured.

Outcome measures

The primary outcome measure was the change in the delivery of the recommended and nonrecommended preventive maneuvers from baseline (January 2004) to the end of the intervention period (December 2005), calculated as the number of appropriate preventive maneuvers completed plus the number of inappropriate maneuvers properly avoided, divided by the number of eligible preventive maneuvers, as determined from chart audit. The patient-level summary measures were aggregated for each practice to generate practice summary scores, which served as the basis for comparison of practice preventive performance at baseline and postintervention.

The secondary outcome measure was operationally defined in an identical manner, except it utilized the information from patient telephone interviews regarding practice preventive performance of maneuvers that focused on healthy lifestyle counseling.

In addition, secondary analyses were conducted on individual maneuvers to determine which were most affected by the intervention.

Sample size

We estimated that a sample size of 21 practices at each time point would have 80% power to detect a 10% mean difference between the prevention performance at pre-intervention baseline and after the intervention, at a .05 level of significance by a 2-tailed test. Previous similar research indicated that 30 charts and 50 patient interviews per practice at each data collection point were needed to adequately quantify a practice’s preventive performance. However, to increase the credibility and strength of the preintervention audit and feedback component of the intervention, auditors and interviewers aimed to complete 70 chart reviews and 70 patient interviews per practice at preintervention baseline.

Statistical analyses

Invalid and illogical data were identified and rectified before all analyses. Descriptive statistics were generated for practice characteristics, as well as the means and 95% confidence intervals for all outcomes. Paired t tests were used to determine if preintervention and postintervention means differed significantly (2-tailed α = .05). In cases where the normality assumption was violated, appropriate nonparametric tests were used: the Wilcoxon signed rank test, if the distribution of change scores was symmetric, or the sign test, which makes no distributional assumptions.15 All analyses were performed using SAS 9.1.16

RESULTS

From the 13 PCNs and 5 FHNs, which made up 99 primary care practice sites in Ontario, 30 practice sites with 58 physicians initially agreed to participate. Baseline practice characteristics are presented in Table 2. Most recruited practices were urban, solo, consisted of male physicians only, and had at least 1 full-time nurse on staff. A total of 63% of practices had electronic medical records. Most were using various kinds of reminder systems for preventive care. Particularly noteworthy was the high percentage of practices with personnel dedicated specifically to looking after prevention activities. Three of the 30 practices withdrew from the study during the baseline data collection phase and 1 during the intervention phase, citing lack of time as a main reason.

View this table:
Table 2

Practice characteristics at baseline: N=30.

The mean proportions of eligible patients who received the preventive maneuvers, as determined by chart audit, are shown in Table 3. Significant improvement in performance at follow-up, conducted after the 3- to 9-month distraction period following the prevention intervention, was observed for both grade A (19.3%, 95% confidence interval [CI] 10.4% to 28.3%) and grade B (9.3%, 95% CI 5.4% to 13.2%) maneuvers, and consequently for grades A and B combined (12.3%, 95% CI 8.1% to 16.5%). In particular, colon cancer screening quadrupled (P < .0001), while visual acuity and hearing tests more than doubled in magnitude (P < .01). At the same time, the inappropriate grade D maneuver of urine dipstick testing for proteinuria significantly decreased by 15.9% (95% CI 9.6% to 22.1%). The simultaneous increase in the delivery of appropriate screening maneuvers and decrease in inappropriate maneuvers resulted in a 12% absolute improvement in the overall preventive care performance or composite primary outcome prevention index (P < .0001).

View this table:
Table 3

Preventive maneuvers performed before and after the intervention, as determined by chart audit

Table 4 presents the mean proportion of eligible patients who received preventive counseling on maneuvers related to healthy lifestyle as determined from telephone interviews with patients. The composite secondary outcome prevention index showed no significant differences in the overall performance related to provision of lifestyle counseling (1.3%, 95% CI −1.0% to 3.7%). Whether considering the appropriate (grade A and B) or inappropriate (grade D) health promotion counseling maneuvers, individually or in aggregate, no statistically significant mean increases were noted.

View this table:
Table 4

Preventive maneuvers performed before and after the intervention, as determined by patient interviews

DISCUSSION

We report that a 12-month, tailored, multifaceted intervention delivered by outreach facilitators was effective in producing longer-term improvements in the delivery of preventive care maneuvers, as measured by chart audit. These improvements, observed at the follow-up conducted after the 3- to 9-month distraction period following the prevention facilitation, are of the same magnitude as those reported immediately after the end of an 18-month intervention study by Lemelin et al,4 and are comparable to those reported in other studies that tested the long-term sustainability of office-systems interventions.7,17,18 The present study lends further credence to the potential of outreach facilitation to produce lasting changes in physician behaviour. In contrast to our main outcome results, there were no changes from baseline observed at the follow-up in the implementation of preventive maneuvers related to lifestyle counseling, assessed via telephone interviews with patients.

Our study has several strengths. It was conducted in community practices across Ontario and targeted a large number of preventive services simultaneously. The observed improvements in preventive care delivery occurred in a sample of practices whose baseline preventive performance was already quite high. By creating a distraction within the practices—the CICM phase—our design addressed the concept of a permanent facilitation program, where the same facilitator works with a practice on a series of quality improvement initiatives. For example, once the facilitator and practice have addressed one content area (preventive medicine) they could change topic (chronic illness) and work together to improve clinical practice in a second area and so on. Finally, our main outcomes were obtained from chart reviews, which are considered more reliable and more accurate sources of data compared with physician or patient reports.19

Limitations

Our findings must be interpreted in the context of the study’s limitations. The most important is the lack of a comparison or control group. Given that the study was conducted in the midst of primary care renewal, the observed rates might reflect concurrent trends toward improved delivery of clinical preventive services and not be directly attributable to the outreach facilitation intervention. For example, there is evidence suggesting that pay-for-performance and computerized decision systems can lead to improvements in the provision of preventive services.20,21 It is unlikely, however, that these factors affected our results, as the performance of those preventive maneuvers where performance bonuses were offered (mammography, Papanicolaou smears, and influenza vaccine) did not substantially improve. With respect to computerization, even though 63% of participating practices had electronic medical records, they were not equipped with features to support comprehensive reminder systems at the time of the study. Another limitation has to do with the timing of publication of the CTFPHC’s recommendation for colon cancer screening in 2001, just before our study. The performance of this preventive maneuver (colon cancer screening) showed the greatest improvement, which could be attributed to the “diffusion of innovation” of this new Canadian guideline.22 What makes this explanation unlikely, however, are the results of studies conducted after the publication of the guideline, which consistently demonstrated very low screening rates and lack of awareness among physicians of the efficacy of colorectal screening tests.23–27 Finally, it is possible that the several other improvement interventions, which were being conducted at the same time as our study as part of primary care renewal, could have created a state of awareness that generally increased the preventive performance in the province. There are no published data at present to confirm this suspicion. Other limitations include using the same nurse facilitators to introduce practices to chronic illness management. The continued presence of the facilitator in the practices might have served as a reinforcer, helping to sustain the improved preventive performance rates. This intriguing possibility is a high priority for future research, as it is critical to determine just how much external facilitation is needed and how long it will take before practices can support and maintain effective prevention care delivery on their own.

The generalizability of our findings is limited by the study’s focus on PCN and FHN practices only, which comprised about a third of practices in Ontario at the time of the study. Comparing the study’s physicians with Ontario FPs or GPs, using results from the 2004 National Physician Survey,28 revealed that a higher percentage of our practices were solo (70% vs 34.7%), used electronic patient scheduling systems (76% vs 49.9%), had electronic medical records (63% vs 22.5%), and had Internet access (83% vs 60.7%). Furthermore, we only measured preventive performance at 2 points in time. The number of practices needed to detect changes at 3 points in time, with an adequate power, increases substantially and would have exceeded the financial resources for the project. Finally, although our results are encouraging, a 3- to 9-month follow-up after the end of the intervention period might not be enough to draw valid conclusions about the intervention’s long-term effects. The lack of improvement in the delivery of preventive lifestyle counseling, observed on the basis of patient reports, could be a result of several factors, such as the already high preventive performance of participating practices at baseline and the well documented limitations of patient surveys.29,30

Conclusion

The results of our study provide reason to be optimistic, as they suggest that sustainability of outreach facilitation beyond the end of an intervention is feasible in the volatile environment of a busy primary care practice. Also important, we were able to show improvements in delivery of preventive services despite the already high baseline proportions of patients appropriately serviced. More longitudinal research, with randomized controlled designs, should be conducted to support the validity and long-term efficiency of the outreach facilitation implementation. A follow-up study of our clinics would be desirable to explore the lifespan of the observed improvements.

Acknowledgment

Funding for this research was provided by the Ontario Ministry of Health and Long-Term Care Primary Health Care Transition Fund. The views expressed in this report are the views of the authors and do not necessarily reflect those of the Ontario Ministry of Health and Long-Term Care.

Notes

EDITOR’S KEY POINTS

  • Outreach facilitation is an effective method of implementing best practice in preventive care delivery in primary care, but are the improvements sustainable, particularly with the many competing demands of clinical practice?

  • In this study, set in 2 primary care practice models, after 12 months of outreach facilitation that focused on preventive care, the researchers purposefully redirected the attention of staff and physicians away from prevention and toward chronic illness management.

  • The researchers found that there was significant improvement in the delivery of preventive care, even after the 3- to 9-month period of redirected care focus. It is not known how long these results will be sustained.

POINTS DE REPÈRE DU RÉDACTEUR

  • La facilitation de proximité est efficace pour améliorer la dispensation des soins préventifs dans les soins primaires, mais ces changements sontils durables, compte tenu notamment des multiples exigencies de la pratique clinique?

  • Dans cette étude portant sur 2 modèles d‘établis-sements de soins primaires, après 12 mois de facilitation de proximité axée sur les soins préventifs, les chercheurs ont intentionnellement redirigé l’attention du personnel et des médecins vers le traitement des maladies chroniques plutôt que sur la prévention.

  • Une amélioration significative de la dispensation des soins préventifs a été observée, même après 3–9 mois d’interventions portant sur un autre sujet. On ignore combien de temps ces résultats dureront.

Footnotes

  • Contributors

    Drs Hogg, Lemelin, Moroz, Soto, and Russell contributed to concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.

  • Competing interests

    None declared

  • ↵* Figure 2 and details about the data collection process are available at www.cfp.ca. Go to the full text of this article on-line, then click on CFPlus in the menu at the top right-hand side of the page.

  • Cet article a fait l’objet d’une révision par des pairs.

  • This article has been peer reviewed.

References

  1. ↵
    FullardEFowlerGGrayMPromoting prevention in primary care: controlled trial of low technology, low cost approachBr Med J (Clin Res Ed)1987294657910802
    OpenUrlPubMed
  2. NagykaldiZMoldJWAspyCBPractice facilitators: a review of the literatureFam Med20053785818
    OpenUrlPubMed
  3. ↵
    Thomson O’BrienMAOxmanADDavisDAHaynesRBFreemantleNHarveyELEducational outreach visits: effects on professional practice and health care outcomesCochrane Database Syst Rev20002CD000409
  4. ↵
    LemelinJHoggWBaskervilleNEvidence to action: a tailored multifaceted approach to changing family physician practice patterns and improving preventive careCMAJ2001164675763
    OpenUrlAbstract/FREE Full Text
  5. ↵
    HoggWBaskervilleNLemelinJCost savings associated with improving appropriate and reducing inappropriate preventive care: cost-consequences analysisBMC Health Serv Res20055120
    OpenUrlCrossRefPubMed
  6. ↵
    CooleyKAFramePSEberlySWAfter the grant runs out. Long-term provider health maintenance compliance using a computer-based tracking systemArch Fam Med199981137
    OpenUrlCrossRefPubMed
  7. ↵
    DietrichAJSoxCHTostesonTDWoodruffCBDurability of improved physician early detection of cancer after conclusion of intervention supportCancer Epidemiol Biomarkers Prev19943433540
    OpenUrlAbstract
  8. GeboersHvan der HorstMMokkinkHvan MontfortPvan den BoschWvan den HoogenHSetting up improvement projects in small scale primary care practices: feasibility of a model for continuous quality improvementQual Health Care1999813642
    OpenUrlAbstract
  9. RebelskyMSSoxCHDietrichAJSchwabBRLabareeCEBrown-McKinneyNPhysician preventive care philosophy and the five year durability of a preventive services office systemSoc Sci Med1996437107381
    OpenUrlCrossRefPubMed
  10. ↵
    BaskervilleNBHoggWLemelinJProcess evaluation of a tailored multifac-eted approach to changing family physician practice patterns improving preventive careJ Fam Pract2001503W2429
    OpenUrlPubMed
  11. ↵
    McCowanCNevilleRGCrombieIKClarkRAWarnerFCThe facilitator effect: results from a four-year follow-up of children with asthmaBr J Gen Pract19974741615660
    OpenUrlAbstract/FREE Full Text
  12. MelnikowJKohatsuNDChanBKPut prevention into practice: a controlled evaluationAm J Public Health2000901016225
    OpenUrlPubMed
  13. ↵
    TierneyWMMillerMEMcDonaldCJThe effect on test ordering of informing physicians of the charges for outpatient diagnostic testsN Engl J Med19903222114991504
    OpenUrlCrossRefPubMed
  14. ↵
    Canadian Task Force on Preventive Health CareEvidence-based clinical preventionLondon, ONCanadian Task Force on Preventive Health Care2004Available from: www.ctfphc.orgAccessed 2007 Aug 1
  15. ↵
    GauvreauKPaganoMNonparametric tests of hypothesisNutrition199395477
    OpenUrlPubMed
  16. ↵
    SAS InstituteSAS. Version 9.1Cary, NCSAS Institute2004
  17. ↵
    StangeKCGoodwinMAZyzanskiSJDietrichAJSustainability of a practice-individualized preventive service delivery interventionAm J Prev Med2003254296300
    OpenUrlCrossRefPubMed
  18. ↵
    GottliebNHHuangPPBlozisSAGuoJLMurphySThe impact of Put Prevention into Practice on selected clinical preventive services in five Texas sitesAm J Prev Med20012113540
    OpenUrlCrossRefPubMed
  19. ↵
    MontañoDEPhillipsWRCancer screening by primary care physicians: a comparison of rates obtained from physician self-report, patient survey, and chart auditAm J Public Health1995856795800
    OpenUrlPubMed
  20. ↵
    RosenthalMBFrankRGLiZEpsteinAMEarly experience with pay-for-performance: from concept to practiceJAMA200529414178893
    OpenUrlCrossRefPubMed
  21. ↵
    GargAXAdhikariNKMcDonaldHRosas-ArellanoMPDevereauxPJBeyeneJEffects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic reviewJAMA200529310122338
    OpenUrlCrossRefPubMed
  22. ↵
    RogersEMDiffusion of innovations5New York, NYFree Press2003
  23. ↵
    RabeneckLPaszatLFA population-based estimate of the extent of colorectal cancer screening in OntarioAm J Gastroenterol200499611414
    OpenUrlCrossRefPubMed
  24. GoelVGrayRChartPFitchMSaibilFZdanowiczYPerspectives on colorectal cancer screening: a focus group studyHealth Expect2004715160
    OpenUrlPubMed
  25. MadlenskyLMcLaughlinJGoelVA comparison of self-reported colorec-tal cancer screening with medical recordsCancer Epidemiol Biomarkers Prev20031276569
    OpenUrlAbstract/FREE Full Text
  26. McGregorSEHilsdenRJMurrayABryantHEColorectal cancer screening: practices and opinions of primary care physiciansPrev Med200439227985
    OpenUrlCrossRefPubMed
  27. ↵
    McGregorSEHilsdenRJLiFXBryantHEMurrayALow uptake of colorectal cancer screening 3 yr after release of national recommendations for screeningAm J Gastroenterol20071028172735
    OpenUrlCrossRefPubMed
  28. ↵
    College of Family Physicians of Canada, Canadian Medical Association, Royal College of Physicians and Surgeons of CanadaNational physician survey, 2004 resultsMississauga, ONNational Physician Survey2004Available from: www.nationalphysiciansurvey.caAccessed 2007 Aug 2
  29. ↵
    GordonNPHiattRALampertDIConcordance of self-reported data and medical record audit for six cancer screening proceduresJ Natl Cancer Inst199385756670
    OpenUrlAbstract/FREE Full Text
  30. ↵
    TisnadoDMAdamsJLLiuHDambergCLChenWPHuFAWhat is the concordance between the medical record and patient self-report as data sources for ambulatory care?Med Care200644213240
    OpenUrlCrossRefPubMed
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Respond to this article
Improving prevention in primary care
William Hogg, Jacques Lemelin, Isabella Moroz, Enrique Soto, Grant Russell
Canadian Family Physician May 2008, 54 (5) 712-720;
Twitter logo Facebook logo Mendeley logo

Jump to section

Subjects