Research ArticleResearch

Evaluating the implementation of collaborative teams in community family practice using the Primary Care Assessment Tool

Grace C. Moe, Jessica E.S. Moe and Allan L. Bailey
Canadian Family Physician December 2019, 65 (12) e515-e522;

Abstract

Objective To examine patients’ perceptions of care outcomes following the introduction of collaborative teams into community family practices.

Design Cross-sectional, longitudinal study comprising 4 patient telephone surveys between 2007 and 2016, using random sampling of telephone records based on postal codes.

Setting Ten WestView Primary Care Network (WPCN) clinics in Alberta, serving a suburban-rural population of approximately 89 000 and an aggregate clinic panel of 61 611 (in 2016).

Participants Adults aged 18 and older with a visit to a family physician in a WPCN clinic at least once in the previous 18 months.

Interventions In 2006, WPCN implemented a decentralized and distributed collaborative team model, integrating nonphysician health care professionals into member clinics.

Main outcome measures The Primary Care Assessment Tool (PCAT) was used to evaluate standardized primary care delivery domains. Between-year changes were compared using ANOVA (analysis of variance). Clinic-level subgroup analyses were performed.

Results The number of completed surveys included 896 in 2007, 904 in 2010, 1000 in 2013, and 1800 in 2016, reaching 90% to 100% of the targeted sample size. In aggregate, the WPCN PCAT summary score and the scores of 4 core and 2 ancillary domains of primary care exceeded the quality threshold of 3.0: extent of affiliation, ongoing care, first-contact utilization, coordination of care, family-centredness, and cultural competence. The first-contact access domain significantly improved from 2007 to 2016 (P < .001). The domains extent of affiliation, first-contact utilization, and coordination of information systems were unchanged. Ongoing care, coordination of care, comprehensiveness, family-centredness, community orientation, and cultural competence decreased. Except for in 2010, the 2 highest scoring clinics were non-participating solo practices; the lowest-scoring clinic was the one with the largest number of physicians. Across survey years, the PCAT summary score increased statistically significantly for 1 solo practice, remained consistent at an above-quality threshold for another, but decreased for all multi-physician clinics. Unattached patients (ie, those without a family doctor) scored the lowest.

Conclusion This study found that WPCN provides high-quality primary care overall, but that patient-perceived outcomes do not indicate global improvement concurrent with team-based initiatives. Decreased standardization of the distributed model likely influenced study-observed variations in clinic performance. Future research should identify clinic and team characteristics that benefit most from team-based care and factors that explain solo practices outperforming models of team-based care.

Much attention has been placed on improving primary health care delivery and understanding the components of high-quality care. Previous work has identified core quality indicators as first-contact accessibility, ongoing care (relational continuity), family-centred care, population orientation, and intersectoral teamwork.1 Primary health care reform in Canada has focused largely on team-based care.2 The presence of allied health professionals in primary care models has been associated with lower rates of emergency department visits and death after discharge from hospital,3 as well as increased quality in primary care indicators.4 The objective of this study was to examine patients’ perceptions of primary care within the WestView Primary Care Network (WPCN) in Alberta following the implementation of collaborative teams in member community family practice clinics.

METHODS

This is a longitudinal study that evaluates patients’ WPCN experience by measuring Primary Care Assessment Tool (PCAT)5 scores derived from survey data of all WPCN-member clinic patient panels, including those clinics not participating in the collaborative team initiative.

Study setting

WestView Primary Care Network6 is a network of community-based family practice clinics currently serving a suburban-rural population of approximately 89 000 in Alberta. Established in 2005 as a provincial initiative to improve primary care delivery, WPCN has doubled from 5 to 10 family practices and has increased from 35 to 45 family physician members since inception.

The network set collaborative team-based care in a distributed “medical home” model7 as its top priority, and has allocated a substantial budget toward this initiative since 2006.

Study population and sampling method

The study population included adults aged 18 years and older residing within the WPCN catchment area who had visited a WPCN family physician at least once in the previous 18 months.

Four cycles of cross-sectional patient surveys were conducted over 9 years by 2 Alberta-based independent research firms—Howard Research (2007, 2010) and Banister Research (2013, 2016)—using a computer-assisted telephone interviewing system administered by live interviewers. Random sampling generated a list of residential telephone numbers (from a database based on postal codes), which were contacted until the targeted sample size was reached.

Based on a 95% confidence level, 5% margin of error, and known sizes of the WPCN catchment and panel populations, and with consideration of proportional representation by age and sex, a minimum sample size requirement of 382 responses per survey was determined. To ensure power for stratification by clinics for subgroup analyses, clinic-specific sample sizes were calculated based on each clinic’s panel size. As a result, target sample sizes were 1000 respondents in 2007, 2010, and 2013, and 1800 respondents in 2016, with a minimum of 50 survey respondents for multi-physician clinics.

Intervention

In 2006, WPCN began implementation of collaborative team-based care as a priority initiative that integrates nonphysician health care professionals into the family practice care team, including nursing, pharmacy, social work, and mental health professionals. In 2010, the proactive office encounter technician (POET) role was created based on the Kaiser Permanente concept8 and using the Alberta Screening and Prevention framework.9 These POETs are trained medical office assistants or licensed practical nurses who prepare prompts displayed in the electronic medical record for the attending physician at the time of the office encounter.

WestView Primary Care Network’s decentralized model allows site-specific implementation and clinic-level governance of the contractual relationships between clinics and collaborative teams. The network provides overarching guidelines and vision for the program, as well as operational support (eg, contract templates, orientation, training).

Since 2006, WPCN provided targeted funding for clinics implementing team-based services, and team-to-physician ratios increased from 0.25 to 1.22 between 2007 and 2016. Two solo-practice clinics declined the team intervention.

Outcome measurement

Survey questions incorporated items from the PCAT, adult short version; survey questions and operational definitions of the PCAT-measured primary care domains are available at CFPlus.* Outcome measures are survey response–generated PCAT scores.

The PCAT was developed by the Johns Hopkins Primary Care Policy Center for the Underserved to evaluate quality and comprehensiveness of primary care services. It is a validated tool measuring patients’ primary care experience and can be used to assess the effect of policy and system changes on the delivery of critical aspects of primary care.5,10

The PCAT measures extent of affiliation and 6 other core domains of primary care: ongoing care (relational continuity), first-contact utilization, first-contact access, coordination of care, coordination of information systems, and comprehensiveness. It also measures 3 ancillary domains: family-centredness, community orientation, and cultural competence.5,10,11 The operational definitions of PCAT domains are available at CFPlus.* Each domain is evaluated using a 4-point Likert scale (4 = definitely; 3 = probably; 2 = probably not; and 1 = definitely not). The 3 ancillary domains were excluded from our tabulation of the PCAT summary primary care score, as they were not measured in 2010 owing to budget constraints. A mean score of 3.0 was chosen as the minimum expected quality level for the PCAT score, consistent with previous literature.12–15

To determine if clinics varied in chronic disease burden, we added a study-specific question asking each respondent if he or she has “any physical, mental, or emotional problem that has lasted longer than 1 year?”

Statistical methods

Between-year changes at the WPCN level in overall PCAT summary and PCAT subscale scores were compared using 1-way ANOVA (analysis of variance). An additional subgroup analysis of both summary and subscale PCAT scores was performed for individual clinics. Subsequent 2-way analyses were performed to explore between-clinic differences; Pearson Embedded Image2 tests were used for nominal variables. A significance level of P < .05 was employed for all statistical tests. All data analyses were performed using SPSS, version 21.0.

Ethics approval was obtained from the University of Alberta Health Research Ethics Board.

RESULTS

Response rates were 64.59% in 2007, 45.56% in 2010, 54.90% in 2013, and 37.67% in 2016. We cannot explain the noted decline but speculate that “survey fatigue” might be a societal phenomenon over the same period of study. The study did not collect information on nonresponders to the telephone survey who might have had a different experience of care. However, the numbers of completed surveys were 896 in 2007, 904 in 2010, 1000 in 2013, and 1800 in 2016, which reached 89.60%, 90.40%, 100.00%, and 100.00%, respectively, of the targeted sample size (Table 1).

View this table:
Table 1.

Study population, sampling, response rates, and respondent subgroups

Respondents were categorized into subgroups: WPCN users (individuals who named a WPCN-member physician as their family doctor) for WPCN-level analyses; unattached (individuals who reported having no family doctor); and WPCN nonusers (individuals naming a family doctor who was not a WPCN-member physician), which is a subgroup that was excluded from study analyses. Characteristics of the 3 groups are reported in Table 2.

View this table:
Table 2.

Characteristics of survey respondents, by subgroup

Characteristics of clinics and PCAT scores, by clinic, per year administered, are available at CFPlus.*

Results of PCAT scores at aggregate WPCN level

Results of the PCAT subscale and summary scores for the WPCN user group are shown in Figure 1.13 Across survey years, both the summary score and 6 of 10 PCAT subscales were greater than the threshold level of 3.0, indicating high-quality primary care delivery. No significant change was seen in measures of extent of affiliation (strength of attachment) and first-contact utilization; the first scored near perfect at above 3.92 and the latter scored around 3.80 consistently. While first-contact access showed a significant improvement (P < .001) from 2007 to 2016 among the core domains, it and coordination of information systems persisted at below-threshold performance in all survey years. Despite significant decreases (P < .001) in subscales measuring ongoing care (relational continuity), coordination of care, and comprehensiveness (services provided), as well as in all 3 ancillary domains, only comprehensiveness and community orientation fell below the threshold score of 3.0 in any of the survey years.

Figure 1.
Figure 1.

The PCAT scores* over time (2007, 2010, 2013, and 2016) at the WPCN level: Assessment of primary care received, by WPCN user population (multiyear telephone surveys—WPCN catchment residents).

NA—not applicable, PCAT—Primary Care Assessment Tool, WPCN—WestView Primary Care Network.

*The PCAT scores range from 0 to 4, with higher scores indicating better performance.

†The 3 ancillary domains (family-centredness, community orientation, and cultural competence) were excluded from the PCAT summary primary care score, as they were not measured in 2010 owing to budget constraints.

Results of PCAT scores at clinic level

A subgroup analysis explored PCAT survey results at a clinic level and these are shown in Figure 2. Mean PCAT summary scores across individual clinics ranged from 3.32 to 3.50 in 2007 (P = .809), 2.97 to 3.30 (P = .370) in 2010, 3.06 to 3.46 (P = .005) in 2013, and 3.12 to 3.47 (P < .001) in 2016. Except for in 2010, the 2 highest-scoring clinics were solo-practice clinics that did not participate in the collaborative team initiative, and the lowest scoring clinic was the clinic with the largest number of physicians. Across survey years, the score increased statistically significantly for 1 solo practice, remained consistent at above the quality threshold for another, but decreased for all multi-physician clinics. There were no between-clinic differences in the proportions of chronic illness sufferers for all survey years.

Figure 2.
Figure 2.

The PCAT summary scores, by clinic

PCAT—Primary Care Assessment Tool.

In 2013 and 2016, unattached patients scored the lowest of any surveyed patients, with mean PCAT summary scores of 2.32 and 2.00, respectively.

DISCUSSION

Our results indicate that the aggregate-WPCN primary care services consistently exceeded established standards for the overall PCAT summary score and in 4 core and 2 ancillary domains of primary care: extent of affiliation (strength of attachment), ongoing care (relational continuity), first-contact utilization, coordination of care, family-centredness, and cultural competence. Their PCAT scores compare favourably to those published in other Canadian jurisdictions.12

WestView Primary Care Network performance perpetuated at below-target thresholds in 3 core domains and 1 ancillary domain: first-contact access, coordination of information systems, comprehensiveness (services provided), and community orientation. Informational continuity outside the clinic is perceived as a “macro-system” challenge. Despite augmented preventive care including the support of POETs in our model, busy family practices struggle to have those conversations with patients. Low “community orientation” scoring might speak to WPCN’s ongoing failure to authentically engage patients in the codesign of the “patient-centred medical home.”

Although scoring the lowest of all core domains across survey years and failing to reach the target quality threshold by 2016, the WPCN-level access indicator (2.41) showed the most improvement over the study period coinciding with escalating support for team resources. Other work has found associations between accessibility and team care.13 Our findings are consistent with Canadian studies that have identified first-contact accessibility as a challenge, with mean scores of 2.30 reported in Quebec13 and 2.28 in Ontario.12

Overall, WPCN did not show a global improvement in primary care delivery measured by the PCAT; this suggests that team-based care might have introduced complexity into patient management, as discussed in previous studies about the Ontario family health teams12 and the Quebec implementation of family medicine groups.14 Our results provide convergent evidence pointing to the challenges of implementing team-based care in the community family practice. We propose that the relatively low team-to-physician ratios (optimal ratios have been reported as between 2:1 and 4:116,17), the “physician as gatekeeper” model, and fee-for-service physician remuneration continue to hinder optimal team-based care and access.

Our subgroup analysis demonstrates statistically significant variation in PCAT scores between individual clinics. WestView Primary Care Network’s distributed model of collaborative team and governance provides flexibility for individual clinics to best serve clinic patients’ specific needs, but also leads to a lack of standardization across clinics, which likely affected the outcomes observed. It is also noted that the 2 solo practices that did not participate in the WPCN-resourced team initiative performed highest in PCAT scores; and, consistent with previous work,13 clinics with fewer physicians performed better on the access subscale than larger clinics did. This supports that high-quality primary care is dependent on many determinants.18 This study suggests that appropriate investment in team-based care might be necessary but not sufficient to achieve all the outcomes desired. A broader understanding and strengthening of those determinants of high-functioning primary care systems is desirable in addition to team resourcing.

Finally, lower PCAT scoring by unattached patients confirms the importance of attachment: patients’ health care experience improves when they are attached to a family doctor, supporting the health policy objective of increasing patient attachment to family physicians.

Limitations

Our longitudinal study results are observational and do not account for the emergence of many changing independent variables over time. The PCAT evaluates patient perceptions of their primary care services but is not designed to objectively evaluate system integration, equity, or efficiency.

Conclusion

The WPCN model has been associated with primary care that exceeds established thresholds for high-quality service delivery; however, non-participating solo family practices scored higher than multi-physician clinic implementing teams did. Concurrent with implementation of team-based care, WPCN first-contact access improved from 2007 to 2016, but this improvement was not seen for other domains of the PCAT, some of which decreased over time. Future interventions should continue to target access and transition of care, and prioritize attachment of patients to family physicians. Decreased standardization of the distributed model had likely influenced study-observed variations in clinic performance. Future research should identify clinic and team characteristics that benefit most from team-based care and factors that explain solo practices outperforming models of team-based care.

Notes

Editor’s key points

  • ▸ The objective of this study was to examine patients’ perceptions of primary care within the WestView Primary Care Network (WPCN)—a network of community-based family practice clinics currently serving a suburban-rural population in Alberta—following implementation of collaborative teams in community family practice clinics.

  • ▸ The Primary Care Assessment Tool (PCAT) was used to measure 7 core domains and 3 ancillary domains of primary care; a mean score of 3.0 was chosen as the minimum expected quality level for the PCAT score. This study found that WPCN primary care services consistently exceeded established standards for the overall PCAT summary score, as well as in 6 of the core and ancillary domains: extent of affiliation, ongoing care, first-contact utilization, coordination of care, family-centredness, and cultural competence.

  • ▸ A global improvement in primary care delivery in the WPCN measured by the PCAT was not found, suggesting that team-based care might have introduced complexity into patient management.

Points de repère du rédacteur

  • ▸ L’étude visait à examiner la perception qu’ont les patients des soins primaires au sein du RSPW, un groupe de cliniques de pratique familiale communautaire, qui dessert une population de banlieue et rurale en Alberta, après la mise en œuvre d’équipes collaboratives dans ces cliniques.

  • ▸ Le Primary Care Assessment Tool (PCAT ou outil d’évaluation des soins primaires) a servi pour mesurer 7 principaux domaines et 3 domaines complémentaires des soins primaires; un score moyen de 3,0 a été choisi comme niveau de qualité minimal attendu. L’étude a conclu que les services de soins primaires du RSPW excédaient constamment les critères établis pour la note de synthèse du PCAT dans l’ensemble, de même que dans 6 des domaines principaux et complémentaires : ampleur de l’affiliation, soins sur une base continue, utilisation comme premier contact, coordination des soins, centralité sur la famille et compétence culturelle.

  • ▸ Le PCAT n’a révélé aucune amélioration globale dans la prestation des soins primaires dans le RSPW, ce qui laisse entendre que les soins en équipe pourraient avoir accru la complexité dans la prise en charge des patients.

Footnotes

  • ↵* The Primary Care Assessment Tool (PCAT), adult short version, survey questions and operational definitions of the PCAT-measured primary care domains, as well as the clinic characteristics and PCAT scores, are available at www.cfp.ca. Go to the full text of the article online and click on the CFPlus tab.

  • Contributors

    All authors contributed to the concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.

  • Competing interests

    None declared

  • This article has been peer reviewed.

  • Cet article a fait l’objet d’une révision par des pairs.

References

  1. 1.↵
    1. Haggerty J,
    2. Burge F,
    3. Levesque J,
    4. Gass D,
    5. Pineault R,
    6. Beaulieu M,
    7. et al
    . Operational definitions of attributes of primary health care: consensus among Canadian experts. Ann Fam Med 2007;5(4):336-44.
    OpenUrlAbstract/FREE Full Text
  2. 2.↵
    1. Strumpf E,
    2. Levesque JF,
    3. Coyle N,
    4. Hutchison B,
    5. Barnes M,
    6. Wedel RJ
    . Innovative and diverse strategies toward primary health care reform: lessons learned from the Canadian experience. J Am Board Fam Med 2012;25(Suppl 1):S27-33.
    OpenUrlAbstract/FREE Full Text
  3. 3.↵
    1. Riverin BD,
    2. Li P,
    3. Naimi AI,
    4. Strumpf E
    . Team-based versus traditional primary care models and short-term outcomes after hospital discharge. CMAJ 2017;189(16):E585-93.
    OpenUrlAbstract/FREE Full Text
  4. 4.↵
    1. Beaulieu MD,
    2. Haggerty J,
    3. Tousignant P,
    4. Barnsley J,
    5. Hogg W,
    6. Geneau R,
    7. et al
    . Characteristics of primary care practices associated with high quality of care. CMAJ 2013;185(12):E590-6.
    OpenUrlAbstract/FREE Full Text
  5. 5.↵
    1. Johns Hopkins Primary Care Policy Center [website]
    . Primary care assessment tools. Baltimore, MD: Johns Hopkins University; Available from: www.jhsph.edu/research/centers-and-institutes/johns-hopkins-primary-care-policy-center/pca_tools.html. Accessed 2019 Oct 16.
  6. 6.↵
    WestView Primary Care Network [website]. Spruce Grove: AB; 2019. Available from: www.westviewpcn.ca. Accessed 2019 Oct 16.
  7. 7.↵
    1. Moe JS,
    2. Bailey AL,
    3. Kroeker S,
    4. Moe G
    . An interprofessional collaborative practice model: primary-care clinical associates at the family practice setting. Healthc Manage Forum 2010;23(4):159-63.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Kanter M,
    2. Martinez O,
    3. Lindsay G,
    4. Andrews K,
    5. Denver C
    . Proactive office encounter: a systematic approach to preventive and chronic care at every patient encounter. Perm J 2010;14(3):38-43.
    OpenUrlPubMed
  9. 9.↵
    1. Alberta Medical Association [website]
    . Improve screening with ASaP (and earn CME credits). Edmonton, AB: Alberta Medical Association; 2013. Available from: www.albertadoctors.org/ASaP. Accessed 2019 Oct 16.
  10. 10.↵
    1. Xu J,
    2. Shi L,
    3. Starfield B
    . Validating the adult Primary Care Assessment Tool. J Fam Pract 2001;50(2):161.
    OpenUrlPubMed
  11. 11.↵
    1. Starfield B
    . Adult Primary Care Assessment Tool—short version. Baltimore, MD: John Hopkins; 1998.
  12. 12.↵
    1. Carroll JC,
    2. Talbot Y,
    3. Permaul J,
    4. Tobin A,
    5. Moineddin R,
    6. Blaine S,
    7. et al
    . Academic family health teams. Part 1: patient perceptions of core primary care domains. Can Fam Physician 2016;62:e23-30. Available from: www.cfp.ca/content/cfp/62/1/e23.full.pdf. Accessed 2019 Oct 16.
    OpenUrlAbstract/FREE Full Text
  13. 13.↵
    1. Haggerty JL,
    2. Pineault R,
    3. Beaulieu M,
    4. Brunelle Y,
    5. Gauthier J,
    6. Goulet F,
    7. et al
    . Practice features associated with patient-reported accessibility, continuity, and coordination of primary health care. Ann Fam Med 2008;6(2):116-23.
    OpenUrlAbstract/FREE Full Text
  14. 14.↵
    1. Tourigny A,
    2. Aubin M,
    3. Haggerty J,
    4. Bonin L,
    5. Morin D,
    6. Reinharz D,
    7. et al
    . Patients’ perceptions of the quality of care after primary care reform. Family medicine groups in Quebec. Can Fam Physician 2010;56(7):e273-82. Available from: www.cfp.ca/content/cfp/56/7/e273.full.pdf. Accessed 2019 Oct 16.
    OpenUrlAbstract/FREE Full Text
  15. 15.↵
    1. Haggerty JL,
    2. Roberge D,
    3. Pineault R,
    4. Larouche D,
    5. Touati N
    . Features of primary healthcare clinics associated with patients’ utilization of emergency rooms: urban-rural differences. Healthc Policy 2007;3(2):72-85.
    OpenUrlPubMed
  16. 16.↵
    1. American Academy of Family Physicians
    . Primary care for the 21st century. Ensuring a quality, physician-led team for every patient. Leawood, KS: American Academy of Family Physicians; 2012.
  17. 17.↵
    1. World Bank
    . World development report 1993: investing in health, world development indicators. New York, NY: Oxford University Press; 1993.
  18. 18.↵
    1. Bodenheimer T,
    2. Ghorob A,
    3. Willard-Grace R,
    4. Grumbach K
    . The 10 building blocks of high-performing primary care. Ann Fam Med 2014;12(2):166-71.
    OpenUrlAbstract/FREE Full Text
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Respond to this article
Evaluating the implementation of collaborative teams in community family practice using the Primary Care Assessment Tool
Grace C. Moe, Jessica E.S. Moe, Allan L. Bailey
Canadian Family Physician Dec 2019, 65 (12) e515-e522;
Reddit logo Twitter logo Facebook logo Mendeley logo

Jump to section

Subjects