Abstract
BACKGROUND T o examine the methodology used to evaluate whether focusing the work of nurse practitioners and a pharmacist on frail and at-risk patients would improve the quality of care for such patients.
DESIGN Evaluation of methodology of a randomized controlled trial including analysis of quantitative and qualitative data over time and analysis of cost-effectiveness.
SETTING A single practice in a rural area near Ottawa, Ont.
PARTICIPANTS A total of 241 frail patients, aged 50 years and older, at risk of experiencing adverse health outcomes.
INTERVENTION At-risk patients were randomly assigned to receive Anticipatory and Preventive Team Care (from their family physicians, 1 of 3 nurse practitioners, and a pharmacist) or usual care.
MAIN OUTCOME MEASURES The principal outcome for the study was the quality of care for chronic disease management. Secondary outcomes included other quality of care measures and evaluation of the program process and its cost-effectiveness. This article examines the effectiveness of the methodology used. Quantitative data from surveys, administrative databases, and medical records were supplemented with qualitative information from interviews, focus groups, work logs, and study notes.
CONCLUSION Three factors limit our ability to fully demonstrate the potential effects of this team structure. For reasons outside our control, the intervention duration was shorter than intended; the practice’s physical layout did not facilitate interactions between the care providers; and contamination of the intervention effect into the control arm cannot be excluded. The study used a randomized design, relied on a multifaceted approach to evaluating its effects, and used several sources of data.
TRIAL REGISTRATION NUMBER NCT00238836 (CONSORT).
The population aging trend affects most industrialized countries, and Canada is no exception. In a 30-year span, between 2001 and 2031, the proportion of Canadians aged 65 years and older is projected to increase by an astonishing 69% (from 17% to 28.5% of the overall population).1 This shift in age distribution has been the impetus for primary health care reform for 2 reasons. First, the annual per capita cost of health care in this age group is 3.8 times higher than that for all age groups combined.1 Second, while the age-standardized risk of mortality due to chronic diseases decreased in Canada by 14% over the 10-year period from 1991 through 2001, the absolute number of deaths increased by 13% over that same period.2 Most deaths (91%) occur in individuals 50 years of age or older and are usually a result of chronic diseases.2 Health care costs and absolute mortality are likely to keep rising with the aging population, challenging the Canadian health care system with competing concerns—how do we curtail costs while continuing to improve the quality of care (QOC) for a growing number of chronically ill patients?
Primary health care is the cornerstone of an efficient health care system. To improve primary health care delivery, many different models of care have been proposed.3 Some evidence suggests that, when dealing with higher-risk patients, interdisciplinary care improves both patient outcomes (such as quality of life and morbidity) and system outcomes (such as tertiary care costs). For example, the addition of a pharmacist to a primary care setting in the management of hypertensive patients produced improved blood pressure among patients and reduced costs.4 In another study, a home-based team-management program for needy veterans led to significant improvements in various measures of patient QOC and caregiver quality of life (P < .05), as well as a reduction in hospitalization (P = .07).5
This study aimed to evaluate the effects of introducing nurse practitioners (NPs) and a pharmacist into a family practice to care for patients who were at risk of negative health outcomes, including functional decline, needing to use emergency services, or physical deterioration. In this paper, we describe both the Anticipatory and Preventive Team Care (APTCare) intervention and the mixed methods used to evaluate its effect on patient care and use of services.
Study objectives
The study’s primary objective was to evaluate the effects of the APTCare program on QOC for chronic disease management (CDM). Secondary objectives included other measures of efficacy; acceptability of APTCare; and its cost-effectiveness.
METHODS
Study design
This is a mixed method evaluation study of the intervention’s efficacy, implementation, and cost. We used a randomized controlled trial design of APTCare versus usual care (1:1) to assess the intervention’s effect on patient care (efficacy). When more than 1 individual in a household was eligible, the household was randomized as a single unit to avoid crossover effects. Treatment arm was assigned by accessing a central automated line containing the concealed random treatment allocation. In order to assess the intervention’s implementation (acceptability), process case studies and focus groups were used to elicit information about patient and provider experiences in the program. Data on service utilization and implementation costs were used for cost-effectiveness analysis. The study was approved by the Ottawa Hospital Research Ethics Board and is registered with CONSORT as NCT00238836.
Setting
The study was performed at a family practice that had been in operation for more than 25 years. In 2004, it became a family health network (FHN), which is a type of group practice that provides primary care services to rostered patients. Physicians were remunerated through a combination of capitation, reformed fee-for-service, bonuses, and incentives. The practice served a patient population of 10 000, with approximately 7500 enrolled in the FHN. The practice offered access to health care every day of the week and on-call services 24 hours a day. All 8 physicians (7 were full-time equivalent) had active staff privileges at the nearby hospital. Personnel supporting care delivery included 5 nurses and 11 administrative personnel. The practice was located in a rural village on the perimeter of a larger city (Ottawa, Ont) and used electronic medical records (EMRs).
Study population
The target population was patients at risk of adverse health outcomes, as it was assumed that this group could benefit most from the intensive follow-up and management provided by the APTCare intervention. Patients were also required to be 50 years of age or older and enrolled in the FHN. Patients who were expected to be away during the study period for 6 weeks or more or who had substantial cognitive impairment, language or cultural barriers, or life expectancy less than 6 months were excluded.
Sample size calculation
The trial was originally designed to detect a difference in emergency department (ED) visits between the 2 arms. We had postulated that 43.9% of patients in the control arm and 23.4% of patients in the intervention arm would require ED visits.6 The sample size required to detect a reduction in ED use from 43.9% to 23.4% (with α = .05 and β = .20) was 168. Allowing for 30% loss to follow-up, the required sample size for the study was 240. However, in the process of identifying at-risk patients, an evaluation of emergency service use for these patients revealed that less than 24% of at-risk individuals had required an ED visit in the previous year. As a consequence, because the sample size could not be increased for practical reasons, we revised the primary outcome to be the differences in the QOC for CDM. Quality of care was evaluated as the extent of adherence to recommended guidelines for 4 chronic diseases (Table 1).7 The CDM QOC score was the average score of adherence to recommended guidelines for each chronic condition, measured as the proportion of indicator maneuvers performed for that condition. Postulating that the baseline CDM QOC score would be 55%8 and that each indicator could be evaluated in approximately 20% of participants, the study had a power of 98% to detect a 10% difference between the 2 arms. This change was approved 5 months into the study and before any outcome data were available.
Maneuvers evaluated for measuring performance in chronic disease management: 1 point was awarded for each maneuver performed (0.5 points were awarded if HbA1c was measured only once in the past y).
Recruitment
Patient selection
The selection process was performed in 2 steps. First, we relied on risk factors established in the literature as being associated with vulnerable populations to identify a list of potential candidates using the EMR.9–15 The risk factor criteria were as follows:
-
at least 1 ED visit in the past year (the EMR was first updated with a list of ED visits to the nearest hospital);
-
multiple health conditions, including at least 2 chronic conditions for which at least 2 visits each were recorded in the previous year, or 4 conditions of any type with at least 2 visits each in the previous year;
-
frequent visits, defined as at least 5 visits to the practice in the previous 6 months or 10 in the previous year; and
-
polypharmacy, defined as 4 or more currently active or chronic medications.
This process produced a list of 1009 potential candidates. A summary profile of each potential candidate was then created and reviewed by each patient’s doctor to determine eligibility and risk level. Physicians were also encouraged throughout the 5-month recruitment period to identify patients who were at risk, based on their knowledge of the population and patient encounters during that period, but who had not been captured using the study risk factor criteria.
Of the 1009 potential candidates, physicians excluded 61 for the following reasons: they were moving or were no longer with the practice (n = 44), they had substantial cognitive impairment (n = 13), they had considerable cultural or language barriers (n = 1), or they had life expectancies less than 6 months (n = 3). The physicians codified the remaining 948 patients into 1 of 4 risk levels: high, medium, low, or very low. Of these, 383 patients were estimated to be at high or medium risk and were, therefore, suitable for the study. Twelve additional patients, not identified through the risk factor process, were separately identified by their physicians as being at risk.
Because this process was labour intensive, evaluation of the entire population took place over a period of 4 months. As a result, 34 of the at-risk candidates were identified after their at-risk partners had been identified and already randomized. These patients were excluded as they could no longer be randomized as unit pairs. A total of 361 patients were approached regarding the study.
Enrolment of participants
As required by the ethics board, FHN staff first approached patients for participation. Of the 361 patients, 43 were deemed to be ineligible (primarily owing to planned prolonged absence in the winter), 80 refused participation, 5 could not be contacted, and 233 agreed to participate. Another 8 patients considered to be at low risk were included in the study and randomized with their at-risk partners. In total, 241 patients participated in the study, including 206 individuals, 16 pairs, and 1 group of 3 (parents and a 55-year-old daughter). Participants were randomized at their baseline home visits. The overall participation rate was 77%.
Intervention
The NPs and the pharmacist worked in an interdisciplinary, integrated capacity with the existing community practice of family physicians and nurses. The NPs and the pharmacist reviewed the clinical charts of newly randomized patients in the intervention group and performed initial home visits to complete their assessments of the patients. Together, they established a care planning document for each patient that contained the results of their assessments, medication information, health screening information, and a breakdown of patient care priorities based on 5 dimensions of care, including disease management, medical review, education and self-care, social support and community integration, and psychological issues. Care plans were reviewed with the patients’ respective family physicians and were implemented and adapted throughout the study period. Care providers (family physicians, NPs, and the pharmacist) communicated regularly, through spontaneous patient-focused dialogue, scheduled case conferences, and sharing detailed progress notes through charting and electronic messaging. Care was provided by the NPs and the pharmacist almost exclusively in patients’ homes and by telephone contact, with few clinical visits taking place at the practice. Education sessions were also performed to provide specific health-related information to intervention participants who shared similar medical concerns. The NPs were actively involved in the team for the duration of the study (18 months). The pharmacist’s role was discontinued when all patients had received their assessments and the pharmacist was confident that appropriate action plans were in place to address areas of concern (12 months). Figure 1 shows the time and event schedule for the intervention period and data collection.
Time and event schedule
We had originally intended for all intervention participants to receive the support of a home telehealth monitoring system. The system included a comprehensive set of electronic vital sign devices (ie, blood pressure monitor, weight scale, glucometer, pulse oximeter, and peak flow metre) and was intended to allow the NPs to monitor the patients’ clinical information remotely and alert them of values that were out of normal range. However, the unit was unavailable for the first 6 months of the study, and the objective was changed to a feasibility evaluation. The unit was installed in the homes of 22 patients purposefully selected by the NPs for their likelihood to benefit from this technology. A research associate installed the unit and provided training and ongoing technical support throughout the study period.
OUTCOMES
The primary outcome measure was a composite score derived from 12 indicator maneuvers16–22 reflecting CDM guidelines for the management of 4 chronic diseases (Table 1).7 For each patient, performance scores for individual diseases were calculated and averaged to create an overall score for CDM in which each chronic disease had equal weight.
The secondary objectives included other measures of efficacy, including intermediate clinical outcomes for chronic diseases, QOC for prevention, quality of life (measured by the Short-Form 36 [SF-36]23 and health-related quality of life scales [HRQoL-4]24), ED visits and hospitalization, activities of daily living, caregiver burden, and medication appropriateness; acceptability of APTCare to patients, caregivers, and care providers; and APTCare’s cost-effectiveness. Acceptability of APTCare was determined through program evaluation, assessment of participants’ perceptions of the program, process documentation, and examination of care provider collaboration and roles.
Data collection
Data collection consisted of a combination of questionnaires, interviews, focus groups, and document analysis. Individual data sources were frequently used to inform more than one objective. For that reason, the data collection plan is reviewed by source of data rather than outcome measure. In order to make clearer the correspondence between outcomes, objectives, and data sources, we include a summary table outlining this (Table 2).
Research objectives, outcome measures, and data sources: A quantitative pre-post randomized controlled trial analytic approach (1.0), analysis of quantitative and qualitative data over time (2.0), and analysis of cost effectiveness (3.0) were used.
Patients and caregivers
Surveys and interviews were used to inform QOC assessments, process evaluation, and the economic study.
Patient information
We surveyed all patients at the study baseline (before randomization, in person), at the midpoint (by telephone), and at the end of the study (before withdrawing APTCare services, in person).
Demographic information
Demographic information was collected at baseline.
Quality of life
Information was collected (at baseline and at the end of the study) using both the SF-3623 and HRQoL-4.24 The SF-36 identifies 8 health domains and constructs physical and mental component summary scores. The 2 scores range from 0 to 100, with higher scores representing better health-related quality of life. Scores were calculated using the norm-based scoring algorithm and were standardized to the 1998 US general population (mean 50, SD 10) using software from QualityMetric Incorporated.23 The HRQoL-4 measures the patient’s perceived health status on a 5-point Likert scale from “poor” to “excellent” and measures the number of days in the previous 30 that were limited owing to physical or mental health conditions.24,25
Functional status
Information was collected (baseline and end of study) using Lawton and Brody’s Instrumental Activities of Daily Living Scale.26 This scale evaluates functional status in 8 areas of daily activity, with 3 to 5 status levels for each question. Higher scores indicate greater loss of functional independence.
Health service utilization
Information was collected (baseline, midpoint, and end of study) using a tool developed in-house to capture use of primary care services, emergency services, hospitalization, and allied health services.
Caregiver burden
Unpaid caregivers (of all patients who identified having caregivers) were assessed for caregiver burden at the beginning and end of the study using the Zarit Burden Scale.27–29 This 5-point (0–4) Likert scale, which addresses 22 items related to caregiver burden, was either left in the patient’s home or mailed to the caregiver’s address (if different) to be completed by the caregiver and returned in the self-addressed stamped envelope included.
Patient and caregiver perceptions of the program
We evaluated patient and caregiver perceptions of the program (intervention patients only) through survey questionnaires and in-depth interviews.
Intervention patients
Patients and their caregivers were surveyed by telephone 6 weeks after randomization and in person at the end of the study during the exit interview. The questionnaire was developed in-house and captured the participants’ understanding of the program, their expectations of it, and their levels of satisfaction with its various components. The early survey results were used to inform program development and service delivery.
Key informants
A subset of 9 key informant patients and their caregivers were interviewed in depth approximately midway through the study to gain a more qualitative view of their perceptions of the program. Each NP identified 3 patients for this process using the following criteria: patients with complex care requirements, at least 1 without a caregiver, and at least 1 man. Patients were asked about their experiences with the program and to identify positive aspects and areas for improvement. A review of patient clinical information was performed before the interview to understand quality-of-life and care issues.
Patients and caregivers with home telehealth devices
At the midpoint and the end of the study, a questionnaire was administered by telephone to the subset of patients and their caregivers who had the home telehealth technology installed. The survey addressed the technical reliability of the equipment, its ease of use, and their general satisfaction with its various components.
Electronic medical records
We used EMR data to perform the efficacy evaluation and inform the process evaluation.
Prevention, CDM, and intermediate outcomes for chronic diseases
We reviewed relevant sections of the EMR, at the end of study, to gather baseline and end-of-study evidence for the evaluation of CDM, prevention, and intermediate outcome for chronic diseases.
Health service utilization
The EMR also served as a second source of information about health service utilization.
Medication appropriateness
The appropriateness of patient medications for intervention participants was evaluated by a pharmacist at baseline and again at the end of study using the Medication Appropriateness Index.30–32 Evaluations included a review of the medical records and face-to-face patient interviews at baseline, and review of the medical records and telephone interviews, as required, at the end of the study.
Multidisciplinary collaboration
We reviewed the content of the electronic messaging “to-do” system, which is part of the EMR and serves as a communication conduit between clinicians, to examine multidisciplinary collaboration. This was done at 7 months after the start of the study and 2 months before the end of the study, as these 2 time periods reflected the processes in the context of a newly established collaborative practice and a more mature one, respectively. Themes associated with roles, communication, and collaboration during these time periods were identified and incorporated into the qualitative component of the study for process evaluation.
Frequency of types of visits
For continuity of information, the NPs and the pharmacist logged all important contact with patients as progress notes in the EMR. All progress notes were reviewed to evaluate the frequency of each type of visit (ie, telephone, home, office) for all care providers (ie, doctor, NP, pharmacist).
Care providers
Collaborative processes
Various methods were used to examine the collaborative processes.
Case studies
The NPs and pharmacist each identified a patient for in-depth follow-up of collaborative processes throughout the study period in order to write a case study. The NP, pharmacist, and family physician for each patient independently documented aspects of collaboration relating to that patient for the first third of the study and reported these aspects together for the second and third portion of the study. We provided the clinicians with the following elements of collaboration along with definitions to be considered when documenting the collaborative process: responsibility and accountability, coordination, communication, cooperation, assertiveness, autonomy, mutual trust, and respect.
Surveys
We used the Way and Jones Collaboration Care Provider Survey33 to evaluate various dimensions of collaboration based on the extent of and degree of satisfaction with collaboration between clinicians. Collaboration surveys were completed by each care provider at 3 points in the study (at 5 months, 12 months, and at the end of the study).
Focus groups
Focus groups were conducted immediately following the collaboration survey administration at 5 and 12 months into the study with the NPs and pharmacist, family physicians, and family practice nurses separately (3 groups) and together. Discussions were guided by a collaboration framework and informed by the results of the collaboration surveys.
Interviews
Midway through the study and again at the end of the study, we interviewed the NPs and pharmacist regarding their views of the process, their roles and responsibilities, workload, and care provision. Interviews were also conducted with all family physicians shortly after the pharmacist’s involvement was discontinued and again at the end of the study. The early interviews served to inform the process evaluation (and to adapt the program at midpoint).
Daily log
The pharmacist and NPs maintained a daily log throughout the study period that captured the time spent on various activities, the instances of collaboration along with details of the process, and other information they thought was relevant. These logs informed the collaboration and process evaluation.
Home telehealth
The research assistant involved in installation and patient orientation for the home tele-health unit completed a questionnaire that captured details about each installation, including the type of peripheral devices used, patient concerns, and extent of support required. The NPs involved in the care of these patients also completed questionnaires to identify why each patient was selected to have the technology installed and to explore details of their experiences with the technology.
Other sources
Administrator’s log
The study administrator maintained a log of discussions with study staff and clinicians that was used to inform the process evaluation, including elements of collaboration.
Minutes
Weekly meetings with study staff, monthly meetings with the NPs and pharmacist, and monthly meetings with coinvestigators were documented in minutes. These were used to inform the process evaluation, including collaboration.
Ministry of Health and Long-Term Care
We requested health care service utilization information from the Ministry of Health and Long-term Care for the 241 patients enrolled in the study. This information included hospitalization, use of emergency services, consultations with health professionals, alternative level of care, long-term care, and medication covered by government insurance plans. The information was used for the economic and outcome evaluations.
Community services
The Community Care Access Centre provided a list of services that were provided to the study population over the course of the study and the associated cost information (ie, date of services, services, length of visit, rate, and cost) for use in the economic evaluation.
DISCUSSION
Limitations
This study evaluating the effects of adding NPs and a pharmacist to a family practice has some methodologic limitations. The first relates to the suboptimal duration of the intervention. The study’s funder required that the study be completed by a fixed date. As a result, and because of the time spent setting up the study, the duration of the intervention was reduced from the originally planned 24 months to 12 to 18 months. Also, the recruitment phase extended beyond the period planned because of the unanticipated workload involved in initial home visits and care planning for new patients by the APTCare team. Consequently the intervention period was reduced to 12 months for those recruited last. The shortened study period likely affected our ability to observe the full effects of the intervention.
This study was conducted at a single family practice, limiting its generalizability. Also, because family physicians cared for patients assigned to both arms, any benefit gained from working with the NP and pharmacist in planning the care of their patients assigned to the intervention arm could have contaminated their practices for patients assigned to the control arm. The before-after design of this evaluation does allow us to detect improvements in the care of the individuals assigned to the control arm during the intervention period; however, because these changes could be related to a number of other factors, not only contamination, the effects of contamination cannot be eliminated.
Also, owing to the lack of office space in the participating family practice, the APTCare clinicians and physicians were located on different floors, rendering spontaneous contact almost impossible. It is likely that this limited their ability to collaborate and influenced the adjustment period for this new multidisciplinary team. The collaboration surveys, focus groups, and interviews will allow us to shed light on the barriers to integration such new team members face. Finally, while the intervention was originally designed for care provision to be delivered in the home, during the study it became apparent that most patients were sufficiently independent to make office visits and, in fact, did so for their regular physician appointments. However, owing to the space restrictions at the practice, the procedures could not be changed to allow for office visits with the NPs or the pharmacist. As a result, we could not improve on the efficiency of the intervention.
Strengths
The study did have several strengths. It was a randomized controlled trial that used mixed methods to measure and understand the effects of the intervention. The evaluation involved feedback from the patients, their caregivers, and the medical care providers. It used a multipronged approach to garnering information, including participant surveys, interviews, and focus groups; patient record review; health ministry encounter data; and other study documentation. The presence of an EMR in the practice allowed us to identify at-risk patients efficiently and facilitated review of documentation.
Conclusion
The APTCare intervention resulted in meaningful improvements in the QOC for CDM in the at-risk population studied. However, the intervention was quite costly, and we recommend that a modified APTCare intervention, for which costs are better contained, be studied. Limitations of the randomized controlled methodology used—including limitations on duration, generalizability, and space—likely affected our ability to observe the full effects of the intervention.
Acknowledgment
We thank Jo-Anne Dusseault, Wei Zhang, and Jennifer Englert for their input in the preparation and editing of the manuscript. Funding for this research was provided by the Ontario Ministry of Health and Long-Term Care Primary Health Care Transition Fund. The views expressed in this report are the views of the authors and do not necessarily reflect those of the Ontario Ministry of Health and Long-Term Care.
Notes
EDITOR’S KEY POINTS
-
As the population continues to age, Canadian health care resources are becoming increasingly stretched to meet the needs of a growing number of chronically ill patients. Evidence suggests that care focused on the more needy and delivered by multidisciplinary primary care teams is effective. Because some Canadian provincial governments are encouraging such care delivery, this project sought to determine whether it improves care and is cost-effective.
-
Although the researchers found that Anticipatory and Preventive Team Care did improve the quality of care for chronic disease management in the at-risk population studied, the intervention was not found to be cost-effective in this study. Limitations of the randomized controlled methodology used—including limitations on duration, generalizability, and space—likely affected the researchers’ ability to observe the full effects of the intervention.
POINTS DE REPÈRE DU RÉDACTEUR
-
Avec le vieillissement de la population, les ressources canadiennes en santé sont de plus en plus sollicitées pour répondre aux besoins d’un nombre croissant de malades chroniques. Certaines données suggèrent qu’il est avantageux d’utiliser des équipes multidisciplinaires qui dispensent les soins primaires à ceux qui en ont le plus besoin. Comme certaines provinces canadiennes favorisent ce type de soins, il paraissait opportun de déterminer si cette approche était rentable et améliorait les soins.
-
Même si l’étude montrait que les Anticipatory and Preventive Team Care améliorent la qualité des soins aux patients chroniques dans la population à risque étudiée, ce type d’intervention n’est pas apparue rentable dans cette étude. On estime que les limitations de la méthodologie randomisée utilisée – incluant celles liées à la durée, la possibilité de généraliser et l’espace – a pu empêcher les chercheurs d’observer l’ensemble des effets de l’intervention.
Footnotes
-
This article has been peer reviewed.
-
Contributors
Ms Dahrouge drafted the manuscript, and all other authors were involved in the design of the study and reviewed and edited the manuscript.
-
Competing interests
None declared
- Copyright© the College of Family Physicians of Canada
References
In this issue
