Abstract
OBJECTIVE To evaluate the cost-effectiveness of Anticipatory and Preventive Team Care (APTCare).
DESIGN Analysis of data drawn from a randomized controlled trial.
SETTING A family health network in a rural area near Ottawa, Ont.
PARTICIPANTS Patients 50 years of age or older at risk of experiencing adverse health outcomes. Analysis of cost-effectiveness was performed for a subsample of participants with at least 1 of the chronic diseases used in the quality of care (QOC) measure (74 intervention and 78 control patients).
INTERVENTIONS At-risk patients were randomly assigned to receive usual care from their family physicians or APTCare from a collaborative team.
MAIN OUTCOME MEASURES Cost-effectiveness and the net benefit to society of the APTCare intervention.
RESULTS Costs not directly associated with delivery of the intervention were similar in the 2 arms: $9121 and $9222 for the APTCare and control arms, respectively. Costs directly associated with the program were $3802 per patient for a total cost per patient of $12 923 and $9222, respectively (P = .033). A 1% improvement in QOC was estimated to cost $407 per patient. Analysis of the net benefit to society in absolute dollars found a breakeven threshold of $750 when statistical significance was required. This implies that society must place a value of at least $750 on a 1% improvement in QOC in order for the intervention to be socially worthwhile. By any of the metrics used, the APTCare intervention was not cost-effective, at least not in a population for which baseline QOC was high.
CONCLUSION Although our calculations suggest that the APTCare intervention was not cost-effective, our results need the following caveats. The costs of such a newly introduced intervention are bound to be higher than those for an established, up-and-running program. Furthermore, it is possible that some benefits of the secondary preventive measures were not captured in this limited 12- to 18-month study or were simply not measured.
TRIAL REGISTRATION NUMBER NCT00238836 (CONSORT).
At the start of this millennium, a federally sponsored report on planning the future of health care in Canada recognized the pivotal role that primary care plays in sustaining the country’s health care system, and strongly advocated that the primary care system be strengthened and reformed.1 As a result, the Primary Health Care Transition Fund was established to help inform and enable this change.2 One of the reform’s objectives was to implement and evaluate multidisciplinary primary care teams. Supported by this transition fund, in 2004 we initiated a $1.2 million randomized controlled study that evaluated the effects of a home-based team care program, Anticipatory and Preventive Team Care (APTCare). This project was designed for at-risk patients, and the team consisted of a nurse practitioner (NP) and a pharmacist working collaboratively with family physicians.
At the time, there was some evidence suggesting that intensifying management of patients with chronic illnesses by channeling community resources toward their care was effective, although the economic effects of this approach remained unclear. For example, one study demonstrated that an NP–family physician team that focused on patients with chronic diseases produced superior quality of care but was associated with higher primary care costs compared with standard care.3 Another study evaluating a pharmacist–family physician team found superior control over blood pressure and lower visit costs with team-based care.4 Both studies limited their economic evaluation to costs associated with primary care office contacts, and did not consider the financial consequences of these outcomes on the broader health care system, including, for example, emergency care visits. A large, randomized controlled study of veterans comparing home-based team care to standard care included a comprehensive evaluation of health care costs and found that the home-based care was more costly, even when reductions in emergency service use were factored in.5
In this paper we focus on the cost-effectiveness (CE) of the APTCare intervention. This study is of particular relevance because of Ontario’s recent thrust to implement programs supporting integration of allied health care workers into family practices. Results obtained from this study will help inform our understanding of the economic consequences of similar interventions.
The primary outcome measure of the trial was the change over the course of the intervention in the composite quality of care (QOC) score for the following 4 chronic diseases: coronary artery disease, diabetes, congestive heart failure, and chronic obstructive pulmonary disease (COPD). For each chronic condition the patient had, a score was constructed by dividing the number of appropriately performed maneuvers by the number of eligible maneuvers. The composite QOC score was calculated as the arithmetic average across the chronic disease QOC scores the patient had. Changes in the QOC score thus depended on the number of new maneuvers performed between the baseline and the end-of-study observations (the numerators of each condition score), as the denominators do not change. Maneuvers for the chronic conditions are listed in Table 1.6
Maneuvers evaluated for measuring performance in chronic disease management: 1 point was awarded for each maneuver performed (0.5 points were awarded if HbA1c was measured only once in the past y).
Elsewhere we report on the process of identifying these at-risk patients in the primary care setting7 and the effects of the intervention on predefined QOC indicators for chronic disease management.6 The results of the latter work show that the intervention led to an overall increase in QOC of 9.2%. As QOC varies between 0 and 1, this result suggests that the effect of the intervention is an improvement on the order of 9.2 percentage points.
METHODS
This was a randomized controlled trial conducted in a semirural family health network (FHN). The study methodology is reviewed briefly here, but more details are provided elsewhere.7
Setting and sample
This particular FHN is composed of a team of health care professionals (8 physicians and 5 nurses) and support staff serving approximately 10 000 patients. In the FHN payment model, physician compensation is based on a blended formula of capitation (principal component) and fee-for-service (FFS). The capitation rate is based on the sex and age of enrolled patients, and covers all core services. The practices also received 10% of the usual FFS costs for these core services and full FFS remuneration for noncore services and all services rendered to nonenrolled patients.
Patients 50 years of age and older who were considered to be at risk of functional decline, physical deterioration, or of requiring emergency services were the target of the study. Patients with considerable cognitive impairment, language or cultural barriers, life expectancy less than 6 months, and those expecting to be away from the geographic area for a period of 6 weeks or more during the study period were excluded. Seventeen percent of the patients 50 years of age or older met the criteria for inclusion; 241 were enrolled in the study (120 intervention, 121 control). The study participation rate was 76%.
The analysis herein makes use of a subsample of patients for whom QOC scores were measured—patients who at the beginning of the study had at least 1 of the 4 chronic diseases used in the QOC measure. The subsample therefore includes 74 intervention and 78 control patients.
Intervention and outcomes
Patients randomized to the control arm continued to receive their usual medical care. Those randomized to the intervention arm (APTCare) were assigned to the care of 1 of 3 NPs, the pharmacist, and their usual family physicians. Care provided by the NPs and pharmacist was delivered almost exclusively in the patients’ homes, while patients continued to see their family physicians in the office. The central thrust of the intervention was to ensure evidence-based disease management and strong social supports for patients. Twenty-two patients also received a telehealth system in the home for remote monitoring of clinical parameters (eg, blood pressure, weight, glucose levels, and blood oxygen levels) by the NPs. The study’s duration was 12 to 18 months, depending on the time of randomization, but was similar in both groups. The differences in baseline and end-of-study QOC scores were computed for comparison between the 2 arms. Of the 241 patients in the study, 152 had at least 1 of the 4 chronic diseases and are included in this analysis.
Measuring costs
The cost of the intervention included costs incurred during the study period, which were measured in Canadian dollars and analyzed from the perspective of the provincial Ministry of Health. Table 2 outlines the details of the approach used to measure costs.
Cost items
Analysis
The economic calculations were based on intention-to-treat analysis. Student t tests were employed for comparing continuous variables, and χ2 or Fisher exact tests were used for categorical variables. We address the question of this intervention’s CE by employing 2 methods. First, we estimated the CE ratio of the intervention:
This value is interpreted as the average per-patient absolute cost required for a 1% improvement in the QOC between the 2 arms. Note that by including the change in the QOC index, this approach controls for any trend in the QOC index that might have been operating on both groups over the course of the trial. The greater the CE ratio, the higher the incremental cost for a 1% increase in effectiveness.
The second approach follows the methodology of Briggs.8 This tool weighs the realized benefits against the realized costs for each individual. Specifically, we estimate the net benefit (NB) to society derived from the intervention as follows:
Here, E denotes the effectiveness and SC denotes the quantifiable social costs stemming from all observed health care services. The parameter λ is a subjective one, representing the monetary value that society would be willing to pay for a 1% improvement in QOC. Conceptually, λ ranges from 0 to infinity, with higher numbers reflecting greater value placed on improved care. Zero indicates that no value is placed on the improved outcome, whereas a value of, say, $1000 reflects a willingness to invest $1000 per patient to obtain a 1% improvement in QOC. While SC and E are observed for each patient, NB for each patient is calculated for a range of possible values of λ. A positive NB indicates that the benefits outweigh the costs, and thus that the intervention is socially beneficial.
For each value of λ, regressions were estimated in which NB was the dependent variable and treatment status (APTCare or control) was the key independent variable. The analysis was performed with and without controlling for patient characteristics (shown in Table 3) and baseline QOC scores, using forward stepwise criteria (entry and exit criteria of 0.05 and 0.10, respectively). A statistically significant, positive estimated coefficient for the treatment status indicator (ie, the APTCare arm) indicates that the NB of the intervention was positive, conditional on the value of λ. Following McCrone et al,9 we report significance at the 10% level, because erring on the side of clinical rather than financial effectiveness is preferable.
Patient characteristics
The above exercise generates only 1 point estimate for the treatment effect for each value of λ. In order to generate an entire distribution of estimates that fully reflect the probabilistic nature of our trial, for each value of λ we estimated each of the 2 specifications (with and without controls) 5000 times with sampling replacement, a statistical approach known as bootstrapping. Then, for each value of λ, we calculated the proportion of the 5000 estimated coefficients for which the treatment effect was both greater than 0 and significant at the P < .10 level. The relationship between the proportion of positive NB values and the bootstrapping value is the CE acceptability relationship and is represented graphically. From these analyses we also calculated the expected value of NB in absolute dollars for each λ value. This relationship was then represented graphically by the relationship between λ and the estimated NB. Values of NB above $0 are considered socially acceptable.
RESULTS
Table 3 demonstrates that the patient demographic and health profiles between the intervention and control arms were similar, with the exception that patients in the APTCare arm were more likely to have made frequent visits to the clinic in the year before randomization.
Quality of care outcomes
The QOC score for chronic disease was similar at baseline in both arms: 74.1% and 76.4% for APTCare patients and controls, respectively. At the end of the study, the QOC score rose to 83.9% and 77.2% in the 2 arms, respectively. There was a significant difference in the change over the study period (P = .0013), reflecting a 9.1% (95% confidence interval [CI] 3.7% to 14.4%) improvement in the QOC associated with APTCare, which rose to 9.2% (95% CI 4.1% to 14.4%) after adjusting for potential confounders.
Service use and costs
Table 4 shows the extent of services used by the patients in the 2 arms during the study period for all patients and for those included in the economic analysis. Patients included in the economic analysis used more services. This was anticipated, given the presence of important chronic diseases in that group. Table 5 shows the costs of all services used in the 2 arms for those individuals included in the economic analysis. Costs not directly associated with the delivery of the intervention were similar in the 2 arms: $9121 and $9222 for patients enrolled in the APTCare and control arms, respectively. Costs directly associated with the program were $3802 per patient for a total cost per patient of $12 923 and $9222, respectively (P = .033). In comparison, the same costs were 9% and 16% lower in the overall study population for the APTCare and control groups, respectively.
Service use during the study period for all patients and those included in the economic evaluation
Average patient costs during the study period for those included in the economic analysis
Cost-effectiveness
The APTCare intervention was both more expensive and more effective than traditional care. The CE ratio was evaluated at $407, suggesting that for a 1% increase in QOC by means of the APTCare intervention, an investment of $407 is required:
That is, if 10 maneuvers needed to be performed for a given patient, given that on average 7.5 (baseline score) were being performed, it would require $4070 to improve the patient’s care from 7.5 to 8.5 maneuvers being performed.
Figure 1 shows 2 CE acceptability curves. For example, given a value of $500 for λ, adjusting for potential confounders, we estimate that there is an approximately 25% probability of the NB of the intervention being positive and significant. The more value the health care administration (ie, the provincial Ministry of Health) places on a 1% improvement in QOC for chronic diseases (λ), the higher the probability that the APTCare intervention will be socially beneficial.
Cost-effectiveness acceptability curve
Figure 2 displays the regression results, with the associated confidence bands, for the estimated values of the coefficient of the treatment indicator, which is interpreted as the estimated effect of the APTCare intervention on the NB in terms of absolute dollars. As one would expect, these values are negative for lower λ values. While the break-even point for the intervention occurs where λ is in the range of $300 in the analyses including the confounders, the point estimates are not statistically significant. Significance is only reached at a λ value of $750. That is, as long as society values a small improvement in QOC at a cost of at least $750 per patient, the intervention is socially acceptable.
Estimated effect of the APTCare intervention on the net benefit to society in absolute dollars
DISCUSSION
A 1% improvement in QOC was estimated to cost $407. As an interpretive example, suppose a patient has coronary artery disease or COPD and is therefore eligible for the 3 maneuvers measured (see Table 1). If, over the study period, 1 new maneuver is performed, QOC would increase by 33%. At a cost of $407 per percent, this 1 new maneuver (say getting a vaccination against influenza) would cost $13 431 ($407 × 33). Similarly, if a patient had all 4 chronic conditions (and is thus eligible for 12 maneuvers), and 1 new maneuver was performed, then QOC would have increased by 8.3%, representing a cost of $3378.
As can be seen in Figure 1, the 2 curves (without and with controls for confounders) do not coincide, suggesting that controlling for observable patient characteristics increases the probability of realizing a positive social NB. Even for high values of λ, we fail to obtain the acceptability benchmark of 80% suggested by McCrone et al10 as being the norm for a cost-effective intervention.
Using the estimated results from the NB in absolute dollars analysis (Figure 2), we arrived at a breakeven threshold of $750 when statistical significance was required. This implies that society must place a value of at least $750 on a 1% improvement in QOC in order for the intervention to be socially worthwhile. For patients with COPD, this translated into a cost of $24 750 ($750 × 33%) for patients with only COPD or $6225 ($750 × 8.3%) for patients with all 4 chronic conditions. By any of these 3 metrics, the APTCare intervention is not cost-effective, at least not in a population for which baseline QOC was already at 75%. These findings are in keeping with those of other studies reporting high costs of similar interventions.8,9 Separate analyses showed, however, that individuals with lower baseline QOC were substantially more likely to benefit from the APTCare intervention than those with already good QOC scores (results not shown). Therefore, APTCare might be more cost-effective when limited to a population with poor baseline care levels.
Caveats
The following should be taken into consideration when interpreting these findings. First, given that this was a new program, the efficiency of execution is not reflective of what could be realized in an established, up-and-running program. When members from different professional backgrounds become involved in a health care team, it takes time to become integrated and to develop productive, collaborative relationships. Costs estimated herein are almost certain to be overestimated relative to what would likely be the case from an experienced program in which team members have successfully adopted collaborative strategies.
Second, some costs (eg, hospitalization and emergency department services) were collected based on patients’ self-reports. While this can clearly cause reporting errors, the degree of resulting bias for our estimates is not known.11
Third, the sample of patients used in the study was fairly small. Thus the indirect costs are distributed over relatively few patients, which militates toward lower estimates of the NB. Finally, it should also be noted that this 12- to 18-month study (difference in duration depended on time of randomization) might not have been of sufficient duration to detect some differences in outcomes (regarding both QOC improvements and treatment costs averted), and thus some benefits of the secondary preventive maneuvers might have been underestimated. For example, benefits like prevention of amputation for patients with diabetes might not have been picked up in our analysis.
Conclusion
Further studies are required to determine the effects of mature programs on QOC outcomes. It is our hope that the caveats of this study mentioned above will be useful in executing future studies that draw upon the randomized control trial approach, as the APTCare project did. There are doubtless benefits to primary care reform involving collaborative team care, including improvement in QOC as described above. This particular benefit, however, does not meet any reasonable CE criteria. This certainly does not preclude the possibility that other benefits stemming from such team care could be cost-effective.
Acknowledgments
We thank Xue Zhao for the work done in performing the bootstrapping and in the preparation of this manuscript. Funding for this research was provided by the Ontario Ministry of Health and Long-Term Care Primary Health Care Transition Fund. The views expressed are the views of the authors and do not necessarily reflect those of the Ontario Ministry of Health and Long-Term Care.
Notes
EDITOR’S KEY POINTS
-
This study analyzed data from a randomized controlled trial of multidisciplinary team care to see if the intervention studied was cost-effective.
-
Analyses showed that the intervention was not cost-effective. However, efficiency of the newly implemented intervention was not reflective of what could be realized in an established program. Further, some costs (eg, hospitalization and emergency services) were collected based on patients’ self-reports, which can clearly cause reporting errors. The sample of patients used in the study was also fairly small, and the indirect costs are distributed over relatively few patients. Finally, the short duration of the study might not have been sufficient to detect some differences in outcomes, and some benefits might have been underestimated (eg, prevention of amputation for patients with diabetes).
-
While there are doubtless benefits to primary care reform involving collaborative team care, including improvement in quality of care as this study found, this particular intervention does not meet any reasonable cost-effectiveness criteria.
POINTS DE REPÈRE DU RÉDACTEUR
-
Cette étude analysait les données d’un essai clinique randomisé portant sur les soins prodigués par une équipe multidisciplinaire pour déterminer si l’intervention étudiée était rentable.
-
Les analyses ont montré que l’intervention n’était pas rentable. Toutefois, l’efficacité d’une intervention nouvellement instaurée ne reflète pas ce qui pourrait être obtenu avec un programme bien établi. En outre, certains coûts (p. ex. pour l’hospitalisation et les services d’urgence) provenaient des déclarations des patients, ce qui peut sûrement entraîner des erreurs. Le nombre de patients inclus dans cette étude était plutôt petit et les coûts indirects provenaient d’un nombre relativement faible de patients. Enfin, cette étude pourrait avoir été trop courte pour détecter des différences dans les issues, et certains bénéfices pourraient avoir été sous-estimés (p. ex. la prévention d’une amputation chez un diabétique).
-
Même s’il ne fait aucun doute que la réforme des soins primaires par les soins d’une équipe travaillant en collaboration est avantageuse, notamment en améliorant la qualité des soins comme le montre cette étude, l’intervention étudiée ici ne répond à aucun des critères habituels de rentabilité.
Footnotes
-
This article has been peer reviewed.
-
Contributors
All the authors contributed to concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.
-
Competing interests
None declared
- Copyright© the College of Family Physicians of Canada
