Gauging to gain

Why measure performance?

Performance measurement in primary care can serve 2 main purposes: improving quality and promoting accountability.5 Measuring performances of individuals, practices, and even regions can highlight strengths and weakness at all levels of the system and guide reform initiatives and use of resources. It can be used to provide summative reports or offer formative feedback. Different goals for performance measurement in primary care are served by different tools and approaches.

There is broad agreement that an evaluation of primary care performance includes more than measuring the technical component of care and service delivery.6,7 It should also seek to understand the structural features of the health care system, the practice context, the patient population, and the organization of the practice—factors that can influence performance.8

The underlying goal of the evaluation and its intended target audience will help shape the measurement approach. For example, a regional policy maker might rely on the ratio of providers to population, availability of after-hours semiurgent care, or wait times for appointments to evaluate performance. The Saskatchewan Health Quality Council adapts its reports to its audience: patients, policy makers, managers, and providers. The first 2 might need more summative “bottom line” information on system-wide outcomes. The latter group needs information tailored to elements within the system that can influence aspects of care under their control, such as prescribing patterns.9

National efforts

The field of performance measurement research has produced a proliferation of measurement tools and performance indicators over the past 15 years. Until recently, most performance measurement efforts focused on large groups such as hospitals or managed care organizations rather than individual providers, with the exception of certain areas in cardiology and cardiac surgery.10 Performance measurement efforts in Canada have often not included primary care. However, this is about to change.

There is increasing agreement that there should be evidence-based standards for primary care, allowing more aspects of the delivery process to be measured against benchmarks or established targets. Several groups across Canada, as well as internationally, have worked to develop and validate primary care performance indicators.11,12 The Canadian Institute for Health Information created a set of 105 pan-Canadian primary health care indicators.13 Today data are routinely collected for a small but ever increasing number of these indicators, and this is a priority area for many research and policy groups nationally.

Evaluation of performance with indicators can use data from many sources, such as provincial billing records, chart extractions, patient surveys, and even practice and population surveys (Table 1). The use of regularly collected data from large administrative databases, such as provincial billing information, offers the potential for relatively inexpensive performance measurement.

Provincial action

Some provinces have made great progress in linking various databases to provide a more complete picture of how parts of the primary care system are performing. For example, the Saskatchewan Health Quality Council used provincial data on hospitalizations, physician billing diagnostic codes, drug claims, and vital statistics to determine the percentage of patients with diabetes in the province aged 20 years and older who were prescribed medications recommended by the Canadian Diabetes Association 2003 clinical practice guidelines.14 However, there are several weaknesses in using the provincial health administrative databases for such measures. For example, they do not capture encounter information for care provided by salaried physicians or by allied health professionals. They also do not include information on aboriginal Canadians, whose health care is funded by the federal government; most of these indices still need to be validated.

Several provinces are currently measuring the health of their populations by combining such administrative data and survey information, as in Quebec’s 2006 “Portrait de Santé.”15 The use of such data will likely increase, as all provinces have committed to reporting annually on the performance of their health care systems.14 Additionally, health quality councils across Canada are reporting on the quality of key aspects of the system.

Performance measurement is also being used at the practice level to give providers specific feedback on how they compare with peers in achieving specific quality targets.16 Initiatives across the country are incorporating performance measurement into practice-level quality improvement programs. For example, an intervention program to improve chronic disease management across the Champlain Local Health Integration Network, the regional health authority in eastern Ontario, will offer primary care providers feedback on performance for well-recognized chronic disease management targets. It will also provide an on-site organization and behavioural change facilitator to help develop interventions for improvement tailored to the practice context.17

An international example of one of the most advanced systems for performance measurement in primary care is the United Kingdom’s Quality and Outcomes Framework, which offers family practitioners the opportunity to earn up to 25% of their income in bonus payments for achieving targets for performance indicators in clinical care, practice organization, and patient experience. The data are supplied by physicians and are extracted directly from computerized records. Providers can exclude inappropriate patients from the calculation of rates for a variety of reasons, such as treatment refusal.18

A word of caution

There have only been a few attempts to evaluate the effects of performance measurement, and results to date show mixed outcomes.1,18 There is little evidence to suggest that public reporting is bringing about sustained change,4 and individual providers are not sufficiently influenced by summative, publicly reported score cards.3

Despite this, performance is being measured more and more frequently. What can be measured can be rewarded, talked about, voted on, and given political weight. Wait times for cataract surgery is a recent example of an area of concern. If primary care is not measured, it might be absent in future debates.

Performance measurement breaks down primary care into discreet elements linked to evidence. Naturally, some aspects of primary care are easier and less expensive to measure than others. Measuring trust, rates of smoking cessation counseling, patient-centred care, and cultural sensitivity is very difficult. Similarly, measurements such as the percentage of patients with diabetes who have seen an ophthalmologist are readily available from the health administration databases but say little about the primary care system.

Priorities

It is essential that care for the individual is not lost. We must be mindful of the aspects of primary care that are not currently being measured and cannot be readily measured with available data sources. There are many aspects of day-to-day primary care that are not accurately captured in billing data or even with chart abstractions. 19 It is important that a full portrait of primary care service and delivery be presented. For performance measurement to work, the providers must buy into the performance ratings; this might not happen if progress is not made in developing a more comprehensive measurement of the quality of care provided. If only part of the story is told, primary care providers will get only part of the credit (and possibly resources) they are due. And as there is an increased focus on measurable targets, practitioners could become more rigid in their care by focusing more narrowly on reported measures rather than on actual quality of care provided to patients.20

While quality indicators break down the process of care into measurable elements, not all elements are of equal importance to the patient, the provider, or the community. Summary scores of quality are often not weighted for proven effect. Therefore, foot inspection for diabetes patients might be weighted equally with achieving glycemic control, leading to unbalanced perspectives and misallocation of resources.21

Finally, there is a danger of performance measurement hindering attempts to improve quality or accountability. If measurement and reporting practices create a culture of blame, are not linked with improvement intervention strategies, or create practice patterns targeting measured aspects rather than whole-patient care, quality and accountability might suffer.

Next steps

Effective primary care performance measurement systems in Canada are still in their infancy. Research into innovative, efficient data collection and reporting methods should be encouraged. Provider input is essential at all stages, from developing the indicators of quality care to collecting data to reporting and interpreting results. Family physicians should be leaders in building these processes to ensure they are compatible with the principles and goals of family medicine.18,22 Performance measurement systems must build upon and help fulfil family physicians’ obligations for self-regulation and maintenance of competency, as well as accountability to their patients and the public.

Performance measurement offers the potential to lead to better and more accountable primary care in Canada. We must ensure performance measurement does not become a distraction from but a foundation for the care we deliver.