CFP
HOME HELP CONTACT US FEEDBACK SUBSCRIPTIONS CURRENT ISSUE PAST ISSUES SEARCH
QUICK SEARCH:   [advanced]


     

Can Fam Physician
Vol. 54, No. 9, September 2008, pp.1215 - 1217
Copyright © 2008 by The College of Family Physicians of Canada
This Article
Right arrow Full Text (PDF)
Right arrow Rapid Responses: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when Rapid Responses are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Johnston, S.
Right arrow Articles by Hogg, W.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Johnston, S.
Right arrow Articles by Hogg, W.

Commentary

Gauging to gain

Primary care performance measurement

Sharon Johnston, CCFP MD LLM
The C.T. Lamont Primary Health Care Research Centre and an Assistant Professor in the Department of Family Medicine at the University of Ottawa in Ontario.

Simone Dahrouge, MClSc
The C.T. Lamont Primary Health Care Research Centre at the University of Ottawa.

William Hogg, FCFP MSc MClSc MD CM
The C.T. Lamont Primary Health Care Research Centre and the Institute for Population Health and is a Professor in the Department of Family Medicine at the University of Ottawa.

Correspondence to: Dr Sharon Johnston, Department of Family Medicine, University of Ottawa, 43 Bruyère St, Ottawa, ON K1N 5C8; e-mail sjohnsto{at}scohs.on.ca

Until our performance-measurement system is based on clinically relevant information and targets high-priority care, performance measurement is likely to remain a great idea that is more of a distraction than a benefit.
Rodney A. Hayward1

High-quality and accountable health care is everyone’s business.2 Over the past few years, performance measurement in health care has been heralded as a key to improving quality and reducing errors, encouraging greater efficiency,3 and improving government accountability for the largest segment of budget expenditure.4 Family physicians should embrace this movement and participate in its design to ensure it does lead to better health care.

Why measure performance?

Performance measurement in primary care can serve 2 main purposes: improving quality and promoting accountability.5 Measuring performances of individuals, practices, and even regions can highlight strengths and weakness at all levels of the system and guide reform initiatives and use of resources. It can be used to provide summative reports or offer formative feedback. Different goals for performance measurement in primary care are served by different tools and approaches.

There is broad agreement that an evaluation of primary care performance includes more than measuring the technical component of care and service delivery.6,7 It should also seek to understand the structural features of the health care system, the practice context, the patient population, and the organization of the practice—factors that can influence performance.8

The underlying goal of the evaluation and its intended target audience will help shape the measurement approach. For example, a regional policy maker might rely on the ratio of providers to population, availability of after-hours semiurgent care, or wait times for appointments to evaluate performance. The Saskatchewan Health Quality Council adapts its reports to its audience: patients, policy makers, managers, and providers. The first 2 might need more summative "bottom line" information on system-wide outcomes. The latter group needs information tailored to elements within the system that can influence aspects of care under their control, such as prescribing patterns.9

National efforts

The field of performance measurement research has produced a proliferation of measurement tools and performance indicators over the past 15 years. Until recently, most performance measurement efforts focused on large groups such as hospitals or managed care organizations rather than individual providers, with the exception of certain areas in cardiology and cardiac surgery.10 Performance measurement efforts in Canada have often not included primary care. However, this is about to change.

There is increasing agreement that there should be evidence-based standards for primary care, allowing more aspects of the delivery process to be measured against benchmarks or established targets. Several groups across Canada, as well as internationally, have worked to develop and validate primary care performance indicators.11,12 The Canadian Institute for Health Information created a set of 105 pan-Canadian primary health care indicators.13 Today data are routinely collected for a small but ever increasing number of these indicators, and this is a priority area for many research and policy groups nationally.

Evaluation of performance with indicators can use data from many sources, such as provincial billing records, chart extractions, patient surveys, and even practice and population surveys (Table 1). The use of regularly collected data from large administrative databases, such as provincial billing information, offers the potential for relatively inexpensive performance measurement.


View this table:
[in this window]
[in a new window]

  		Table 1 Examples of performance indicators for primary care

 
Provincial action

Some provinces have made great progress in linking various databases to provide a more complete picture of how parts of the primary care system are performing. For example, the Saskatchewan Health Quality Council used provincial data on hospitalizations, physician billing diagnostic codes, drug claims, and vital statistics to determine the percentage of patients with diabetes in the province aged 20 years and older who were prescribed medications recommended by the Canadian Diabetes Association 2003 clinical practice guidelines.14 However, there are several weaknesses in using the provincial health administrative databases for such measures. For example, they do not capture encounter information for care provided by salaried physicians or by allied health professionals. They also do not include information on aboriginal Canadians, whose health care is funded by the federal government; most of these indices still need to be validated.

Several provinces are currently measuring the health of their populations by combining such administrative data and survey information, as in Quebec’s 2006 "Portrait de Santé."15 The use of such data will likely increase, as all provinces have committed to reporting annually on the performance of their health care systems.14 Additionally, health quality councils across Canada are reporting on the quality of key aspects of the system.

Performance measurement is also being used at the practice level to give providers specific feedback on how they compare with peers in achieving specific quality targets.16 Initiatives across the country are incorporating performance measurement into practice-level quality improvement programs. For example, an intervention program to improve chronic disease management across the Champlain Local Health Integration Network, the regional health authority in eastern Ontario, will offer primary care providers feedback on performance for well-recognized chronic disease management targets. It will also provide an on-site organization and behavioural change facilitator to help develop interventions for improvement tailored to the practice context.17

An international example of one of the most advanced systems for performance measurement in primary care is the United Kingdom’s Quality and Outcomes Framework, which offers family practitioners the opportunity to earn up to 25% of their income in bonus payments for achieving targets for performance indicators in clinical care, practice organization, and patient experience. The data are supplied by physicians and are extracted directly from computerized records. Providers can exclude inappropriate patients from the calculation of rates for a variety of reasons, such as treatment refusal.18

A word of caution

There have only been a few attempts to evaluate the effects of performance measurement, and results to date show mixed outcomes.1,18 There is little evidence to suggest that public reporting is bringing about sustained change,4 and individual providers are not sufficiently influenced by summative, publicly reported score cards.3

Despite this, performance is being measured more and more frequently. What can be measured can be rewarded, talked about, voted on, and given political weight. Wait times for cataract surgery is a recent example of an area of concern. If primary care is not measured, it might be absent in future debates.

Performance measurement breaks down primary care into discreet elements linked to evidence. Naturally, some aspects of primary care are easier and less expensive to measure than others. Measuring trust, rates of smoking cessation counseling, patient-centred care, and cultural sensitivity is very difficult. Similarly, measurements such as the percentage of patients with diabetes who have seen an ophthalmologist are readily available from the health administration databases but say little about the primary care system.

Priorities

It is essential that care for the individual is not lost. We must be mindful of the aspects of primary care that are not currently being measured and cannot be readily measured with available data sources. There are many aspects of day-to-day primary care that are not accurately captured in billing data or even with chart abstractions. 19 It is important that a full portrait of primary care service and delivery be presented. For performance measurement to work, the providers must buy into the performance ratings; this might not happen if progress is not made in developing a more comprehensive measurement of the quality of care provided. If only part of the story is told, primary care providers will get only part of the credit (and possibly resources) they are due. And as there is an increased focus on measurable targets, practitioners could become more rigid in their care by focusing more narrowly on reported measures rather than on actual quality of care provided to patients.20

While quality indicators break down the process of care into measurable elements, not all elements are of equal importance to the patient, the provider, or the community. Summary scores of quality are often not weighted for proven effect. Therefore, foot inspection for diabetes patients might be weighted equally with achieving glycemic control, leading to unbalanced perspectives and misallocation of resources.21

Finally, there is a danger of performance measurement hindering attempts to improve quality or accountability. If measurement and reporting practices create a culture of blame, are not linked with improvement intervention strategies, or create practice patterns targeting measured aspects rather than whole-patient care, quality and accountability might suffer.

Next steps

Effective primary care performance measurement systems in Canada are still in their infancy. Research into innovative, efficient data collection and reporting methods should be encouraged. Provider input is essential at all stages, from developing the indicators of quality care to collecting data to reporting and interpreting results. Family physicians should be leaders in building these processes to ensure they are compatible with the principles and goals of family medicine.18,22 Performance measurement systems must build upon and help fulfil family physicians’ obligations for self-regulation and maintenance of competency, as well as accountability to their patients and the public.

Performance measurement offers the potential to lead to better and more accountable primary care in Canada. We must ensure performance measurement does not become a distraction from but a foundation for the care we deliver.

Footnotes

Cet article se trouve aussi en français à la page 1220

Competing interests

None declared

The opinions expressed in commentaries are those of the authors. Publication does not imply endorsement by the College of Family Physicians of Canada.

References

  1. Hayward RA. Performance measurement in search of a path. N Engl J Med 2007;356(9):951–3.[Free Full Text]
  2. Health Canada. First Ministers’ health communiqué: accountability and reporting. Ottawa, ON: Health Canada; 2000. Available from: www.tbs-sct.gc.ca/rma/eppi-ibdrp/hrs-ceh/5/FMH-PMS_e.asp. Accessed 2008 Jul 23.
  3. Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States and the United Kingdom. Health Aff (Millwood) 2003;22(3):134–48.[Abstract/Free Full Text]
  4. Brown AD, Porcellato C, Barnsley J. Accountability: unpacking the suitcase. Healthc Q 2006;9(3):72–5.[Medline]
  5. Freeman T. Using performance indicators to improve health care quality in the public sector: a review of the literature. Health Serv Manage Res 2002;15(2):126–37.[Abstract/Free Full Text]
  6. Donabedian A. Evaluating the quality of medical care. 1966. Milbank Q 2005;83(4):691–729.[Medline]
  7. Watson D, Broemeling A, Reid R, Black C. A results-based logic model for primary health care. Vancouver, BC: Centre for Health Services and Policy Research; 2004. Available from: www.chspr.ubc.ca/files/publications/2004/chspr04-19.pdf. Accessed 2008 Jul 23.
  8. Hogg W, Rowan M, Russell G, Geneau R, Muldoon L. Framework for primary care organizations: the importance of a structural domain. Int J Qual Health Care; 2008 Jun, 13. Epub ahead of print.
  9. Chan BT, Smadu M, McMillan JS. Quality councils as catalysts and leaders in quality improvement: the experience of the health quality council in Saskatchewan. Healthc Pap 2006;6(3):38–45.[Medline]
  10. Epstein AM, Lee TH, Hamel MB. Paying physicians for high-quality care. N Engl J Med 2004;350(4):406–10.[Free Full Text]
  11. Barnsley J, Berta W, Cockerill R, MacPhail J, Vayda E. Identifying performance indicators for family practice: assessing levels of consensus. Can Fam Physician 2005;51:700–1, e1–7. Available from: www.cfp.ca/cgi/reprint/51/5/700. Accessed 2008 Aug 6.[Abstract/Free Full Text]
  12. McGlynn EA, Asch SM. Developing a clinical performance measure. Am J Prev Med 1998;14(3 Suppl):14–21.[Medline]
  13. Canadian Institute for Health Information. Pan-Canadian primary health care indicators. Ottawa, ON: Canadian Institute for Health Information; 2006. Available from: http://secure.cihi.ca/cihiweb/dispPage.jsp?cw_page=GR_1489_E. Accessed 2008 Jul 23.
  14. Health Canada. 2003 First Ministers’ accord on health care renewal. Ottawa, ON: Health Canada; 2006. Available from: www.hc-sc.gc.ca/hcs-sss/delivery-prestation/fptcollab/2003accord/index-eng.php. Accessed 2008 Jul 23.
  15. Institut national de santé publique du Québec. Portrait de santé du Québec et de ses régions 2006: deuxième rapport national sur l’état de santé de la population du Québec. Quebec, QC: Institut national de santé publique du Québec; 2006. Available from: www.inspq.qc.ca/pdf/publications/546-PortraitSante2006_Analyses.pdf. Accessed 2008 Jul 23.
  16. Rowan MS, Hogg W, Martin C, Vilis E. Family physicians’ reactions to performance assessment feedback. Can Fam Physician 2006;52:1570–1, e1–6. Available from: www.cfp.ca/cgi/reprint/52/12/1570. Accessed 2008 Aug 6.[Abstract/Free Full Text]
  17. Champlain Cardiovascular Disease Prevention Network. Improved delivery of cardiovascular preventive care through outreach facilitation. Ottawa, ON: Champlain Cardiovascular Disease Prevention Network; 2007. Available from: www.ccpnetwork.ca/priorities/primary-care_e.pdf. Accessed 2008 Jul 23.
  18. Roland M. Linking physicians’ pay to the quality of care—a major experiment in the United Kingdom. N Engl J Med 2004;351(14):1448–54.[Free Full Text]
  19. National Primary Care Research and Development Centre, RAND Corporation. Measuring general practice: a demonstration project to develop and test a set of primary care clinical quality indicators. London, Engl: The Nuffield Trust; 2003. Available from: www.rand.org/pubs/monograph_reports/2005/MR1725.pdf. Accessed 2008 Jul 23.
  20. Werner RM, Asch DA. Clinical concerns about clinical performance measurement. Ann Fam Med 2007;5(2):159–63.[Abstract/Free Full Text]
  21. Casalino LP. The unintended consequences of measuring quality on the quality of medical care. N Engl J Med 1999;341(15):1147–50.[Free Full Text]
  22. Lee TH, Meyer GS, Brennan TA. A middle ground on public accountability. N Engl J Med 2004;350(23):2409–12.[Free Full Text]




This Article
Right arrow Full Text (PDF)
Right arrow Rapid Responses: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when Rapid Responses are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Johnston, S.
Right arrow Articles by Hogg, W.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Johnston, S.
Right arrow Articles by Hogg, W.

HOME HELP CONTACT US FEEDBACK SUBSCRIPTIONS CURRENT ISSUE PAST ISSUES SEARCH