Abstract
OBJECTIVE To develop a taxonomy of errors derived solely from the content of error reports using Canadian data from the Primary Care International Study of Medical Errors.
DESIGN Secondary analysis of data from a descriptive, cross-sectional, self-report survey.
SETTING Community-based family medicine clinics.
PARTICIPANTS Family physicians.
INTERVENTION Implementation of an error-reporting system for family medicine.
MAIN OUTCOME MEASURES Type of error, type of causal factor.
RESULTS Six types of errors or adverse events (administrative, communication, diagnostic, documentation, medication, and surgical or procedural) and 10 causal factors (case complexity, discontinuity of care, failure to follow protocol or accepted practice, fatigue, gap in knowledge, high workload, insufficient information on pharmacologic properties of medication, medication side effects, relationship dynamics, and structural problems) were identified.
CONCLUSION Our taxonomy differs from that adopted by the Primary Care International Study of Medical Errors. We propose that our taxonomy is better suited for the purposes of family physicians reporting errors in Canada.
Though Schimmel1 and Jick et al2 stressed the importance of studying patient safety up to 40 years ago, and the issue received cogent discussion by Illich,3 the topic received relatively little attention from researchers until the last decade. The Institute of Medicine in the United States reported in 2000 that between 44 000 and 98 000 deaths yearly occurred in US hospitals because of errors or adverse events.4 Davies and colleagues5 reported that 21% of 4119 patient charts screened in New Zealand hospitals contained evidence of adverse events. Wilson et al6 reported that adverse events were associated with 17% of hospital admissions in Australia. In acute care Canadian hospitals, errors or adverse events were estimated to occur in 7.5% of patient admissions each year.7
Much of the variation in these findings derives from the diverse definitions used in identifying cases and the varied methods of assessing incidence.8 Nonetheless, the evidence indicates in general that medical errors and adverse events are of concern and can have serious effects.9–11
Nearly all research into the topic has involved data from hospital settings. Given that most contact between patients and the health system takes place in community-based primary care, and that general practitioners see a greater variety of conditions and patients than specialists do,12,13 research relating to the type and frequency of errors in community-based primary care is conspicuously lacking. The “patient-centred” typology of errors in family medicine by Kuzel et al14 is an exception, but because it focuses on lay perceptions of error, we believe that considerable domains of clinical and systemic knowledge are excluded from their classification.
The Primary Care International Study of Medical Errors (PCISME)15 solicited general practitioners in Australia, New Zealand, Canada, the Netherlands, the United Kingdom, and the United States to report errors they made or encountered in day-to-day practice. Although these countries have different health care systems, they share “first-world” standards of medical care. Canadian data were reported by Rosser et al.16 The taxonomy used in the PCISME study was adopted before the data were analyzed17,18 and has not been replicated nor validated either generally or specifically in Canada.
This paper reports a re-examination of the Canadian data from the PCISME study with the aim of either establishing the validity of the PCISME international taxonomy in Canada or creating a valid Canadian variant. To our knowledge, while other national data sets from the PCISME study have been reported,19,20 no other country has sought to reevaluate the original taxonomy of errors applied in that study (though some countries have indicated through personal communication that the original taxonomy does not suit their own national data). Also, no existing taxonomy of medical errors or adverse events in primary care has been subjected to statistical validation.
METHOD
The techniques of sampling and data collection used in the PCISME project have been described previously.15 In Canada, study participants were mainly drawn from the North Toronto Primary Care Research Network, a group of community-based family physicians, many of whom hold both hospital and university appointments. Six physicians were drawn from rural Ontario. A total of 84 reports were submitted by a maximum of 22 Canadian doctors. Because reporting was anonymous, the number of physicians who actually submitted reports is unknown.
Study participants were asked to report, either electronically or on paper, any episodes in their practices that they considered to be errors according to the following definition:
Errors are events in your practice that made you conclude, “That was a threat to patient well-being and should not happen. I don’t want it to happen again.” Such an event affects or could affect the quality of care you give your patients. Errors can be large or small, administrative or clinical, or actions taken or not taken. Errors might or might not have discernible effects. Errors in this study are anything you identify as something wrong, to be avoided in the future.
This definition mixes “errors” (preventable and correctable events that do not necessarily cause harm) with “adverse events” (incidents that cause harm, but might or might not be preventable) and reflects the unclear usage from which these terms commonly suffer.21–23 We considered all the reports to refer to “error” according to the definition provided. A similar approach has been taken in several reports based on the same database.15,16,19
Using the Canadian data, 2 of the authors (S.J. and L.V.) independently, and with no knowledge of the preliminary international taxonomy developed by Dovey et al,24 devised working taxonomies based on the original error reports. All the authors then discussed points of discrepancy between the 2 working taxonomies until consensus was reached on individual types of errors. The same process was undertaken in relation to reports attributing cause(s) to each error. Reports could be coded for more than 1 type of error or causal factor due to the complexity of many of the episodes.
With the taxonomy agreed to in principle, 2 judges not involved in the project, a third-year linguistics student and a third-year psychology student, independently assigned each error and causation report to the taxonomy. These judges received no training as to how episodes should be classified and were told simply to consider each report and to classify it in 2 ways using the definitions identified for the taxonomy: first according to type or types of errors and second according to type or types of causal factors. Their judgments were then tested for inter-rater reliability.
Approval for the original Canadian arm of the PCISME study was granted by the Research Ethics Board of the University of Toronto in Ontario. Dr Rosser was Canadian principal investigator and Dr Drummond was a co-investigator in that study.
RESULTS
Six types of errors were identified (Table 1). No reported errors were considered unclassifiable. Our choice to distinguish medication errors from surgical or procedural errors was influenced by several factors. First, the large number of medication errors reported indicated that a unique category was warranted. Second, reports relating to medications all contained errors of commission (eg, writing prescriptions) rather than errors of omission, which were common to surgical and procedural reports (ie, failing to do something). Third, medication-related episodes share a considerable element of hindsight bias,21 as they are seldom recognized until a pharmacologic reaction occurs. Last, our categorization follows a convention in the literature to treat pharmacologic errors or adverse events as distinct from others.
Types of errors
In identifying causal factors (Table 2), we did not attempt to interpret the attributions cited by physicians in their reports by “reading between the lines” in order to understand what really happened. Where uncertainty existed as to the cause of the error, we referred to physicians’ suggestions regarding future preventive measures: the proposed solution invariably clarified what the problem was perceived to be.
Types of causal factors
To validate the taxonomies, the classifications created by our judges were analyzed to determine inter-rater reliability (Table 3). A reliability measurement was calculated for each type of error using the kappa statistic. This is based on observed and expected percentages of concordant responses. A kappa value >0.75 denoted high reproducibility, a value of 0.4 to 0.75 (inclusive) denoted good reproducibility, and a value <0.4 denoted marginal or poor reproducibility. To explore whether 1 judge tended to identify more examples of errors or causal factors than the other, a McNemar test was done for each type of episode based on discordant pairs: a probability >.05 indicated that 1 rater was not more likely to identify a type of error or causal factor more often than the other.
Inter-rater reliability for types of errors or adverse events and causal factors
DISCUSSION
Construction of taxonomies in health care has become an important feature of health services research. Taxonomists aim to enable systematic comparison of groups of items by identifying and classifying them according to common characteristics (phenetic method) or common ancestry (cladistic method).25 Taxonomies have been developed to explain, for example, shared care models,26 physicians’ attitudes to end-of-life care,27 and primary care management alternatives.28 Taxonomies promote recognition of diversity and similarity and provide explanatory power by defining and identifying contributory factors. A generally agreed upon and validated taxonomy is a requisite step toward preventing and reducing errors and adverse events. Despite the current interest in health care taxonomies, they have rarely been validated through statistical analysis.
Our proposed taxonomy identifies 6 types of errors and 10 causal factors contributing to them. Analysis indicated that neither judge was more likely than the other to identify either a type of error or a causal factor and that their judgments were reliable for all errors and all causal events except “failure to follow protocol” and “gap in knowledge.” Our findings indicate that the taxonomy is robust as to types of errors (in relation to the kappa statistic), though not so robust as to types of causal factors. Woolf and colleagues20 suggest that cascade analysis, which examines the chain of events leading to an error, might be required to determine cause. We believe our taxonomy of causal factors requires further study.
Our taxonomy is distinct from that proposed by Dovey et al,24 which recognizes 171 types of errors grouped into 5 main categories. Differences could be attributable to 2 factors. First, there is more than 1 but not an infinite number of ways to classify any set of items or events. For example, focusing on Australian general practitioners, Britt and colleagues29 used methods similar to those used in the PCISME study and arrived at a taxonomy containing 4 types of “mishaps” and 27 “contributing factors.” It is unlikely that the types of errors and causal factors encountered by Australian physicians are inherently different from those encountered by the international PCISME sample. Differences probably reflect a dichotomy among taxonomists between the “lumpers” and the “splitters”: the categories of “diagnostic imaging errors” (process errors) and “delay in diagnosis” (knowledge and skills errors) described by Dovey et al24 could simply be merged within the category “diagnostic errors” described by Britt and colleagues.29
In attempting to produce a taxonomy that is conducive to self-reporting, we have devised a relatively short taxonomy that accounts for all the data. Although other short typologies exist, we believe that these either are too terse, without categories that account for communication, administration, and documentation,29 or contain an uninformative or misleading taxon that lumps “other” types of errors.
A second reason our taxonomy differs from that of Dovey et al24 might be that we considered it important to classify errors on the basis of their shared characteristics rather than their shared ancestry, because the latter could suffer from combining cause (eg, being fatigued) with effect (eg, writing a prescription with the wrong dosage because of fatigue). To satisfy our phenetic criterion, we were careful to distinguish between types of errors and the causal factors contributing to them. This not only reflects the structure of the error reports themselves, which asked physicians to explain what happened and then what contributed to the episode, but also makes for more straightforward reporting.
It is interesting to note that in the few studies that have examined cause of error, as distinct from type, the factors identified are similar to ours. Ely et al18 suggest, based on interviews with physicians, that memorable errors can be organized into 4 causal categories: physician stressors (eg, workload), process-of-care factors (eg, discontinuity of care), patient-related factors (eg, case complexity), and physician characteristics (eg, lack of knowledge). Our causal categories are also broadly congruent with those identified by Dean et al30 and Neale et al31 for inpatient hospital settings and with those of Jacobson et al21 for primary care. This suggests that physicians probably have many reasons in common for committing errors, regardless of context, although the importance of any causal factor might vary from setting to setting, and some causes might be uniquely tied to particular structural factors and practice styles.
Though our taxonomy does not replicate the findings of Dovey et al,24 this does not make their taxonomy invalid. We contend that our variant is better suited to the circumstances of family physicians in Canada. Much attention has been paid to the “systems approach” to medical errors and adverse events, which is drawn from work on safety in aviation and other organizations facing complex human and technological factors.32 The systems approach uses techniques of incident monitoring and root cause analysis to identify and mitigate errors in a no-blame environment. Although some incident monitoring has taken place in family medicine,29,33,34 without a generally agreed-upon and valid taxonomy of errors, results of such studies are difficult to compare.
With the trend toward increased error reporting in mind, we believe that the taxonomy presented here is appropriate as the basis of an error reporting system for Canadian family medicine and primary care. We are currently developing a system for Alberta and plan to incorporate the taxonomy into a quick and simple paper-, Web-, and telephone-based reporting designs that could be extended to include near misses, “safe catches” (latent errors22), and other types of hazards. Further development and validation of the taxonomy, especially in relation to other primary care practitioners (eg, nurses and pharmacists), will be an important aspect of this work.
Limitations
We acknowledge some limitations in relation to this study. The sample size was relatively small and limited to Ontario, and the data were limited in terms of detail to what participating physicians elected to report. While our study team consisted of 2 family physicians who contributed to development of the taxonomy, limited resources allowed for only 2 judges during the validation process. In using nonphysicians for this task, we consciously decided that to do so helped establish that this form of classification was relevant and feasible in other locations and without specialized knowledge. We believe these qualities are essential for establishing an open, no-blame culture of safety in Canadian family medicine and primary care.
Conclusion
We propose that our taxonomy of errors and adverse events for community-based family medicine is relevant, feasible, and valid for the circumstances of Canadian family physicians.
Acknowledgment
This study was supported in Canada by members of Nortren (North Toronto Primary Care Research Network) and by 6 additional anonymous Ontario family physicians. Nortren is supported by the Primary Care Research Unit at Sunnybrook Health Sciences Centre, and by North York General Hospital, Scarborough Hospital, and the Department of Family and Community Medicine at the University of Toronto in Ontario.
Notes
EDITOR’S KEY POINTS
-
Errors are common in family medicine, as in other areas of medical practice.
-
To reduce errors, it is important to first understand the types of errors that occur and then the factors that contribute to them.
-
This study developed a uniquely Canadian family medicine taxonomy of errors that might facilitate error reporting and subsequently the development of processes to prevent errors.
POINTS DE REPÈRE DU RÉDACTEUR
-
Les erreurs sont fréquentes en médecine familiale comme dans les autres domaines de la pratique médicale.
-
Pour réduire ces erreurs, il importe de bien comprendre les différents types d’erreurs ainsi que les facteurs qui y contribuent.
-
Les auteurs ont créé une taxonomie des erreurs propre à la médecine familiale canadienne qui est susceptible de faciliter la déclaration des erreurs et de permettre l’élaboration subséquente de moyens de les prévenir.
Footnotes
-
This article has been peer reviewed.
-
Contributors: Drs Drummond and Rosser conceived and designed the study, analyzed and interpreted the data, and prepared the article for submission. Ms Jacobs, Dr O’Beirne, Ms Derflingher, and Ms Vlach contributed to data analysis and interpretation and critically revised the manuscript. All the authors approved the final text of the article.
-
Competing interests: None declared
- Copyright© the College of Family Physicians of Canada
References
In this issue
Jump to section
Related Articles
Cited By...
- Validation of the Primary Care Patient Measure of Safety (PC PMOS) questionnaire
- Developing a primary care patient measure of safety (PC PMOS): a modified Delphi process and face validity testing
- Safety incidents in family medicine
- Safety learning system development--incident reporting component for family practice