Abstract
Objective To evaluate the link between the quantity and quality of continuing professional development (CPD) activities completed by family physicians in Quebec and the quality of their practice.
Design Retrospective analysis of data collected during professional inspection visits (PIVs).
Setting Quebec.
Participants Three groups were created from among Quebec family physicians who had been subject to PIVs (peer evaluation) by the Collège des médecins du Québec between 1998 and 2005. Group 1 was composed of physicians who were members of the College of Family Physicians of Canada, which requires participation in 250 hours of CPD in every 5-year cycle. Group 2 was composed of family physicians who were not members of the College of Family Physicians of Canada but who had declared at least 50 hours a year of CPD on their Collège des médecins du Québec annual notice of assessment for the same period. Group 3 was composed of family physicians who had declared fewer than 10 hours of CPD a year.
Main outcome measures During the PIV, the following characteristics were examined: record keeping, quality and number of hours of CPD activities, and quality of professional practice based on 3 components— clinical investigation, accuracy of diagnosis, and appropriateness of treatment plan and follow-up.
Results The factors associated with a high quality of practice were privileges in a hospital or local community health centre (institution) and a substantial number of accredited CPD hours (Mainpro-M1, Credit I, or Mainpro-C). The factors associated with a poor quality of practice were advanced age of the physician, absence of privileges in an institution (hospital or local community health centre), and participation in CPD activities that were more informal, such as reading and non-accredited activities (Mainpro-M2).
Conclusion This study supports earlier research showing that CPD activities of sufficient quality and quantity are correlated with a high quality of professional practice by family physicians.
Continuing professional development (CPD) includes all of the means used by physicians to maintain and enhance their competencies in order to improve the health of patients.1
For nearly 25 years, educators and professional associations have taken an interest in the effects of CPD activities on clinical performance.2–8 Several studies have attempted to better understand the characteristics of CPD that are associated with better results in the clinical performance of physicians. Davis and colleagues and Mansouri and Lockyer identified a close link between the intensity of the strategies used and their effects on clinical performance.3–6 Thus, CPD centred on interaction and active participation (eg, case discussion, role-play, hands-on practice sessions) would effect changes in the performance of physicians. These changes were less marked for didactic CPD activities limited only to the transmission of knowledge (eg, lectures, seminars). On the other hand, McLeod and McLeod suggest that the ideal approach is a mixture of formal and informal CPD.9
Because it has been demonstrated that didactic-type interventions produce modest results, one would be inclined to believe that their use remains limited. But they are in fact the most used form of CPD.9–11 Given the limited scope of certain educational activities, one might question what motivates physicians and authorities to persist with this type of continuing education or its promotion. Unquestionably, their cost and the ease with which they can be organized are the most likely explanations.5,9 It is also well known that physicians continue to participate in these in order to collect information, to be reassured about the quality of their practice, and to network with colleagues. Although certain types of activities have failed to demonstrate positive results, a number of studies have shown the benefits of interactive and more personalized activities. Therefore, we can expect that physicians who accumulate a substantial number of varied CPD activities will be better practitioners.
The College of Family Physicians of Canada (CFPC) is a national voluntary organization of family physicians that makes CPD mandatory as a condition of membership.12 Considering that CPD is generally the key to maintaining competence, the CFPC resolved to impose a continuing education standard on its members. It holds that by encouraging physicians to perfect their knowledge and skills, Canadian patients will inevitably reap the benefits.
The CFPC has divided CPD activities into 3 broad categories. There are activities that are generally didactic in nature such as conferences, workshops, presentations, or teaching sessions (Mainpro-M1); activities involving reflection on the effects of an educational activity on practice and follow-up (Mainpro-C); and individual activities (no accreditation process needed) such as reading, discussions with colleagues, or the creation of forms to be integrated into patient records (Mainpro-M2). The value of CPD activities is expressed in the form of credits, and practitioners must report 250 hours over a period of 5 years to meet the requirements of the CFPC and keep their status as members. These 250 hours must include a minimum of 125 Mainpro-M1 or Mainpro-C credits, and a maximum of 125 Mainpro-M2 credits.13
Many studies have focused on the influence of CPD-type activities on the practice of physicians. Yet, the link between the type and intensity (number of hours) of CPD activities and the quality of medical practice has never been clearly established. It is estimated that most physicians already practise appropriately and in an up-to-date manner14,15; thus, the effect on practice of a single CPD activity is hard to measure because of what we call the ceiling effect. It is difficult to measure difference when the difference to be measured is very slight. Furthermore, most physicians continue to participate in CPD activities of all kinds so as to be reassured about the quality of their practice. Evaluating the effect of one CPD activity on clinical performance might therefore show no difference if the competence of the participating physicians is already satisfactory.14 Also, most of the instruments used to measure the effects of a single CPD activity on competence or clinical performance focus on changes in behaviour that relate specifically to the learning objectives established for the activity. Changes and improvements that do not relate to these objectives, or changes that result from informal discussions between peers attending the same activity, are often considered a side benefit of the CPD activity and are not formally evaluated.
The relationship between the quantity and quality of CPD activities and the quality of practice has already been addressed incidentally in an earlier article, but without any further analyses.14 The present study intends to examine more specifically the link between the number of hours and the quality (type and relevance) of CPD and the clinical performance of family physicians as measured during professional inspection visits (PIVs), which involve a peer-review process.
METHODS
Subjects
The project was approved by the ethics committee of the Cité-de-la-Santé de Laval in Quebec. The subjects studied were a non-random sample of registered family physicians who had been subject to PIVs between 1998 and 2005, inclusive. The PIV program is a peer-review process for evaluating professional practice based on the candidate’s patient records, as well as a structured chart-stimulated interview.15 The PIVs are included in the regular programs of the Professional Inspection Committee of the Collège des médecins du Québec (CMQ). In order to evaluate the link between CPD activities and quality of medical practice, 3 CPD groups were created. The first group was composed of physicians who were CFPC members who had participated for a 5-year period in the CPD program of the CFPC. All physicians who had received PIVs and who were members of the CFPC were included in the study. The 2 other groups were random samples of physicians paired according to the professional inspection program that had led to their PIVs. The second group was composed of physicians who were not members of the CFPC but who had declared a minimum of 50 CPD hours per year in the 5 years preceding their PIVs on their CMQ annual notice of assessment. This CPD could have consisted of participation in formal group activities (eg, symposia, courses, conferences, lectures, workshops) approved and accredited by the CFPC or by other Quebec accredited CPD organizations (Fédération des médecins omnipraticiens du Québec, Fédération des médecins spécialistes du Québec, Médecins francophones du Canada, Quebec medical schools, and the Royal College of Physicians and Surgeons of Canada) or informal individual or group activities (eg, reading, video learning, Internet learning). The third group included physicians who were not CFPC members and who had declared little (less than 10 hours annually) or no CPD activity for at least 1 year during the 5-year period preceding their PIVs.
Professional inspection visit programs
During the period studied, there were 6 main programs that led to PIVs. These were directed at physicians who were methadone prescribers; those who had graduated more than 35 years ago or who were older than 65 years of age; those who had been the subject of one or more complaints made to the Inquiries Division of the CMQ; those without privileges in hospitals or local community health centres; physicians who were chosen randomly by the Professional Inspection Committee or who had failed their licensing examinations more than twice; and physicians about whom information had been received raising concerns regarding the quality of their practice. A seventh program brought together physicians from different professional inspection programs with small numbers of physicians. This last group included physicians who had changed their professional address more than twice, those who had acted as replacement physicians in remote regions, those who had changed their field of practice, those who had renewed their restrictive licences, those referred by the review committee, those practising cosmetic medicine, and those practising psychotherapy more than 25% of the time, performing therapeutic acts more than 60% of the time, or practising outside of their specialties more than 30% of the time.
Evaluation of practice
During a PIV, particular aspects of medical practice were evaluated using a combination of explicit and implicit criteria to assess patient records and the content of a semistructured interview with the physician visited. The inspectors evaluated and assigned a CPD score to each physician based on the quantity and quality of CPD activities. The quality of CPD activities was assessed mainly in terms of their relevance to the physician’s practice and their potential for interaction or active participation.
The quality of medical record keeping and the physician’s overall quality of clinical practice were also assessed. Overall quality of practice was divided into 3 components: clinical investigation (history taking, physical examination, and investigation plan), accuracy of diagnosis, and pharmacologic and nonpharmacologic treatments as well as patient follow-up. These aspects were evaluated using simple scores (1 = satisfactory, 2 = unsatisfactory, 3 = not evaluated, 4 = not measurable). Recommendations were generally made to the physician after the PIV report was written. The tools used to assess physician performance in PIVs have been shown to be valid and reliable.14,15
Statistical analysis
The data were collected retrospectively from physicians’ files kept in the Practice Enhancement Division of the CMQ. Sociodemographic characteristics (ie, sex, age, country where the medical degree was obtained, year of graduation, and place of practice) and various aspects of the practice of physicians in the 3 groups were compared using χ2 tests or Fisher exact tests, as appropriate. Post hoc pairwise comparisons were also performed among the 3 groups.
The components of quality of clinical practice were also compared among groups using χ2 tests. In order to create a quality score that reflected the overall competence of physicians, a composite score was developed for which 1 point was assigned for each of the 3 components of quality of clinical practice that was evaluated as satisfactory. Accordingly, a physician whose overall clinical practice was judged satisfactory received the maximum of 3 points. A stepwise multiple regression model was applied to this composite score in order to identify the associated variables and their unique contributions. The following independent variables were used: CPD group, age group, sex, country of graduation, PIV program, principal place of practice, and completion of a residency in family medicine. The CPD groups and PIV programs were coded as dummy variables. An ANOVA (analysis of variance) test on main effects without interaction was used to estimate F and mean-square values.
All tests were bilateral and considered statistically significant at an α level of 5%. No correction was made for multiple comparisons.
Professional development activities
For members of the CFPC, CPD activities fall into 2 categories. Mainpro-M1 and Mainpro-C activities are group or individual activities that have been accredited by the CFPC or by one of its provincial chapters. The activities must follow an educational process that includes needs assessment, identification of educational objectives, a learning method adapted to the objectives, evaluation of the activity, and respect for a CPD code of ethics. The other category, Mainpro-M2 CPD activities, includes mainly individual activities that have not been accredited.
For non-members of the CFPC, various Quebec organizations that are authorized and accredited by the CMQ to provide CPD offer accredited activities (credits I). Organizations seeking such authorization must follow an educational process similar to that of the CFPC in order to grant credits for educational activities.
RESULTS
Sociodemographic data
The characteristics of the 3 CPD groups are presented in Table 1. With respect to the age distribution of physicians affiliated with each of the categories, the proportion of physicians younger than 50 years of age was 53% in the first group (CFPC members), 39% in the second group (non-members with CPD), and 26% in the third group (non-members with little or no CPD). The average (SD) age of the first group was 51.7 (13.6) years, compared with 53.9 (9.2) years for the second group, and 58.7 (12.8) years for the third group. The difference with respect to age distribution between the 3 groups was statistically significant (P < .01).
Most physicians in the study were men (70% in group 1, 84% in group 2, and 85% in group 3). A significant sex difference was observed between groups 1 and 2 and between groups 1 and 3 (P = .03). As for country of graduation, the proportion of physicians who had obtained their medical degrees in Canada was 71% in group 1, 75% in group 2, and 84% in group 3. No significant differences were observed between the 3 groups. There was no statistically significant difference in the PIV program distribution between the 3 groups.
Predictably, most physicians in the first group had completed a residency in family medicine (76%), whereas a minority in the second (36%) and third (16%) groups had done so. This difference was significant between groups 1 and 2, groups 2 and 3, and groups 1 and 3 (P = .001). More physicians (P < .05) in group 1 practised mainly in health care institutions (54%) than in groups 2 and 3 (27% and 25%, respectively).
Quality of practice
Table 2 presents the findings of the evaluation of quality of practice in the 3 CPD groups. During their visits, the inspectors judged that the record keeping was satisfactory for 60% of physicians in the first group, 53% in the second group, and 25% in the third group. The proportion of satisfactory scores linked to the quality (relevance) and quantity of CPD was higher in group 1 (83%) and group 2 (76%), and was clearly absent in group 3 (0%). For these 2 evaluation criteria, a statistically significant difference was noted between group 1 and group 3, and between group 2 and group 3 (P ≤ .001).
As for the physicians’ quality of clinical practice, 3 components were evaluated. First, the inspectors checked the quality of the clinical investigation plan (ie, history taking, physical examination, investigations). The investigation plan was considered satisfactory for 77% of physicians in group 1, 60% in group 2, and 40% in group 3. A significant difference was noted between the first and second groups (P = .03), between the first and third groups (P = .001), and between the second and third groups (P = .001). Next, inspectors evaluated the diagnoses that were made; a satisfactory score was given to 79% of the physicians in group 1, 70% in group 2, and 49% in group 3. The difference was statistically significant (P < .05) between groups 1 and 3, as well as between groups 2 and 3. Finally, the treatment plan appeared appropriate for 81% of physicians in group 1, 75% in group 2, and 50% in group 3. A significant difference appeared between the first and second groups (P = .04), between the first and third groups (P = .05), and between the second and third groups (P = .05).
As Table 3 shows, for the evaluation of quality of practice (record keeping, investigation plan, diagnosis, and treatment plan) there was no significant difference between the 3 groups with respect to sex and postgraduate training in family medicine. However, significant differences were observed in the various components of quality of clinical practice according to the physician’s age and principal place of practice.
As high-quality clinical practice generally integrates the 3 components of clinical practice, a composite score was obtained by adding 1 point for each of the 3 components of quality of clinical practice (investigation, diagnosis, and treatment plan) that was judged satisfactory. Physicians whose practice was judged unsatisfactory for all 3 criteria received a total of 0 points and those whose practice was satisfactory for all 3 criteria received the maximum allowed of 3 points. Table 4 presents the results of the composite score. Significant differences (P < .05) were observed between groups 1 and 3 and between groups 2 and 3.
Table 5 presents the results of a stepwise multiple linear regression analysis showing that little or no CPD activity (ie, being in group 3), private practice without institutional privileges, and age were the 3 most important factors negatively influencing the quality of clinical practice. Physicians who were selected for PIVs through the methadone program clearly demonstrated a better quality of clinical practice. The same regression model was rerun excluding them. The results were similar, showing being in group 3 (β = −0.21), private practice (β = −0.20), and older age (β = −0.18) as the only significant (P < .05) predictors of poor quality of practice.
DISCUSSION
Our findings show that group 1, composed of CFPC members, had the highest proportion of physicians judged to be satisfactory in terms of the quality and quantity of CPD activities. This group performed better on all the components of quality of practice. Group 2, composed of family physicians who were not members of the CFPC but who had at least 50 hours a year of CPD activities, scored similarly to group 1 for record keeping and diagnosis, and groups 1 and 2 had the same median composite score. Group 3, composed of physicians with little or no CPD activity, had lower scores for all quality-of-practice components.
Group 2 had a CPD activity profile similar to that of group 1, even though physicians in group 2 were a little older. Groups 2 and 3 differed on the basis of their CPD scores, but were similar in terms of PIV program distribution, country of graduation, principal place of practice, and whether or not they completed a residency in family medicine.
Overall, physicians in group 2 obtained better scores than those in group 3 with respect to quality of practice; they were younger and more likely to have completed a residency in family medicine.
Caution is essential in interpreting these findings, as it was difficult to balance the groups perfectly in terms of sex, age, and completion of a family medicine residency. Group 1 was made up of younger physicians, and most practised at least part-time in health care institutions with privileges. They were also more likely to have competed family medicine residency. With respect to the program that led to the PIV, the 3 groups were very similar, except for the methadone program; very few physicians in the methadone PIV program had little or no CPD activity. For physicians in this particular PIV program, the overall quality of practice proved more likely to be satisfactory, but this did not affect the group results.
It seems reasonable to argue that younger physicians working in health care institutions have greater opportunities to be in closer contact with peers and that their CPD activities are complemented by other, more informal, educational and networking activities.
Three characteristics were associated with lower quality of clinical practice: belonging to CPD group 3 (little or no CPD), older physician age, and practising only in private practice without institutional privileges. The most important factor was belonging to group 3, characterized by a small number of hours and poor relevance of CPD activities.
As other authors have underlined, many factors influence quality of practice.16–22 Continuing professional development is an important one. Physicians certified by the CFPC generally work more in hospitals, and they have access to better structured and more relevant continuing education activities. McAuley and colleagues found 3 factors related to poor physician performance: older age, not being a member of the CFPC, and solo practice.19 Choudhry et al discussed the relationship between the number of years of experience and quality of practice.20 Turnbull et al21 and Williams22 reported also that a substantial number of dyscompetent physicians had cognitive difficulties. In the present study, no formal cognitive evaluation of the assessed physicians was done; cognitive impairment might be a confounding factor.
Since July 2007, the CMQ has emphasized CPD activities that are based on a process of self-reflection about one’s practice. Time will tell whether this process has an effect on the quality of medical practice.
This study, which employed an independent and objective evaluation of practice using an evaluation tool with recognized validity and reliability, allowed us to sketch a preliminary portrait of the link between the quantity and quality of CPD activities and the quality of practice of Quebec’s family physicians.
Conclusion
This study enabled us to objectively reaffirm that the quality and quantity of CPD have a positive influence on the quality of medical practice. Furthermore, the physician’s age and degree of professional isolation, particularly with reference to office practice only and not having privileges in a health care institution, are determining factors in the quality of practice.
Acknowledgments
The Collège des médecins du Québec, the College of Family Physicians of Canada, and the Federation of Medical Regulatory Authorities of Canada contributed generously to the funding of this research project.
Notes
EDITOR’S KEY POINTS
-
The College of Family Physicians of Canada requires that its members earn and report a minimum of 250 hours of continuing professional development (CPD) activities in the Mainpro® program over each period of 5 years. This study aimed to assess whether the amount of, and quality or relevance of, CPD was associated with the quality of practice.
-
Because most physicians already practise appropriately and in an up-to-date manner, it can be difficult to measure the effect on practice of a single CPD activity—it is difficult to measure difference when the difference is very slight. This study examined CPD completed during a 5-year period and assessed physician performance using professional inspection visits. These visits are generally intended for physicians whose quality of practice might be in question for various reasons.
-
The group of College of Family Physicians of Canada members had the highest proportion of physicians judged to be satisfactory in terms of the quality and quantity of CPD activities, and this group performed better on all the components of quality of practice. The group of physicians with little or no CPD activity had lower scores for all quality-of-practice components.
POINTS DE REPÈRE DU RÉDACTEUR
-
Le Collège des médecins de famille du Canada exige de ses membres qu’ils effectuent et déclarent un minimum de 250 heures d’activités de formation médicale continue (FMC) du programme Mainpro au cours de chaque période de 5 ans. Cette étude voulait déterminer s’il existe une relation entre la quantité et la qualité ou la pertinence de la FMC et la qualité de la pratique.
-
Parce que la plupart des médecins pratiquent déjà de façon adéquate et conforme aux données récentes, il peut être difficile d’évaluer l’effet sur la pratique d’une seule activité de FMC - il est en effet difficile de mesurer une très petite différence. Cette étude a examiné la FMC effectuée par des médecins au cours d’une période de 5 ans et évalué leur performance au moyen de visites d’inspection professionnelle. Ces visites visent généralement des médecins dont la qualité de pratique pourrait être remise en question pour diverses raisons.
-
C’est dans le groupe des membres du Collège des médecins de famille du Canada qu’on a observé la plus forte proportion de médecins pour lesquels la qualité et la quantité des activités de FMC était jugée satisfaisantes et c’est aussi dans ce groupe qu’on observait la meilleure performance dans toutes les composantes de la qualité de pratique. Le groupe de médecins qui avaient peu ou pas d’activités de FMC avaient des scores plus bas dans toutes les composantes de la qualité de pratique.
Footnotes
-
This article has been peer reviewed.
-
Cet article a fait l’objet d’une révision par des pairs.
-
Contributors
All authors contributed to the concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.
-
Competing interests
Dr Lemire is Executive Director and Chief Executive Officer of the College of Family Physicians of Canada.
- Copyright© the College of Family Physicians of Canada