Abstract
Problem being addressed Medical schools aim to integrate the values of generalism into their undergraduate programs. However, currently no program has been described to measure the degree to which formal curricular materials represent generalist principles.
Objective of program To quantify the generalism principles present in undergraduate medical education learning materials and to provide recommendations to enhance generalism content.
Program description A review of the literature and accreditation documents was conducted to identify key elements of medical generalism. An evidence-informed tool, the Toronto Generalism Assessment Tool, was developed and applied to the new preclerkship undergraduate cases at the University of Toronto in Ontario. The findings regarding the presence of generalism principles and recommendations to enhance generalism content were provided to case developers. The recommendations were valued and were incorporated into subsequent iterations of the cases.
Conclusion This is the first report of a successful evidence-informed program to assess the degree of generalism reflected in undergraduate medical education curricular documents. This program can be used by other institutions wishing to review their curricula through a generalist lens.
Medical generalism is a philosophy of care that looks at the whole person in the context of his or her life, with a commitment to the person, as opposed to the disease itself, and to providing continuous and coordinated care in collaboration with the larger health care team.1-8 A focus on generalism has been advocated for by multiple international4-8 and national groups3,9-15 at the individual, population, and systems levels as a means to address complexity in health care. Whether or not practitioners are seen as generalists depends on their training, their attitudes, their scope of practice, and their work setting.4 Family doctors are often viewed as the quintessential generalists because they routinely see people across all types of problems, at all stages of the life cycle, and at any stage of problem from health promotion to the management of severe illness.1,2,4,8
During the past 10 years, the increasing focus on generalism in practice has led to a greater emphasis on teaching medical generalism in both undergraduate and postgraduate medical education curricula.1-15 The Future of Medical Education in Canada project recognized generalism as foundational for all physicians and recommended that medical education must focus on broadly based generalist content, including comprehensive family medicine.12 Specific recommendations included identifying and addressing elements of the hidden curriculum that devalue generalism and family medicine, and increasing representation of generalists within faculties and among preceptors.12 Moreover, the Future of Medical Education in Canada project recommended the provision of learning opportunities for students to experience undifferentiated patients and early presentations of illness in natural contexts.12 The report by the Canadian Generalism and Generalist Task Force highlighted 3 key elements of the role and responsibilities of generalists: generalists as managers of “the whole patient”; generalists as coordinators of care; and generalists as advocates for patients.3 They highlighted implications for medical education that included recommending the integration of the values of generalism into medical education for every physician and emphasizing the important role of curriculum and structure.3
The University of Toronto Faculty of Medicine launched a new preclerkship curriculum in 2016-2017. In preparation for the launch, the Undergraduate Education Committee of the Department of Family and Community Medicine was asked to review the curriculum plan and instructional materials from a generalist perspective. It became apparent through this process that there was a need to clarify definitions and unite perspectives on generalism among faculty members and other stakeholders. However, a search of the literature revealed there were no formalized methods described to assess the degree to which learning materials (cases, lecture slides, assessment items, etc) represented generalist principles.
Program objective
We aimed to develop a program that would quantify the generalism principles present in undergraduate medical education learning materials and to provide recommendations to enhance generalism content.
Program description
Construction of an evidence-based generalism framework. A literature review was conducted using a combination of the terms generalism/generalist, curriculum, medical education, and evaluation/assessment. Our search using PubMed, MEDLINE, ERIC, EMBASE, Scopus, and Google Scholar databases did not identify any existing programs or frameworks to assess the degree of medical generalism reflected in medical education learning materials. A search of the gray literature identified professional college and accreditation documents relevant to generalism in medical education.1-15 These included documents endorsed by the College of Family Physicians of Canada,10,11 the Association of Faculties of Medicine of Canada,10,12-14 the Royal College of Physicians and Surgeons of Canada,3,9,10 the Collège des médecins du Québec,10 the University of Toronto Generalism Task Force,15 the Royal College of General Practitioners,4-7 and the Australian Primary Health Care Research Institute.8 A review of these documents similarly did not identify any currently existing programs or frameworks. Members of the program team identified recurring conceptual elements and used them to develop a generalism framework to quantify the generalist principles present in medical education curricular materials (Figure 1).
Generalism framework: Key elements of medical generalism.
Application of the framework to draft cases. The framework was applied to all 61 of the draft cases that had been developed by faculty members in various disciplines for the new preclerkship curriculum at the University of Toronto. Two family medicine faculty raters independently reviewed each case. After reading the case, faculty raters were asked to provide 1 of 4 responses for each of the generalism elements: yes, indicating the generalism domain is clearly present; no, indicating the generalism domain is clearly absent; uncertain, indicating the case is not developed enough to be able to choose yes or no; or not applicable. Results were averaged across raters. This revealed an overall low inclusion of generalist principles, ranging from 10% to 30% (Figure 2). Detailed feedback and suggestions for change were provided to course and curriculum developers to enhance generalist content. Examples of common feedback provided to case writers are provided in Table 1. The case reviewer communicated directly with the case writer. In all but 1 of the 61 cases reviewed, the case writers were receptive to incorporating the suggestions of the case reviewers. The course director intervened in the review of the remaining case to ensure case reviewer feedback was incorporated.
Generalism content in draft cases of the new curriculum
Examples of feedback provided to case writers
Expert feedback and consensus. The generalism framework, and the data collected in the application phase of the review of the draft cases, were presented to multidisciplinary clinician educators with expertise and interest in medical generalism at national and international conferences. Feedback was collated and used to revise and refine the framework and the process for making recommendations for curricular modifications, which culminated in the development of the Toronto Generalism Assessment Tool (T-GAT) (Figure 3).
Toronto Generalism Assessment Tool
Psychometric characteristics of the T-GAT. With an ethics exemption from the Research Ethics Board of the University of Toronto, the feasibility and interrater reliability of the T-GAT were determined using a sample of new faculty raters. Twenty-one cases were randomly selected from the University of Toronto’s undergraduate medical education curriculum and analyzed in a nested balanced incomplete block design. Fourteen family medicine faculty members volunteered to assess these cases using the T-GAT. Each physician reviewed a total of 3 cases, and each case was reviewed by 2 raters. It took approximately 20 to 30 minutes for a single case to be assessed by 1 rater. Our data collection and psychometric analyses followed the approaches outlined by Streiner et al16 and Fleiss.17 Accordingly, data were analyzed using generalizability theory from variance components calculated in SPSS and G String software. The interrater reliability of the T-GAT was 0.64 (ie, with 2 raters the average score across 10 items had an interrater reliability of 0.64), indicating moderate reliability. Overall reliability of the total score was 0.48 and internal consistency was 0.92.
Discussion
Many regulatory bodies have emphasized the importance of incorporating generalism principles into medical education.1-15 However, no program previously existed to assess the presence of generalist principles in medical education documents. This is the first report of a program designed to use a generalist lens to provide institutional guidance in the implementation of a new undergraduate curriculum.
Our goal was to develop an approach that would allow undergraduate medical curricular stakeholders a feasible and actionable process by which to assess the inclusion of generalist principles in their programs. The program that we developed included an evidence-informed tool, the T-GAT, that allowed for systematic review of curricular documents and the development of recommendations to enhance generalism content. Notably, family medicine faculty members were eager to engage in the review process and case developers were open to the feedback and recommendations generated through the application of the T-GAT. Course directors were very responsive to the recommendations generated and curricular changes were implemented. We believe that this approach also allowed us to uncover important elements of the hidden curriculum and to frame the identification of these issues in a manner that allowed curriculum designers to improve their cases. Thus, the program was both feasible and effective.
The implementation of our program was successful because the medical school’s leadership was truly invested in assessing the curriculum for generalist principles. Their willingness to support and act on the recommendations has enhanced the level of generalism in the curriculum and demonstrates a respect for generalist values.
Limitations
There were several limitations to the implementation and assessment of this program. We assessed case-based materials only. We did not assess lectures, laboratories, or clinical experiences. Thus, these curricular components might or might not have included generalist elements and it was not possible to provide feedback about this. A number of limitations were identified with regard to testing the psychometric properties of the T-GAT, including that, by random chance, some pairs of raters reviewed identical cases; some raters showed little to no variance with respect to their responses; and a nested balanced incomplete block design to assess reliability is somewhat inefficient. This design implicitly results in a large degree of variance surrounding the calculated reliability coefficients. As such, even though we believe that case-level factors account for a large proportion of variation in the ratings, we cannot rule out that the reliability might be lower than estimated. A solution to this issue would be to conduct a nested balanced complete block design, where all raters review all cases. However, this was not feasible in our program owing to the time commitment this would require from the faculty volunteer reviewers.
Moving forward, we plan for ongoing systematic application of our generalism framework and the T-GAT to the curriculum to assess its ongoing effect on the curricular learning materials. We also plan to evaluate the effect of this approach on faculty and student perspectives on medical generalism. There is national interest in this program and we look forward to reviewing undergraduate educational documents with our colleagues across Canada to inform ongoing curricular development through a generalist lens.
Conclusion
This is the first report of the development and use of an evidence-informed tool as part of a program to assess the degree of generalism reflected in medical education curricular documents. This approach helped to provide important feedback, which was welcomed by program leaders and, even more importantly, incorporated into case-based materials. The success of this program depends on the willingness of stakeholders, including leadership and faculty members, to embrace the inclusion of generalist principles in undergraduate medical education. We expect this program can be used by other institutions wishing to review their curricula through a generalist lens.
Acknowledgment
We gratefully acknowledge Dr Geoff Norman, Dr Rahim Moineddin, and Mr Christopher Meaney for their statistical support. The program described was an educational scholarship project conducted in the Department of Family and Community Medicine (DFCM) at the University of Toronto. It received both financial and material support from the DFCM Office of Education Scholarship.
Notes
Editor’s key points
▸ The goal of this program was to develop an approach that would allow undergraduate medical curricular stakeholders a feasible and actionable process by which to assess their programs for the inclusion of generalist principles. The program developed at the University of Toronto in Ontario included an evidence-informed tool, the Toronto Generalism Assessment Tool, that allowed educators to systematically review curricular documents and develop recommendations to enhance generalism content.
▸ Family medicine faculty members were eager to engage in the review process and case developers were open to the feedback and recommendations generated through the application of the Toronto Generalism Assessment Tool. Course directors were very responsive to the recommendations generated and curricular changes were implemented.
▸ The implementation of this program was successful because the medical school’s leadership was truly invested in assessing the curriculum for generalist principles. The leaders’ willingness to support and act on the recommendations has enhanced the level of generalism in the curriculum and demonstrates a respect for generalist values.
Points de repère du rédacteur
▸ Ce programme avait pour but délaborer une approche permettant aux personnes concernées par le cursus médical prédoctoral de disposer d’un processus pratique et réalisable au moyen duquel évaluer leurs programmes en fonction de l’inclusion des principes du généralisme. Le programme élaboré à l’Université de Toronto (Ontario) comportait un outil éclairé par des données probantes, le Toronto Generalism Assessment Tool, qui a permis aux enseignants de passer systématiquement en revue les documents pédagogiques et d’élaborer des recommandations pour améliorer le contenu relatif au généralisme.
▸ Les membres du corps professoral en médecine familiale étaient enthousiastes à l’idée de participer au processus de révision, et les responsables de l’élaboration des cas ont été ouverts à la rétroaction et aux recommandations générées par la mise en application du Toronto Generalism Assessment Tool. Les directeurs de cours ont été très réceptifs aux recommandations présentées, et des changements au cursus ont été mis en œuvre.
▸ La mise en Suvre de ce programme a connu cette réussite parce que la direction de la faculté de médecine s’est véritablement investie dans l’évaluation du cursus en fonction des principes du généralisme. La volonté du leadership d’appuyer les recommandations et d’agir en conséquence a amélioré le degré de généralisme contenu dans le cursus et a fait preuve de respect pour les valeurs généralistes.
Footnotes
Contributors
All authors contributed to the concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.
Competing interests
None declared
This article has been peer reviewed.
Cet article a fait l’objet d’une révision par des pairs.
- Copyright© the College of Family Physicians of Canada
In this issue
