Abstract
Problem addressed Many courses are offered to health care professionals to improve educational scholarship and scholarly teaching. The literature on the effect of such courses on promoting educational scholarship and scholarly teaching is currently suboptimal.
Objective of program To evaluate scholarly productivity of health care professional learners participating in 2 graduate courses in which curricula and assignments facilitated experiential learning.
Program description A retrospective analysis of course assignments and publications of learners from 2007 to 2014 was conducted. Learners’ current positions were identified through Google Scholar searches, and publication of course work was identified through PubMed or EMBASE author searches. There were 137 learners, with a male to female ratio of 3:7, consisting of physicians (73%) and other health care professionals (27%). During the 7 years, 50% completed both courses, 42% only the first course, and 8% only the second course. Of the learners whose current positions could be identified, 66% worked at academic centres, 20% at community hospitals or office practices, and 5% were in senior leadership positions. Current positions were unidentifiable through public records for 9% of learners. Sixty-eight percent of learners (93 of 137) published 1050 articles in peer-reviewed journals. Twenty-six percent of learners (35 of 137) published 1 or more articles based on their course assignments, for a total of 49 peer-reviewed articles; 80% of articles were published within 3 years of completing the course.
Conclusion Experiential learning facilitated by curricular design and assignments coupled with mentorship stimulated scholarly publications. Educational courses should design curricula to promote scholarship in learners and evaluate their effect.
Educational scholarship leads to knowledge generation in medical education and promotes the delivery of high-quality training through evidence-based practices. Scholarly productivity facilitates academic promotion and career satisfaction of clinician educators.1 Scholarly educational productivity of medical educators has generally been suboptimal, with physicians reporting limited involvement and a need for training in medical education research, as many clinician educators are interested.2–4 Barriers to scholarship include a lack of time, skills, and mentors, and unclear academic expectations.5 A recent article attributes suboptimal academic productivity to a lack of clarity in research questions, inadequate methodology to assess study questions, loss of momentum, lack of follow-through, and lack of expertise.6 To address this, several institutions have created faculty development programs to foster educational scholarship skills. A few studies examining the outcomes of such programs reveal promising results in terms of participant satisfaction and scholarly productivity.7–10
Many universities have created master’s degree programs as part of their faculty development efforts for health educators.11 The content of these master’s programs usually consists of curriculum development, instructional methods or teaching, learning, assessment strategies, program evaluation, research design, and sometimes leadership and management.12 There have been calls to establish criteria and mechanisms for the evaluation of these programs and to explore their effects beyond teaching effectiveness by examining interventions targeting other faculty roles such as educational scholarship.12,13 There is currently limited literature evaluating master’s programs in health professional education.
Program description
Continuing Education in Health Professions (CEHP) and Continuing Education Planning, Management and Evaluation (CEPME) are 2 graduate courses that are part of the Health Practitioner Teacher Education program leading to a master’s degree at the University of Toronto in Ontario. Both courses provide a half-credit individually and complement each other. They are delivered in a modular fashion over 5 days to provide 40 hours of face-to-face learning. The primary author (S.R.) assumed teaching responsibilities for the CEHP course in 2007 and developed the CEPME course in 2008, assuming responsibility for curriculum development, assessment, and grading. Both courses subscribe to the theoretical model of “experiential learning” and are grounded in a constructivist perspective, proposing that experience plays a critical role in knowledge construction and acquisition, where learning occurs when someone creates knowledge through experiential transformations.14,15 All learners were working health professionals. The learners were asked to choose topics that were relevant to their practices or an educational activity they were involved in to work on during the courses.
Before 2007, the CEHP course included several individual and group assignments that tested knowledge of key concepts of health professional education, and the final assignment was to write a scholarly essay on an important concept related to health professional education. The essay instructions did not specify a word count, but essays were generally between 3000 and 4000 words. The curriculum for the CEHP course was modified in 2007 to incorporate workshops on knowledge synthesis and systematic reviews. A librarian facilitated a literature search workshop and provided assistance with literature searches for learners who requested it. The main modification was the final assignment, which was updated to have learners write a systematic or scoping review. All group assignments were completed in class and individual assignments built on each other to create the final article. The final assignment had a weight of 65% of the final course grade; requirements were to write about 2000 to 2500 words (with an absolute maximum of 3000 words) and to include 20 to 25 references. Learners were allowed to choose any clinical or educational topic and received feedback on their work from the course instructor and other learners when they presented their abstracts in class. Learners were also provided examples of good writing in the form of published previous assignments.
The CEPME course provided a theoretical basis and practical strategies for conducting educational research. The learners were expected to submit an educational research proposal as the course assignment. All learners completed a needs assessment survey before the course began and were asked to identify an educational project they would like to complete as part of the course. Learners participated in workshops on both quantitative and qualitative research protocol developments in addition to a writing workshop during the course. In this course, all group assignments were also completed in class, and individual assignments built on each other to create the final article. The first individual assignment was a summary of a proposal for an educational project or program (500 to 600 words); learners then presented their protocols in class for evaluation and feedback and submitted their education research proposal as the final assignment. Learners were strongly advised to obtain ethics approval, apply for funding, and complete their research projects.
All learners were offered mentorship during and after course completion to publish their work. At the time of writing, the program did not formally collect information on learners applying their knowledge to create scholarly publications, and it was not known whether any graduates of the program had published anything.
The objective of the study was to evaluate scholarly productivity of health care professional learners participating in 2 graduate courses teaching continuing education in health professions at the University of Toronto. Scholarly publications were chosen to evaluate scholarly productivity. The study was approved by the Research Ethics Board of the Hospital for Sick Children in Toronto.
Program evaluation
A retrospective analysis of course assignments and publications of learners from 2 graduate courses (CEHP and CEPME) during 2007 to 2014 was conducted. Class lists and their course assignment topics were obtained after institutional ethics approval was received. All learners who participated in 1 or both courses during the study period of 7 years were identified and tabulated with the topic of their course assignments. Learners’ current positions were identified through Google Scholar searches and publication of course work was identified through PubMed or EMBASE author searches. Searches for published articles for each learner were conducted and tabulated by year. The following information was extracted: program of study, number of years between publication of course assignment and course completion, number of publications to date, occupation, gender, current position, and working environment (eg, academic centre, community hospital). Descriptive statistics were used as appropriate, and inferential statistics were used for identified variables.
Results
There were 137 learners, with a male to female ratio of 3:7 during the 7-year study period. Most were physicians (73%), and 73% of these were family physicians. Other health professionals represented 27% of learners. Most learners were from Canada (75%), followed by Saudi Arabia (16%) and other Middle East countries (7%). Approximately 50% of the students completed both graduate courses, 42% completed only the CEHP course, and 8% completed only the CEPME course. Public record searches of those who completed 1 or more courses identified 66% of learners working at academic centres, 20% at community hospitals or office practices, and 5% in senior leadership positions. Current positions were unidentifiable through public records in 9% of cases.
Sixty-eight percent of learners (93 of 137) published a total of 1050 articles in peer-reviewed journals during the study period. Topic and abstract review identified that 26% of learners (35 of 137) published 1 or more papers based on their course assignments, for a total of 49 peer-reviewed papers. Of these 35 learners, 66% (n = 23) published 1 article, 29% (n = 10) published 2 articles, and 5% (n = 2) published 3 articles. There were 6 other publications (4 conference abstracts and 2 non–peer-reviewed publications) for a total of 55 publications based on course assignments. Physicians had a higher number of publications of their assignments compared with other health professionals (29% vs 16%, respectively). There was no significant difference between learners working in academic centres and those working in community or office practices in publication rates of course assignments. Nine percent published their course assignments within a year of course completion, 34% were published within 2 years, and 80% were published within 3 years. Table 1 provides the complete list of health care professional learners who participated in master’s degree courses and the number of publications based on course assignments.
Learners’ occupations and publications based on course assignments
Discussion
Almost all doctoral programs in medical education are research based and result in some published work.16 Although publication rates have been used by faculty development courses that aim to improve educational scholarship to show effectiveness,17 most master’s programs that focus on theoretical frameworks to prepare learners for ongoing scholarship and research have not published their effects. This is the first report that evaluates the effects of master’s-level courses on medical education using publication of course assignments as a surrogate marker for course effectiveness in promoting educational scholarship.
Although course work might have contributed to publication, other learner-related factors such as self-selection of learners to the master’s degree program probably influenced academic productivity.
Results of a study evaluating effectiveness of a faculty development course in promoting educational scholarship showed the following contributed to the students’ success: “content (the ‘how’, not just the ‘what’); format (project-based, forced protected time); social network (group work promoting group synergy); and the role of the Course facilitator (functional mentor, coaching).”18 Scholarly publication by learners who completed these graduate courses might have been facilitated by similar factors, such as curriculum design to provide theoretical frameworks and practical strategies including experiential learning; assignments designed to promote and evaluate knowledge synthesis (ie, writing up a systematic or scoping review for CEHP and an educational research protocol for CEPME); and ongoing mentorship. A follow-up qualitative study might be needed to provide more understanding of how these courses foster productivity.
There have been calls for caution that minimal guidance or unguided instructional approaches have been less effective and less efficient than instructional approaches that emphasize guidance in teaching novice learners.19 This caution is probably also applicable for master’s courses where learners who might be experienced health care professionals with excellent teaching skills have no publication record and lack the skills to improve their scholarly productivity without guidance.
A Canadian study examining graduate writing assignments across 12 faculties in a university found that the commonest assignment format was the scholarly essay, accounting for 23% of assignments across all faculties.20 This study showed that writing a literature review as an assignment was most common in applied sciences (32%) and the least common in education (3%), with an overall rate of 14% across all faculties, whereas proposal writing was the most common in medicine (16%), with an overall rate of 9% across all faculties.20 It has been shown that graduate students value learning scientific writing skills and value courses that teach them.21 We argue that having a course where the goal is for the students to generate a review article that is submitted for publication in a peer-reviewed journal not only allows students to learn a new topic and build scientific reading and writing skills, but also enhances their resumes while concurrently contributing knowledge to the broader scientific community.21
The results of this study show that graduate courses in education have to not only focus on content but also pay attention to the design of assessments and evaluate their effects on scholarship. Further studies should evaluate other graduate courses and master’s programs in education to explore how and why educational scholarship is promoted or not promoted.
Limitations
This study is limited by its retrospective design and data extraction process confined to information available in the public domain. This study only captured publication data as a marker for educational scholarship and did not attempt to identify other scholarly, teaching, or leadership activities. As such, conference presentation, grants, curriculum development, program development, teaching activities, policy development, and leadership activities that were unpublished were not captured. Future studies can prospectively collect such information to provide a holistic view of the effect of graduate courses and programs on medical education.
Conclusion
Curricular design and assessment strategies for experiential learning coupled with mentorship might stimulate scholarly publication from course assignments. Medical educators should pay attention to curricula and assignments of educational courses to encourage learners to increase scholarly productivity. Evaluating the effect on learners’ educational scholarship is an important aspect of program evaluation and will help the program focus on facilitating scholarship in education.
Acknowledgments
We acknowledge Elizabeth Uleryk for her support in performing the literature searches. This work was funded by an Art of Possible Grant from the Department of Family and Community Medicine at the University of Toronto.
Notes
Editor’s key points
▸ This study found that 68% of learners (93 of 137) published a total of 1050 articles in peer-reviewed journals during the study period. Topic and abstract review identified that 26% of learners (35 of 137) published 1 or more papers based on their course assignments, for a total of 49 peer-reviewed papers. Physicians had a higher number of publications of their assignments compared with other health professionals.
▸ Scholarly publication by learners who completed the graduate courses might have been facilitated by factors such as curriculum design to provide theoretical frameworks and practical strategies including experiential learning; assignments designed to promote and evaluate knowledge synthesis; and ongoing mentorship.
▸ Master’s degree programs in health professional education should ensure curricula provide instructional guidance for health professional learners to increase their scholarly productivity.
Points de repère du rédacteur
▸ Dans cette étude, on a constaté que 68 % des apprenants (93 sur 137) ont publié au total 1050 articles dans des revues révisées par des pairs durant la période à l’étude. Une revue des sujets et des résumés a permis de déterminer que 26 % des apprenants (35 sur 137) ont publié 1 article ou plus basés sur leurs travaux de cours, pour un total de 49 articles révisés par des pairs. Les médecins comptaient un plus grand nombre de publications de leurs travaux par rapport à d’autres professionnels de la santé.
▸ Les publications scientifiques des apprenants qui ont suivi les cours d’enseignement supérieur pourraient avoir été facilitées par des facteurs tels qu’un cursus conçu pour offrir des structures théoriques et des stratégies pratiques, y compris l’apprentissage par l’expérience; des travaux de cours visant à promouvoir et à évaluer la synthèse des connaissances; et le mentorat continu.
▸ Les programmes de maîtrise de haut niveau en éducation des professionnels de la santé devraient faire en sorte que le cursus offre des conseils pédagogiques afin que les professionnels de la santé apprenants augmentent leur productivité en matière de publications scientifiques.
Footnotes
Contributors
Both authors contributed to the concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.
Competing interests
None declared
This article has been peer reviewed.
Cet article a fait l’objet d’une révision par des pairs.
- Copyright© the College of Family Physicians of Canada
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.