Abstract
Objective To evaluate the effectiveness of field notes in assessing teachers’ confidence and perceived competence, and the effect of field notes on residents’ perceptions of their development of competence.
Design A faculty and resident survey completed 5 years after field notes were introduced into the program.
Setting Five Dalhousie University family medicine sites—Fredericton, Moncton, and Saint John in New Brunswick, and Halifax and Sydney in Nova Scotia.
Participants First- and second-year family medicine residents (as of May 2009) and core family medicine faculty.
Main outcome measures Residents’ outcome measures included beliefs about the effects of field notes on performance, learning, reflection, clinical skills development, and feedback received. Faculty outcome measures included beliefs about the effect of field notes on guiding feedback, teaching, and reflection on clinical practice.
Results Forty of 88 residents (45.5%) participated. Fifteen of 50 faculty (30.0%) participated, which only permitted a discussion of trends for faculty. Residents believed field note–directed feedback reinforced their performance (81.1%), helped them learn (67.6%), helped them reflect on practice and learning (66.7%), and focused the feedback they received, making it more useful (62.2%) (P < .001 for all); 63.3% believed field note–directed feedback helped with clinical skills development (P < .01). Faculty believed field notes helped to provide more focused (86.7%) and effective feedback (78.6%), improved teaching (75.0%), and encouraged reflection on their own clinical practice (73.3%).
Conclusion Most surveyed residents believed field note use improved the feedback they received and helped them to develop competence through improved performance, learning, reflection, and clinical skills development. The trends from faculty information suggested faculty believed field notes were an effective teaching, feedback, and reflection tool.
The Department of Family Medicine at Dalhousie University in Halifax, NS, is developing a competency-based evaluation and training program in accordance with College of Family Physicians of Canada (CFPC) evaluation objectives. Training and evaluation of family medicine residents must be done in a comprehensive family medicine environment to assess competence and ensure that the assessment of competence is discipline specific.1,2
Field notes are brief documents printed on a prescription-sized note pad. They are used in a clinical setting to both provide and document specific, behaviour-based feedback from faculty to residents. The field note used by the Dalhousie University Department of Family Medicine at the time of the survey is available from CFPlus.* This note has been used as a documentation and teaching tool in the Dalhousie University Family Medicine Residency Training Program since 2005, when preceptors in small communities throughout the Maritimes and core teaching preceptors in larger centres identified difficulty in documenting feedback for residents. One barrier was the use of multiple forms, which were dependent on the encounter type, and issues with accessing these forms. The Department of Family Medicine at Dalhousie University began using a version of the University of Alberta field note for family medicine. Preceptors were pleased with the simplicity of a single form, and the portability issues appeared to be solved.
Literature is available on the importance of appropriate feedback and evaluation. Turnbull et al1 and Donoff3 have identified that much of the validity of in-training evaluation stems from the application of skills and knowledge in an actual clinical practice setting. Field notes can be used to identify and document behaviour, and to provide feedback to both residents facing challenges and residents with strong clinical skills within actual practice circumstances.3,4
Field notes provide brief, behaviour-specific feedback to residents. The intention is to take approximately 5 minutes to complete, discuss, and document this feedback.5–7 Sargeant et al have proposed that directed self-assessment might be used to facilitate change based on external feedback.8 Acceptance of feedback depends on multiple factors.9–14 The field note provides a framework in which to share feedback, stimulate reflection, encourage specificity, ensure credibility, reinforce appropriate behaviour, and facilitate necessary change. We are particularly interested in encouraging reflection, as it appears to be necessary for accepting constructive feedback.15 When used appropriately, the field note can be used to both teach and learn reflection.8,15
Several factors that interfere with the assessment of resident performance have been identified.9 Fortunately, the field note addresses these factors. Frequency of field note use and its immediacy address the issues of incomplete sampling, faculty memory constraints, and hidden performance deficits. The field note is a formative tool that permits faculty to examine residents’ performance at the behaviour level. This helps to mitigate issues involving the “halo” effect in the evaluation of residents—seeing strong or weak residents as globally strong or weak. Completion of field notes using the CFPC evaluation objectives as a guide establishes a common performance standard.16
Five years after the introduction of field notes into the program, a faculty and resident survey was completed to evaluate this tool in the assessment of teachers’ confidence and perceived competence. Further, the residents’ perceptions of the effect of field notes on their development of competence were evaluated.
METHODS
The Department of Family Medicine at Dalhousie University is a multisite family medicine teaching program spanning the Maritime provinces in Canada. The 5 sites surveyed in this study were Fredericton, Moncton, and Saint John in New Brunswick, and Halifax and Sydney in Nova Scotia.
All first- and second-year family medicine residents (as of May 2009) and core family medicine faculty from all 5 sites were surveyed. Research ethics approval was obtained from each site. Residents and faculty were provided with surveys and return postage–paid envelopes in May 2009.
The resident survey was pilot-tested by 2 senior residents and 1 recent graduate. None of these residents participated in the survey; the senior residents graduated before the ethics proposals were complete. The faculty survey was pilot-tested and modified after review by members of the CFPC Working Group on the Certification Process, which was an expert group of 7 family physicians from different backgrounds with many years of experience in both family medicine program management and clinical teaching. This group was enhanced by the addition of an evaluation consultant.16
The survey instrument was a questionnaire composed of 4 demographic questions, 5 closed-ended questions, and 14 open-ended questions. Several of the open-ended questions consisted of 2 parts: in the first part, the candidates answered on a Likert scale, and in the second part, they justified the answers they gave. For the statistical analyses, the answers to the open-ended questions were coded for ease of use.
Assuring the validity of an instrument depends on its development processes, as well as on the judgment of experts. For the field notes survey, validity was assured through the development process and pilot testing. Furthermore, in a second step, the members of the CFPC Working Group on the Certification Process confirmed that the field notes had content validity for the purposes proposed by the authors.
For the analysis of supervisors’ and residents’ comments, we developed a grid with a matrix in which both qualitative and quantitative data were compiled. The qualitative data were identified using codes to represent the key words reflecting positive, negative, and neutral answer themes. This permitted the calculation of their frequency. In a second step, these were divided into 2 groups (yes and no) for statistical analysis using a χ2 approach.
We estimated that 36 participants were required to discern a minimum difference of 5 percentage points on the frequencies of yes and no, with an α of .05 and β of .2, minimizing type II error.
RESULTS
Forty of 88 residents (45.5%) responded to the survey, which allowed statistical analysis of the results. In the faculty survey, 15 of 50 faculty members (30.0%) replied; this permitted only a discussion of trends, as power was not sufficient to assess the statistical significance of responses.
Resident survey results
Completed resident surveys represented all 5 teaching sites, with equal representation from first- and second-year residents. Table 1 shows residents’ reports of the rates of field note completion by preceptors.
Resident survey of the rates at which weekly field notes were provided by preceptors
Residents stated that field notes directed feedback: the field notes reinforced performance (81.1%), helped them learn (67.6%), helped them reflect on their practice and learning (66.7%), and focused the feedback they received, making it more useful (62.2%) (P < .001 for all). In addition, 63.3% of residents believed that field note– directed feedback helped with clinical skills development (P < .01). For the 6 skill dimensions, residents believed that field note–directed feedback promoted their learning (Table 2). Fifty-one percent stated that they used the CFPC evaluation objectives to implement suggestions from field note–directed feedback. Likert scales with more than 5 choices behave like interval scales. Thus, our survey with 10 choices (1 to 10) made these calculations possible.17
Resident perceptions of promotion of learning in skill dimensions
Faculty survey results
Among faculty participants, 86.7% believed that field notes helped them provide more focused feedback. This is reinforced by the fact that 73.3% of faculty focused their field note feedback on 1 or 2 skill dimensions when completing an individual field note. Fifty percent found procedural skills easiest to assess; 23.1% found communication and 30.8% found professionalism the most challenging to assess. Fifty percent of faculty addressed professionalism least frequently, 78.6% believed field notes helped them provide more effective feedback, and 75.0% believed the notes improved their teaching. Finally, 73.3% believed field notes were effective in helping them reflect on their clinical practice.
DISCUSSION
Field notes were implemented as a feedback tool in the Dalhousie University Family Medicine Residency Training Program in 2005 because of evidence in the literature that they were useful. In addition, the program needed to provide better-documented feedback to residents.
Our survey showed that most resident respondents believed field note–guided feedback helped them develop competence. The trend suggested by the information provided by faculty indicated faculty believed field notes were an effective teaching, feedback, and reflection tool.
Despite these facts, field notes are not used universally to achieve these ends. Barriers to their regular daily use must be studied further and addressed. Some possible explanations include the need for further faculty and resident development in the effective use of field notes, the perception that providing feedback is time consuming and thus not achievable in a busy clinical setting, and the variability of motivations for giving and receiving feedback among faculty and residents.
Professionalism and communication permeate all clinical encounters. Helping faculty assess these skill dimensions is essential to residents’ development of overall competence. Faculty surveyed identified difficulty in assessing and providing feedback for both communication skills and professionalism. This finding might be the result of the seemingly subjective nature of these skills; however, the CFPC has developed documents with lists of specific positive and negative observable behaviours in each of these domains.18,19 Faculty and residents in the Dalhousie family medicine program have been made aware of these documents. Other strategies to encourage the use of these documents might be beneficial, such as further faculty development on both field notes and CFPC evaluation objectives, as well as workshops on the effective use of field notes.
The literature suggests that organizing field notes might enhance their ability to document competence.3 Research is currently under way to identify the most effective way to achieve this organization, and to demonstrate how organizing field notes might allow their more effective use in summative and portfolio evaluations.
Limitations
As no survey of residents and faculty was completed before the introduction of field notes into the Dalhousie University family medicine residency program, our survey relied on residents’ comparison of their undergraduate feedback experiences with those experiences in residency with the use of field notes. In addition, the faculty survey relied on teachers’ recollection of their teaching abilities 5 years earlier, before field note use. Finally, the total number of faculty respondents was small, and did not allow the determination of statistical significance. This might introduce bias toward more motivated or experienced faculty. Despite these limitations, however, strong and recurrent trends did emerge.
Conclusion
This subjective survey of residents and faculty indicates that through the use of field notes, feedback was perceived as being provided and being focused. Residents believed field note–guided feedback helped them learn their role and improve their performance, and also helped them reflect on their learning and practice. Faculty believed that using field notes improved the quality of their teaching and allowed them to reflect on their own clinical practice. Of importance is the fact that ongoing faculty development and resident education sessions on the effective use of field notes have been helpful: half of residents were using CFPC evaluation objectives to implement suggestions from field note–directed feedback. The Dalhousie University Department of Family Medicine is currently using information from field notes to populate the standardized in-training evaluation reports. Since this survey, site directors have begun performing a twice-yearly resident file review, during which field notes are reviewed as an important component in the assessment of developing competence.
Acknowledgments
This work was completed under the auspices of the Department of Family Medicine at Dalhousie University, with support in part from the College of Family Physicians of Canada.
Notes
EDITOR’S KEY POINTS
-
The field note is a generic tool that can be used to provide specific, behaviour-based feedback to residents in a timely and reflective manner.
-
Competence must be assessed and documented in day-to-day clinical practice, and field notes are useful for facilitating this process.
-
Field note feedback can facilitate the completion of in-training evaluation reports with specific examples of residents’ strengths and challenges.
POINTS DE REPÈRE DU RÉDACTEUR
-
Le mémo clinique est un outil générique qui, à partir des comportements, peut servir à fournir aux résidents une rétroaction spécifique, et ce, de façon opportune et propice à la réflexion.
-
En clinique, la compétence doit être évaluée et documentée quotidiennement et les mémos sont utiles pour faciliter ce processus.
-
La rétroaction fournie par les mémos peut faciliter la rédaction des rapports d’évaluation en cours de formation en fournissant des exemples spécifiques des forces et des difficultés des résidents.
Footnotes
-
This article has been peer reviewed.
-
Cet article a fait l’objet d’une révision par des pairs.
-
↵* The Dalhousie University Department of Family Medicine field note is available at www.cfp.ca. Go to the full text of this article online, then click on CFPlus in the menu at the top right-hand side of the page.
-
Contributors
All authors were responsible for the conceptual development of the project, the design of the study, data collection, writing the article, and editing the final manuscript.
-
Competing interests
None declared
- Copyright© the College of Family Physicians of Canada
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.