Skip to main content

Main menu

  • Home
  • Articles
    • Current
    • Published Ahead of Print
    • Archive
    • Supplemental Issues
    • Collections - French
    • Collections - English
  • Info for
    • Authors & Reviewers
    • Submit a Manuscript
    • Advertisers
    • Careers & Locums
    • Subscribers
    • Permissions
  • About CFP
    • About CFP
    • About the CFPC
    • Editorial Advisory Board
    • Terms of Use
    • Contact Us
  • Feedback
    • Feedback
    • Rapid Responses
    • Most Read
    • Most Cited
    • Email Alerts
  • Blogs
    • Latest Blogs
    • Blog Guidelines
    • Directives pour les blogues
  • Mainpro+ Credits
    • About Mainpro+
    • Member Login
    • Instructions
  • Other Publications
    • http://www.cfpc.ca/Canadianfamilyphysician/
    • https://www.cfpc.ca/Login/
    • Careers and Locums

User menu

  • My alerts

Search

  • Advanced search
The College of Family Physicians of Canada
  • Other Publications
    • http://www.cfpc.ca/Canadianfamilyphysician/
    • https://www.cfpc.ca/Login/
    • Careers and Locums
  • My alerts
The College of Family Physicians of Canada

Advanced Search

  • Home
  • Articles
    • Current
    • Published Ahead of Print
    • Archive
    • Supplemental Issues
    • Collections - French
    • Collections - English
  • Info for
    • Authors & Reviewers
    • Submit a Manuscript
    • Advertisers
    • Careers & Locums
    • Subscribers
    • Permissions
  • About CFP
    • About CFP
    • About the CFPC
    • Editorial Advisory Board
    • Terms of Use
    • Contact Us
  • Feedback
    • Feedback
    • Rapid Responses
    • Most Read
    • Most Cited
    • Email Alerts
  • Blogs
    • Latest Blogs
    • Blog Guidelines
    • Directives pour les blogues
  • Mainpro+ Credits
    • About Mainpro+
    • Member Login
    • Instructions
  • RSS feeds
  • Follow cfp Template on Twitter
Research ArticleResearch

Defining competency-based evaluation objectives in family medicine

Procedure skills

Stephen Wetmore, Tom Laughlin, Kathrine Lawrence, Michel Donoff, Tim Allen, Carlos Brailovsky, Tom Crichton and Cheri Bethune
Canadian Family Physician July 2012, 58 (7) 775-780;
Stephen Wetmore
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: stephen.wetmore@schulich.uwo.ca
Tom Laughlin
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kathrine Lawrence
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michel Donoff
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tim Allen
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carlos Brailovsky
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tom Crichton
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Cheri Bethune
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF
Loading

Abstract

Objective To develop evaluation objectives for assessing competence in procedure skills using a key-features approach. This was part of a multiyear project to develop competency-based evaluation objectives for Certification in Family Medicine.

Design Nominal group technique.

Setting The College of Family Physicians of Canada in Mississauga, Ont.

Participants An expert group of 7 family physicians and 1 educational consultant, all of whom had experience in assessing competence in family medicine. Group members represented the Canadian context with respect to region, sex, language, community type, and experience.

Methods Using a nominal group technique, the expert group developed the general key features for procedure skills. The expert group also linked the key features to already established skill dimensions in the domain of competence, to the 4 principles of family medicine, and to the CanMEDS roles.

Main findings The general key features were developed after 5 iterations. Ten key features were outlined and were shown to reflect all the essential skill dimensions in the domain of competence for family medicine. The key features were linked to 2 of the 4 principles of family medicine and to 4 of the CanMEDS roles.

Conclusion The general key features for procedure skills were developed to assess competence in procedure skills in family medicine.

For many years, “see one, do one, teach one” has been the accepted philosophy for teaching and learning procedural skills in medicine. This concept primarily referred to teaching and learning the technical steps necessary for successful procedure performance. Learning the technical steps of a given procedure in family medicine is fairly easily accomplished during training by repetitive practice.1,2 True competence in procedure skills, however, involves much more than just learning the technical aspects of any procedure.

With the current focus on patient safety and medical error, and the move to competency-based education, the “see one, do one, teach one” principle has been recognized as inadequate for training for most procedures in medicine.3,4 Furthermore, there has been considerable variability in the procedures taught during training in both Canadian and American family medicine training programs.1,5

Given this variation in procedures taught and the challenges to fundamental procedures teaching concepts, there was a need for uniform evaluation objectives that would more accurately assess competence in procedure skills that could be used by all teaching programs of family medicine in Canada.

In this paper we have focused on procedure skill teaching using a key-features approach and we describe the development of the general key features for procedure skills. We also point to their utility for evaluating competence in procedure skills.

Background to the process

In 1998, the College of Family Physicians of Canada’s (CFPC) Board of Examiners decided to identify what constituted clinical competence for the purposes of Certification in Family Medicine. The process of developing the definition of competence has been described elsewhere, and began with a survey of practising family physicians to elicit a description of competence in family medicine.6 From this initial survey 5 essential skill dimensions, 8 phases of the clinical encounter, and 99 priority topics were identified, constituting the domain of competence. The 5 skill dimensions were a patient-centred approach, communication skills, clinical reasoning skills, selectivity, and professionalism.6 Procedure skills were not initially identified among the skill dimensions. The CFPC Board of Examiners, however, recognized that Certification in Family Medicine did imply competence in certain procedures. Therefore, competence in procedure skills was added to the definition of competence in family medicine, as a sixth essential skill dimension.

With respect to competence in procedure skills, a list of 65 core procedures, germane to family physicians practising in any setting, and an enhanced list of 15 procedures was then developed by Wetmore et al.7 The core procedures list was included as part of the domain of competence, and was considered as equivalent to the priority topics previously mentioned.

Thus, the domain of competence consisted of 6 skill dimensions, 8 phases of the clinical encounter, 99 priority topics, and a core list of procedure skills. This description was, however, thought to still not be sufficiently detailed for the purposes of assessment. More detailed evaluation objectives for each of the 99 priority topics were developed using a key-feature approach, as described by Lawrence et al.8 Similarly, because the core procedures list, by itself, was not sufficient to define competence in performing procedures, the general key features for procedure skills were needed to enhance this definition. Competence in procedure skills could then be evaluated using the core procedures list and the general key features for procedure skills.

Key-feature approach

The key-feature approach is a practical method of defining competence for the purposes of assessment, first described by Bordage and Page.9 Page and Bordage described a key feature as a critical point in the resolution of a problem, where examinees are most likely to make errors and which is a difficult aspect of the identification and management of the problem in practice.10 The overall objective of the key feature approach is 2-fold. The first aim is to identify these essential or critical steps specific to the problem; the second is to determine why they are difficult and what processes are involved in successfully completing them. Page and Bordage identified that key features for a given problem are not typically generic; they vary according to the clinical presentation of the problem relative to other issues, such as age and sex.10 A general skill might be used in any given key feature; however, an individual key feature is problem specific. Generally, key features are observable actions; they are not simply knowledge. They are generated from practical experience, not theoretical analysis or published references. Key features are pragmatic, suggesting where assessment should be concentrated in order to be both effective and efficient. They are useful tools when planning assessment.

Overall competency frameworks

This paper also describes how the general key features for procedure skills are related to the overall competency frameworks that exist for Canadian physicians: the 4 principles of family medicine, from the CFPC,11 and the CanMEDS roles, developed by the Royal College of Physicians and Surgeons of Canada.12 Although useful, such frameworks do not describe competence in sufficient detail to usefully direct assessment.13 Others suggest that the CanMEDS roles are so intertwined that assessing each of them separately makes little sense.14 It might be reassuring, however, to know that any new method of assessment of competence in procedure skills can be linked back to these 2 overall competency frameworks.

METHODS

An expert group of 7 family physicians and 1 evaluation consultant used a nominal group technique15 to derive the general key features for procedure skills, using multiple iterations to achieve consensus. Box 18 describes the nominal group technique used by the expert group. All members of the expert group had experience in assessing competence in family medicine, and represented the Canadian context for region and language. Both sexes were represented. The group collectively provided the full scope of family practice including community, inpatient, intrapartum, and emergency care, and all of the family physicians had experience performing common procedures in family medicine. They met several times a year, and membership remained unchanged during the 4 years in which the work was conducted.

As the final step of key-feature generation, each was coded or linked by the expert group back to the skill dimensions most essential for successful resolution of the problem. In addition, the key features were each coded to the 2 existing overall frameworks of competency—the 4 principles of family medicine and the CanMEDS roles. A consistent iterative method was used, alternating individual work with group discussion until consensus was achieved. A maximum of 2 skills, principles, or roles was permitted per key feature, as part of the linking exercise.

FINDINGS

Ten general key features for procedure skills were generated and are presented in Table 1. Each key feature stands on its own, but some are grouped under a stem statement reflecting a phase in the preparation for or performance of a procedure. Table 1 also documents how the key features are linked to all of the other skill dimensions and to 2 of the 4 principles of family medicine and 4 of the CanMEDS roles.

View this table:
  • View inline
  • View popup
Table 1.

The general key features for procedure skills

DISCUSSION

Clearly, performing procedures in family medicine is much more than just being able to complete the technical components of a procedure. The general key features for procedure skills described in this paper are crucial to competent performance of procedures in family medicine. It is important to note that these do not detail all there is to know about any one procedure but do reflect an important overall approach to procedure skills competence. For example, anatomy might vary from procedure to procedure, but not knowing the anatomy and failing to review it ahead of time could get one into trouble, no matter what the procedure. Similarly, when the procedure is going wrong, the not-yet competent physician might push on stubbornly to complete the procedure with the resulting complications. The competent physician would stop, reevaluate the situation, and seek assistance in order to correct the situation or halt the procedure. Thus, the general key features for procedure skills point to a different level of competence than just mastery of the technical steps of any procedure, and incorporate the skills, attitudes, and knowledge necessary for overall competence in procedure skills.

Box 1.

Key-features generation process for a single topic

  • Step 1: One group member is assigned a topic. This author independently identifies and lists all the potential key features for the assigned topic. In addition, all other group members identify the 1 or 2 key features they think are most critical for this same topic. This preparation occurs individually, and the results are compiled by topic before a meeting (first iteration).

  • Step 2: The assigned member leads a discussion among all group members. This discussion is centred on his or her list of key features, as well as on other members’ additions. The purpose of this step is to ensure that the tasks or competencies presented are truly the critical ones and that none was overlooked. This also requires clarification of the clinical cognitive processes involved for each key feature (eg, gathering data selectively is not the same as interpreting given data). The responsible member then integrates the discussion from step 2 into his or her list of key features (second iteration).

  • Step 3: The revised list of key features is then discussed again, led by the member responsible for the particular topic. At this time, the list of key features is revised to ensure that the starting clinical point, appropriate task, and rationalization are articulated for each key feature, and that the wording is sufficiently precise to guide assessment. This process is repeated until a consensus is achieved (third iteration).

  • Step 4: Group members code the key features independently, focusing on the skill dimensions most essential for successful resolution of the problem and on the phase of the clinical encounter in which the problem occurs. Almost all problems require many of the skills in a number of phases. Coding is limited, however, to a maximum of 2 skill dimensions and 2 phases for each key feature, namely those most critical for the competent resolution of the problem. The individual coding results are tabulated, and the entire group then meets to discuss discrepant codes and reach consensus for each of the key features. Occasional final corrections occur in the wording at this iteration (fourth iteration).

    In this study the key features were coded concurrently to the 4 principles of family medicine and to the 7 CanMEDS roles.

  • Adapted from Lawrence et al.8

  • Assessment of competence using the general key features for procedure skills is also measuring competence in all 5 of the other essential skill dimensions and reflects the 4 principles of family medicine and the CanMEDS roles. As such, evaluation objectives based on a key-feature approach are framework-independent, and reflect what truly needs to be assessed, in day-to-day practice, to determine competence.

    Overall competence in procedure skills could legitimately be inferred based on evaluation using the general key features and observation over a sufficient number of the core procedures. This approach should also enable a physician to learn new skills and procedures throughout his or her career.

    The strength of this work is that the key features were derived by an expert group of practising family physicians. As such they constitute a practical tool, grounded in clinical practice. Their validity is supported by the coding that showed that all the other skill dimensions of competence were captured in the key features. We also believe that the general key features for procedure skills are generic to performing procedures in any medical discipline and that they could be adapted by any discipline, after suitable testing or review.

    The next steps in this work would include looking at the individual procedures on the core procedures list and developing technical key features for each procedure. These specific key features for each procedure, combined with the general key features for procedure skills, could form the basis for tools to enhance the evaluation of procedure skills competence during training and as part of any examination process. The general key features for procedure skills should also be evaluated for their utility at assessing competence in procedure skills at different levels of training. Appropriate measures should be developed to determine if the key features were being successfully incorporated into practice.

    Limitations

    The nominal group process used in this research was an excellent process for building consensus but lacked the additional step of external validation that would have added strength to the validity of the general key features. The specific content of these general key features for procedure skills has not been validated. Their validity is suggested by both their method of development and by the results of the linking exercise to the 4 principles of family medicine and to the CanMEDS roles. The general key features for procedure skills might be criticized for lacking the specific technical components for any individual procedure, but this was not the intent. Procedure-specific key features will need to be developed.

    Conclusion

    The general key features for procedure skills are crucial to the competent performance of any procedure in family medicine and, together with the list of core procedures, could form the basis for defining competence in procedure skills at the start of independent practice in family medicine. Further research will be necessary to validate the content of the general key features for procedure skills and their incorporation into training and practice.

    Notes

    EDITOR’S KEY POINTS

    • The general key features for procedure skills were developed to assist in evaluating competence in procedure skills in family medicine.

    • The key features reflect all of the skill dimensions in the domain of competence.

    • The general key features for procedure skills along with the core procedures list can form the basis for evaluating competence in procedure skills.

    POINTS DE REPÈRE DU RÉDACTEUR

    • Les caractéristiques clés générales correspondant aux habiletés techniques ont été développées afin de faciliter l’évaluation de la compétence relative à ce type d’habiletés en médecine familiale.

    • Ces caractéristiques clés reflètent tous les aspects des habiletés relativement à la compétence.

    • Les caractéristiques clés générales correspondant aux habiletés techniques peuvent, avec la liste des techniques principales, servir de base à une évaluation de la compétence en matière d’habiletés techniques.

    Footnotes

    • This article has been peer reviewed

    • Cet article a fait l’objet d’une révision par des pairs

    • Contributors

      All authors were responsible for the conceptual development of the project, the design of the study, data collection, writing the draft, and editing the final manuscript. Dr Wetmore had the additional responsibility of writing the final manuscript.

    • Competing interests

      None declared

    • Copyright© the College of Family Physicians of Canada

    References

    1. ↵
      1. Norris TE,
      2. Cullison SW,
      3. Fihn SD
      . Teaching procedural skills. J Gen Int Med 1997;12(Suppl 2):S64-70.
      OpenUrlCrossRefPubMed
    2. ↵
      1. Holman JR,
      2. Marshall RC,
      3. Jordan B,
      4. Vogelman L
      . Technical competency in flexible sigmoidoscopy. J Am Board Fam Pract 2001;14(6):424-9.
      OpenUrlAbstract
    3. ↵
      1. Mason WT,
      2. Strike PW
      . See one, do one, teach one—is this how it still works? A comparison of the medical and nursing professions in the teaching of practical procedures. Med Teach 2003;25(6):664-6.
      OpenUrlCrossRefPubMed
    4. ↵
      1. Rodriguez-Paz JM,
      2. Kennedy M,
      3. Salas E,
      4. Wu AW,
      5. Sexton JB,
      6. Hunt EA,
      7. et al
      . Beyond “see one, do one, teach one”: toward a different training paradigm. Qual Saf Health Care 2009;18(1):63-8.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Van der Goes T,
      2. Grzybowski SC,
      3. Thommasen H
      . Procedural skills training. Canadian family practice residency programs. Can Fam Physician 1999;45:78-85.
      OpenUrlPubMed
    6. ↵
      1. Allen T,
      2. Brailovsky C,
      3. Rainsberry P,
      4. Lawrence K,
      5. Crichton T,
      6. Carpentier MP,
      7. et al
      . Defining competency-based evaluation objectives in family medicine. Dimensions of competence and priority topics for assessment. Can Fam Physician. Vol. 57. 2011. p. e331-40. Available from: www.cfp.ca/content/57/9/e331.full.pdf+html. Accessed 2012 May 23.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Wetmore SJ,
      2. Rivet C,
      3. Tepper J,
      4. Tatemichi S,
      5. Donoff M,
      6. Rainsberry P
      . Defining core procedure skills for Canadian family medicine training. Can Fam Physician. Vol. 51. 2005. p. 1364-5. Available from: www.cfp.ca/content/51/10/1364.long. Accessed 2012 May 18.
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Lawrence K,
      2. Allen T,
      3. Brailovsky C,
      4. Crichton T,
      5. Bethune C,
      6. Donoff M,
      7. et al
      . Defining competency-based evaluation objectives in family medicine. Key-feature approach. Can Fam Physician. Vol. 57. 2011. p. e373-80. Available from: www.cfp.ca/content/57/10/e373.full.pdf+html. Accessed 2012 May 23.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Bordage G,
      2. Page G
      . An alternative approach to PMPs: the “key features” concept. In: Hart IR, Harden R, editors. Further developments in assessing clinical competence. The Second Ottawa Conference on Assessing Clinical Competence. Ottawa, ON: Can-Heal Publications; 1987.
    10. ↵
      1. Page G,
      2. Bordage G
      . The Medical Council of Canada’s key features project: a more valid written examination of clinical decision-making skills. Acad Med 1995;70(2):104-10.
      OpenUrlPubMed
    11. ↵
      1. College of Family Physicians of Canada
      . Four principles of family medicine. In: Section of Teachers of Family Medicine Committee on Curriculum. The postgraduate family medicine curriculum: an integrated approach. Mississauga, ON: College of Family Physicians of Canada; 2003. Available from: www.cfpc.ca/uploadedFiles/Education/Postgraduate_Curriculum.pdf. Accessed 2012 May 18.
    12. ↵
      1. Frank JR
      , editor. CanMEDS: better standards, better physicians, better care. Ottawa, ON: Royal College of Physicians and Surgeons of Canada; 2005. Available from: http://rcpsc.medical.org/canmeds/index.php. Accessed 2012 May 18.
    13. ↵
      1. Albanese MA,
      2. Mejicano G,
      3. Mullan P,
      4. Kokotailo P,
      5. Gruppen L
      . Defining characteristics of educational competencies. Med Educ 2008;42(3):248-55.
      OpenUrlCrossRefPubMed
    14. ↵
      1. Jones J,
      2. Hunter D
      . Qualitative research: consensus methods for medical and health services research. BMJ 1995;311(7001):376-80.
      OpenUrlFREE Full Text
    15. ↵
      1. Ten Cate O
      . Trust, competence, and the supervisor’s role in postgraduate training. BMJ 2006;333(7571):748-51.
      OpenUrlFREE Full Text
    PreviousNext
    Back to top

    In this issue

    Canadian Family Physician: 58 (7)
    Canadian Family Physician
    Vol. 58, Issue 7
    1 Jul 2012
    • Table of Contents
    • About the Cover
    • Index by author
    Print
    Download PDF
    Article Alerts
    Sign In to Email Alerts with your Email Address
    Email Article

    Thank you for your interest in spreading the word on The College of Family Physicians of Canada.

    NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

    Enter multiple addresses on separate lines or separate them with commas.
    Defining competency-based evaluation objectives in family medicine
    (Your Name) has sent you a message from The College of Family Physicians of Canada
    (Your Name) thought you would like to see the The College of Family Physicians of Canada web site.
    CAPTCHA
    This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
    Citation Tools
    Defining competency-based evaluation objectives in family medicine
    Stephen Wetmore, Tom Laughlin, Kathrine Lawrence, Michel Donoff, Tim Allen, Carlos Brailovsky, Tom Crichton, Cheri Bethune
    Canadian Family Physician Jul 2012, 58 (7) 775-780;

    Citation Manager Formats

    • BibTeX
    • Bookends
    • EasyBib
    • EndNote (tagged)
    • EndNote 8 (xml)
    • Medlars
    • Mendeley
    • Papers
    • RefWorks Tagged
    • Ref Manager
    • RIS
    • Zotero
    Respond to this article
    Share
    Defining competency-based evaluation objectives in family medicine
    Stephen Wetmore, Tom Laughlin, Kathrine Lawrence, Michel Donoff, Tim Allen, Carlos Brailovsky, Tom Crichton, Cheri Bethune
    Canadian Family Physician Jul 2012, 58 (7) 775-780;
    Reddit logo Twitter logo Facebook logo Mendeley logo
    • Tweet Widget
    • Facebook Like
    • Google Plus One

    Jump to section

    • Article
      • Abstract
      • METHODS
      • FINDINGS
      • DISCUSSION
      • Limitations
      • Conclusion
      • Notes
      • Footnotes
      • References
    • Figures & Data
    • Info & Metrics
    • eLetters
    • PDF

    Related Articles

    • No related articles found.
    • PubMed
    • Google Scholar

    Cited By...

    • Des medecins de famille adaptatifs: Lattention accordee aux habiletes techniques est-elle une solution?
    • Adaptive family physicians: Is attention to procedural skills a key?
    • Uncharted territory: Knowledge translation of competency-based continuing professional development in family medicine
    • Territoire inexplore: Le transfert des connaissances sur le developpement professionnel continu axe sur les competences en medecine familiale
    • Google Scholar

    More in this TOC Section

    • Systematic assessment of opioid advertisements in general medical journals
    • Timely access to primary care in New Brunswick
    • Older persons living with dementia and their use of acute care services over 2 years in Alberta
    Show more Research

    Similar Articles

    Subjects

    • Collection française
      • Résumés de recherche

    Navigate

    • Home
    • Current Issue
    • Archive
    • Collections - English
    • Collections - Française

    For Authors

    • Authors and Reviewers
    • Submit a Manuscript
    • Permissions
    • Terms of Use

    General Information

    • About CFP
    • About the CFPC
    • Advertisers
    • Careers & Locums
    • Editorial Advisory Board
    • Subscribers

    Journal Services

    • Email Alerts
    • Twitter
    • RSS Feeds

    Copyright © 2023 by The College of Family Physicians of Canada

    Powered by HighWire