Defining competency-based evaluation objectives in family medicine

Background to the process

In 1998, the College of Family Physicians of Canada’s (CFPC) Board of Examiners decided to identify what constituted clinical competence for the purposes of Certification in Family Medicine. The process of developing the definition of competence has been described elsewhere, and began with a survey of practising family physicians to elicit a description of competence in family medicine.⁶ From this initial survey 5 essential skill dimensions, 8 phases of the clinical encounter, and 99 priority topics were identified, constituting the domain of competence. The 5 skill dimensions were a patient-centred approach, communication skills, clinical reasoning skills, selectivity, and professionalism.⁶ Procedure skills were not initially identified among the skill dimensions. The CFPC Board of Examiners, however, recognized that Certification in Family Medicine did imply competence in certain procedures. Therefore, competence in procedure skills was added to the definition of competence in family medicine, as a sixth essential skill dimension.

With respect to competence in procedure skills, a list of 65 core procedures, germane to family physicians practising in any setting, and an enhanced list of 15 procedures was then developed by Wetmore et al.⁷ The core procedures list was included as part of the domain of competence, and was considered as equivalent to the priority topics previously mentioned.

Thus, the domain of competence consisted of 6 skill dimensions, 8 phases of the clinical encounter, 99 priority topics, and a core list of procedure skills. This description was, however, thought to still not be sufficiently detailed for the purposes of assessment. More detailed evaluation objectives for each of the 99 priority topics were developed using a key-feature approach, as described by Lawrence et al.⁸ Similarly, because the core procedures list, by itself, was not sufficient to define competence in performing procedures, the general key features for procedure skills were needed to enhance this definition. Competence in procedure skills could then be evaluated using the core procedures list and the general key features for procedure skills.

Key-feature approach

The key-feature approach is a practical method of defining competence for the purposes of assessment, first described by Bordage and Page.⁹ Page and Bordage described a key feature as a critical point in the resolution of a problem, where examinees are most likely to make errors and which is a difficult aspect of the identification and management of the problem in practice.¹⁰ The overall objective of the key feature approach is 2-fold. The first aim is to identify these essential or critical steps specific to the problem; the second is to determine why they are difficult and what processes are involved in successfully completing them. Page and Bordage identified that key features for a given problem are not typically generic; they vary according to the clinical presentation of the problem relative to other issues, such as age and sex.¹⁰ A general skill might be used in any given key feature; however, an individual key feature is problem specific. Generally, key features are observable actions; they are not simply knowledge. They are generated from practical experience, not theoretical analysis or published references. Key features are pragmatic, suggesting where assessment should be concentrated in order to be both effective and efficient. They are useful tools when planning assessment.

Overall competency frameworks

This paper also describes how the general key features for procedure skills are related to the overall competency frameworks that exist for Canadian physicians: the 4 principles of family medicine, from the CFPC,¹¹ and the CanMEDS roles, developed by the Royal College of Physicians and Surgeons of Canada.¹² Although useful, such frameworks do not describe competence in sufficient detail to usefully direct assessment.¹³ Others suggest that the CanMEDS roles are so intertwined that assessing each of them separately makes little sense.¹⁴ It might be reassuring, however, to know that any new method of assessment of competence in procedure skills can be linked back to these 2 overall competency frameworks.