Selection for family medicine residency training in Canada: How consistently are the same students ranked by different programs?

Keith Wycliffe-Jones; Kent G. Hecker; Shirley Schipper; Maureen Topps; Jeanine Robinson; Tasnima Abedin

Article Figures & Data

Tables

View popup

Table 1.

Family medicine residency program site subsets created from analysis of 109 sites available to 17 Canadian medical school residency programs in 2013

SITE SUBSETS	HOW SITES WERE CHOSEN FOR GROUP
Western province rural FM group	Sites in the same western province in Canada with focus on training for future rural FM practice
Western Canada rural FM group	Sites in 4 western provinces in Canada with focus on training for future rural FM practice
Central and eastern Canada rural FM group	Sites in 4 central and eastern provinces in Canada with focus on training for future rural FM practice
Large urban-based FM group	Sites based in locations where immediate population is > 100 000
3-schools group	Sites in 3 medical schools in the same province in Canada with the same stated selection criteria

FM—family medicine.

View popup

Table 2.

Consistency of ranking of students across all FM residency training sites in each CaRMS cycle from 2007 to 2013, by year

YEAR	NO. OF SITES	NO. OF STUDENTS WHO WERE RANKED BY MORE THAN 1 SITE (TOTAL NO. OF STUDENTS RANKED)	ICC
2013	109	1518 (1753)	0.37
2012	102	1452 (1702)	0.34
2011	94	1283 (1483)	0.35
2010	84	1206 (1519)	0.35
2009	73	1167 (1383)	0.35
2008	65	1062 (1257)	0.39
2007	59	969 (1129)	0.39

CaRMS—Canadian Resident Matching Service, FM—family medicine, ICC—intraclass correlation coefficient.

View popup

Table 3.

Subset analysis of CaRMS data for consistency of ranking of students applying to more than 1 FM training site in each CaRMS cycle from 2007 to 2013, by site group

SUBSET	YEAR (NO. OF SITES)	NO. OF STUDENTS WHO WERE RANKED BY MORE THAN 1 SITE (TOTAL NO. OF STUDENTS RANKED)	ICC^*
Western province rural FM group	2013 (5)	144 (195)	0.55
	2012 (5)	93 (146)	0.39
	2011 (4)	110 (157)	0.50
	2010 (4)	77 (131)	0.51
	2009 (4)	90 (143)	0.42
	2008 (4)	64 (124)	0.27
	2007 (4)	49 (102)	0.19
Western Canada rural FM group	2013 (19)	321 (434)	0.40
	2012 (16)	249 (391)	0.17
	2011 (12)	214 (387)	0.28
	2010 (11)	175 (367)	0.38
	2009 (10)	179 (316)	0.20
	2008 (9)	141 (311)	0.25
	2007 (8)	136 (213)	0.24
Central and eastern Canada rural FM group	2013 (34)	345 (699)	0.38
	2012 (32)	342 (645)	0.39
	2011 (33)	303 (629)	0.30
	2010 (26)	246 (582)	0.25
	2009 (21)	205 (489)	0.16
	2008 (17)	199 (417)	0.44
	2007 (17)	213 (405)	0.51
Large urban-based FM group	2013 (27)	1313 (1665)	0.26
	2012 (27)	1201 (1625)	0.23
	2011 (26)	1028 (1383)	0.25
	2010 (25)	992 (1401)	0.27
	2009 (25)	987 (1327)	0.28
	2008 (24)	881 (1171)	0.31
	2007 (22)	840 (1074)	0.36
3-schools group	2013 (25)	371 (534)	0.17
	2012 (24)	354 (522)	0.07
	2011 (22)	320 (463)	0.21
	2010 (22)	303 (460)	0.25
	2009 (19)	290 (428)	0.22
	2008 (17)	287 (396)	0.33
	2007 (16)	285 (351)	0.44

CaRMS—Canadian Resident Matching Service, FM—family medicine, ICC—intraclass correlation coefficient.
↵* The ICC values in each of the subsets over the years demonstrate generally weak to fair levels of consistency in ranking by schools of the same students applying to more than 1 FM residency training site. The exception to this is the western province rural FM group, which demonstrated improved consistency in ranking over recent years to a moderate level of consistency (in 2013 ICC was 0.55). For the other subsets, with one exception in 2007, the level of consistency did not rise above fair over all years studied (ICC = 0.16–0.44), and for the 3-schools group, the level of consistency decreased from fair in 2007 (ICC = 0.44) to very weak in 2013 (ICC = 0.17).