Abstract
Objective To describe the postgraduate medical education (PGME) examination outcomes and work locations of international medical graduates (IMGs); and to identify differences between Canadians studying abroad (CSAs) and non-CSAs.
Design Cohort study using data from the National IMG Database and Scott’s Medical Database.
Setting Canada.
Participants All IMGs who had first entered a family medicine residency program between 2005 and 2009, with the exclusion of US graduates, visa trainees, and fellowship trainees.
Main outcome measures We examined 4 outcomes: passing the Medical Council of Canada Qualifying Examination Part 2 (MCCQE2), obtaining Certification in Family Medicine (CCFP), working in Canada within 2 years of completing PGME training, and working in Canada in 2015.
Results Of the 876 residents in the study, 96.1% passed the MCCQE2, 78.1% obtained a specialty designation, 37.7% worked in Canada within 2 years after their PGME, and 91.2% worked in Canada in 2015. Older graduates were more likely (odds ratio [OR] = 3.45; 95% CI 1.52 to 7.69) than recent graduates were to pass the MCCQE2, and residents who participated in a skills assessment program before their PGME training were more likely (OR = 9.60; 95% CI 1.29 to 71.63) than those who had not were to pass the MCCQE2. Women were more likely (OR = 1.67; 95% CI 1.20 to 2.33) to obtain a specialty designation than men were. Recent graduates were more likely (OR = 1.36; 95% CI 1.03 to 1.79) than older graduates were to work in Canada following training. Residents who were eligible for a full licence were more likely (OR = 3.72; 95% CI 2.30 to 5.99) to work in Canada in 2015 than those who were not eligible for a full licence were.
Conclusion While most IMGs who entered the family medicine PGME program passed the MCCQE2, 1 in 5 did not obtain Certification. Most IMG residents remain in Canada. Canadians studying abroad and non-CSA IMGs share similar examination success rates and retention rates.
Studies suggest that international medical graduates (IMGs) in family medicine residency programs do not perform as well as Canadian medical graduates (CMGs) do on the Certification Examination in Family Medicine. A study of University of British Columbia family medicine residents found that 58% of IMG residents passed the Certification Examination in Family Medicine compared with 95% of CMGs.1 Similarly, 66.6% of IMG but 90.4% of CMG family medicine residents in Quebec passed the 2007 Certification examination.2 In both studies, IMGs had been carefully screened and had completed skills assessment programs before being admitted into the family medicine residency program. Canada-wide data indicate that pass rates on the Certification Examination in Family Medicine have held steady for CMGs (at > 90%) while the pass rate for IMGs has declined from 66% in 2007 to 51% in 2010.3 Moreover, reports from Canada suggest that a substantial number of IMGs withdraw from their residency programs or require remedial training to complete their programs.2,4
Residents who pass the examination (and are in good standing) are awarded the Certification in Family Medicine (CCFP) designation, which, along with passing the Medical Council of Canada Qualifying Examination Part 2 (MCCQE2), is required for full licensure in Canada. However, residents who do not pass the examination can still work in Canada under a provisional licence (also known as a restricted or temporary licence), usually in rural areas.5,6 While studies have shown that a sizable number of IMG family medicine residents do not pass the CCFP examination, it is unclear how lack of a CCFP designation affects retention.
There is considerable competition among IMGs for the limited number of residency positions, including in family medicine, where most IMGs are admitted. Among IMG applicants are a growing number of Canadians who study abroad (CSAs; citizens and permanent residents before entering medical school). Recent studies estimate that nearly 700 CSAs graduate annually from medical schools abroad and many return to Canada hoping to complete residency training and work.7 It is not known whether CSA trainees fare better on the MCCQE2 and CCFP examinations, or whether CSAs are more likely to remain in Canada than their non-CSA counterparts are.
The purpose of this study is to describe the PGME examination outcomes and work locations of IMG family medicine residents in Canada. For educational outcomes, we examined passing the MCCQE2 and obtaining a specialty designation. For retention, we examined whether IMGs were working in Canada within 2 years of completing PGME training and in 2015. The study provides information on the relationship between qualifying for full licensure and work location of IMG family medicine trainees. It is important to understand how these trainees contribute to the overall supply of physicians in Canada if, as suggested in the literature, a substantial number of IMG residents do not pass all the examinations needed for full licensure.
METHODS
The Newfoundland and Labrador Health Research Ethics Board approved this study. We linked data from the National IMG Database with data from Scott’s Medical Database. The National IMG Database captures longitudinal data to track IMGs as they qualify for licensure and join the physician work force8 and is one of the databases held by the Canadian Post-MD Educational Register (CAPER), an agency that gathers and holds individual-level data on all postgraduate medical trainees in Canada.9 The database includes data from various agencies involved in the training, assessment, certification, and licensing of IMGs. Scott’s Medical Database is a list of physicians in Canada and is the most comprehensive data source available to track physician locations.10,11 As part of its ongoing monitoring of work locations of PGME residents,12 CAPER had data from Scott’s Medical Database for the entire period from 2005 to 2015, with the exception of 2010 and 2014. Scott’s Medical Database does not keep historical data, so it was not possible to retroactively find location data for these 2 years.
We examined 4 outcomes: passing the MCCQE2 (yes or no), obtaining the CCFP designation (yes or no), working in Canada after training (yes or no), and working in Canada in 2015 (yes or no). Examination data were reported to the National IMG Database by the Medical Council of Canada, the College of Family Physicians of Canada, and the Royal College of Physicians and Surgeons of Canada (RCPSC). The National IMG Database records the year in which an IMG passes (or is exempt from) the MCCQE2 and the year in which he or she was awarded a specialty designation. It does not include whether an IMG wrote or failed the examination. In all our analyses, we assumed that all IMGs would attempt to obtain full licensure. To determine whether IMGs worked in Canada immediately after their training, we identified whether IMGs were listed in Scott’s Medical Database in the 2 years after their PGME training. The National IMG Database includes the last year of PGME enrolment, which we used to identify the 2 years of interest. We also identified whether the IMG was listed in the 2015 Scott’s Medical Database.
Our independent variable was type of IMG. We defined CSAs as IMGs who were born in Canada or who were Canadian citizens or permanent residents before entering medical school. Non-CSAs were IMGs who were neither born in Canada nor Canadian citizens or permanent residents before medical school. Preliminary analyses suggested that IMGs who graduated from medical school in Western or Caribbean countries might have different outcomes than would IMGs who did not graduate from medical schools in these countries.13 We therefore created an independent variable that captured both legal status and training site, and examined 4 groups of IMGs:
Group 1 included IMGs who were Canadian citizens or permanent residents before medical school and who graduated from medical schools in Western or Caribbean countries (CSA-Western).
Group 2 included IMGs who were Canadian citizens or permanent residents before medical school and who did not graduate from medical schools in Western or Caribbean countries (CSA–non-Western).
Group 3 included IMGs who were not Canadian citizens or permanent residents before medical school and who graduated from medical schools in Western or Caribbean countries (non-CSA–Western).
Group 4 included IMGs who were not Canadian citizens or permanent residents before medical school and who did not graduate from medical schools in Western or Caribbean countries (non-CSA–non-Western).
Covariates considered in the analysis were sex (male or female), age at the start of PGME (< 30 years or ≥ 30 years), years between obtaining a medical degree and PGME (younger graduate: ≤ 5 years between graduating from medical school and entering PGME; older graduate: ≥ 6 years between graduating from medical school and entering PGME), and participation in a skills assessment or training program (yes or no).8,12 For the retention outcomes, we also considered the variable eligible for full licence (ie, passed both the MCCQE2 and CCFP examination; yes or no).
To be included in the analysis, IMGs had to have first entered a family medicine residency program between 2005 and 2009. The cutoff allows sufficient time to qualify to write the examinations and ensure that the passed examination would be recorded in the database. International medical graduates who passed the MCCQE2 before 2005 (before the start of the database) would not have the event recorded. We excluded US graduates because a number of agencies in Canada (eg, Canadian Resident Matching Service, Collège des Médecins du Québec) classify graduates of accredited US medical schools as CMGs.2,14 We excluded visa trainees because they are expected to return to practise in their home countries.15,16 We also excluded fellowship trainees because there were too few in family medicine for a meaningful analysis.
Using SPSS, version 23.0, we described the characteristics of the sample and used χ2 tests between each outcome and relevant predictor variables. We used multiple logistic regression to identify significant (P < .05) predictors for each outcome. We selected other potential predictor variables for each regression model on the basis of the χ2 tests. Variables were examined for possible collinearity a priori. If variables were correlated (eg, age at start of PGME and years between medical degree and PGME) we included only 1 of the variables in the model. Predictors were removed from the model if they were not significant (based on the Wald test) and if they did not significantly improve the change in the −2 log likelihood value.17 The tables list the variables included in the final regression models. For each outcome, we hypothesized that Canadians, regardless of where they graduated from medical school, would outperform non-Canadians (that is, would be more likely to pass the MCCQE2, more likely to obtain a specialty designation, and more likely to work in Canada following residency and in 2015).
RESULTS
There were 876 IMGs who first entered a family medicine residency program between 2005 and 2009. The largest proportion of IMG trainees (43.5%) were from the non-CSA–non-Western group. Most were female (58.3%), were 30 years or older when they started their residency (78.4%), were older graduates with 6 or more years between graduating from medical school and entering the residency program (57.1%), and had not participated in a skills assessment or training program (71.5%). More than three-quarters (76.7%) of the cohort had the credentials (passed MCCQE2 and obtained CCFP or other specialty designation) to be eligible for a full licence (Table 1).
Almost all (96.1%) IMGs in the cohort passed the MCCQE2. Compared with IMGs who did not pass the MCCQE2, a larger proportion of IMGs who passed the examination were from the non-CSA–non-Western group, were female, were 30 years or older when they entered the residency program, were older graduates (≥ 6 years between graduating medical school and entering the PGME program), and had participated in a skills assessment or training program (Table 2). After controlling for other significant predictors, older graduates were 3.45 (95% CI 1.52 to 7.69) times more likely than recent graduates were to pass the MCCQE2. International medical graduates who had participated in a skills assessment or training program were 9.60 (95% CI 1.29 to 71.63) times more likely to pass the MCCQE2 than were IMGs who had not participated in a program (Table 3).
More than three-quarters (78.1%) of the cohort obtained a specialty designation. Compared with IMGs who did not obtain a designation, a larger proportion of IMGs who obtained a specialty designation were female (Table 2). Female IMGs were 1.67 (95% CI 1.20 to 2.33) times more likely to obtain specialty credentials than men were (Table 3).
Roughly one-third (37.7%) of IMG family medicine residents were working in Canada in the 2 years after their training. Compared with those who were not working in Canada, a larger proportion of recent graduates remained in Canada in the 2 years after their PGME training. Specifically, recent graduates were 1.36 (95% CI 1.03 to 1.79) times more likely to work in Canada in the 2 years after their PGME training than older graduates were.
Most (91.2%) of the cohort was working in Canada in 2015. Compared with those who did not work in Canada in 2015, a larger proportion of IMGs who were eligible for a full licence worked in Canada than IMGs who were not eligible for a full licence (Table 4). International medical graduates who were eligible for a full licence were 3.72 (95% CI 2.30 to 5.99) times more likely to work in Canada in 2015 than those who were not eligible were (Table 5).
DISCUSSION
While most IMGs who entered the family medicine residency program passed the MCCQE2, almost 1 in 4 did not obtain the credentials needed for a full licence. The National IMG Database records only positive events, such as passing an examination. The data do not allow us to distinguish IMGs who did not write examinations from those who wrote but did not pass them. In addition, the National IMG Database only captures data from 2005 to 2011 and some IMGs might have passed these examinations after 2011. A recent CAPER study has shown that most family medicine residents take longer than 2 years (2.16 years in 2006 for IMGs) to complete their programs.18 When we limited the cohort to IMGs who exited a family medicine PGME residency by 2010 (n = 753), the proportion of trainees who passed the MCCQE2, obtained a specialty designation, and were eligible for a full licence increased to 97.2%, 82.7%, and 81.9%, respectively.
While all the IMGs in this study were admitted to a family medicine residency program, 44 (5.0%) wrote RCPSC specialty examinations to obtain specialist credentials. It is uncommon for postgraduate medical residents (CMGs and IMGs) to change PGME programs.12 Between 2011 and 2014, less than 1% of all family medicine residents (CMGs and IMGs) in Canada changed programs.12 When we exclude IMGs who were known to change programs or who wrote RCPSC examinations (n = 832), 96.2% passed the MCCQE2, 79.7% obtained the family medicine credentials, and 78.2% were eligible for a full licence.
The proportion of IMG family medicine residents who worked in Canada in 2015 (91.2%) is substantially greater than the proportion of IMG family medicine residents who worked in Canada in the 2 years following their training (37.7%). We found a similar pattern among all specialists and visa trainees (that is, the retention rate was higher in 2015 than in the 2 years after PGME training).13 We suspect that many new-to-practice physicians take short-term or locum positions when they first start practising, making it difficult for the Scott’s Medical Directory to locate a valid address for them. By comparison, 10.7% and 4.6% of CMGs are not found to be practising in Canada 2 and 5 years, respectively, after exiting PGME programs.12
Although obtaining credentials for a full licence was a predictor of working in Canada, roughly one-fifth (20.8%) of IMG family medicine residents who remained in Canada in 2015 were not eligible for a full licence and would likely have been working under a provisional (or restricted) licence. These findings highlight the need for further research into why a substantial proportion of IMGs (CSAs and non-CSAs) do not realize entry-to-practice milestones needed for full licensure.
Contrary to our hypothesis, there was no difference in the educational outcomes and retention of CSA and non-CSA IMGs. These findings suggest that IMGs, regardless of connections to Canada before their medical education or location of medical school, have similar examination performance and retention patterns. Given that they have more clinical experience, it is not surprising that older graduates are more likely than their younger counterparts are to pass the MCCQE2 (an examination written after the first year of residency training); there is no difference between recent and older graduates’ performance on certification (eg, CCFP) examinations. Moreover, while participation in a skills assessment or training program was positively associated with passing the MCCQE2, it was not significantly related to obtaining the CCFP credential. These findings highlight the need to identify criteria other than legal status, medical school, or participation in skills assessment and training programs for admission to PGME training.
Limitations
As noted above, we might have underestimated the proportion of IMGs who passed credentialing examinations because of the limited period covered by the National IMG Database. Regulatory authorities (provincial colleges of physicians and surgeons) submitted data to the National IMG Database that should have allowed us to identify where trainees were working (on full and provisional licences). However, preliminary analyses revealed that the data were unusable.13 We therefore used a proxy variable (whether a resident passed both the MCCQE2 and Certification Examination in Family Medicine) to identify IMGs who qualified for a full licence. We identified 684 IMGs who received a specialty designation (CCFP or other), but only 672 were eligible for a full licence. Our records suggest that 12 of the 684 did not pass the MCCQE2. We used Scott’s Medical Database to identify work locations; some physicians who work in Canada might not be included in Scott’s Medical Database, and as a result, we might have underestimated retention rates. The sample is limited to the 876 residents who could reasonably complete a family medicine residency during the years (2005 to 2011) captured by the National IMG Database; we risk a type 2 error given the multiple comparisons we conducted.
Conclusion
Roughly 3 in 4 (76.7%) family medicine residents obtain the credentials needed for a full licence. Most (91.2%) were working in Canada in 2015. There was no difference in the educational outcomes or retention of CSA and non-CSA IMGs. Given that 1 in 4 family medicine residents do not obtain the credentials needed for a full licence, and that eligibility for a full licence was a predictor of working in Canada, the study findings highlight the need to understand and improve IMG performance in family medicine residency programs.
Acknowledgments
This study was funded by the Canadian Institutes of Health Research with in-kind contributions from the Canadian Post-MD Education Registry. We gratefully acknowledge the contributions of Lynda Buske and Les Forward to the project. The opinions expressed in the article are of the authors alone and not of the Canadian Post-MD Education Registry, Health Canada, or the Royal College of Physicians and Surgeons of Canada.
Notes
EDITOR’S KEY POINTS
Roughly 3 in 4 (76.7%) international medical graduate (IMG) family medicine residents obtain the credentials needed for a full licence. Roughly one-fifth (20.8%) of IMG family medicine residents who remained in Canada in 2015 were not eligible for a full licence. These findings highlight the need for further research into why a substantial proportion of IMGs (Canadians studying abroad [CSAs] and non-CSAs) do not realize entry-to-practice milestones needed for full licensure.
There was no difference in the educational outcomes or retention of CSA and non-CSA IMGs. These findings suggest that IMGs, regardless of connections to Canada before their medical education or location of medical school, have similar examination performance and retention patterns.
Given that 1 in 4 IMG family medicine residents do not obtain the credentials needed for a full licence, and that eligibility for a full licence was a predictor of working in Canada, the study findings highlight the need to understand and improve IMG performance in family medicine residency programs.
POINTS DE REPÈRE DU RÉDACTEUR
Environ les trois quarts (76,7 %) des diplômés internationaux en médecine (DIM) qui sont résidents en médecine familiale obtiennent les crédits nécessaires pour un permis de pratique complet. Parmi ces DIM résidents en médecine familiale restant au Canada en 2015, environ un cinquième (20,8 %) n’avait pas les résultats pour obtenir un permis sans restriction. Ces résultats montrent qu’il faudra d’autres études pour connaître les raisons pour lesquelles un pourcentage important des DIM (des Canadiens étudiant à l’étranger [CEE] et des non-CEE) n’obtient pas les résultats nécessaires pour obtenir un permis de pratique complet.
Il n’y avait pas de différence entre les résultats scolaires ou la rétention des CEE et des non-CEE. Ces résultats donnent à croire que les DIM ont des résultats aux examens et des taux de rétention similaires, quels aient été leurs liens avec le Canada avant leurs études médicales ou le lieu de la faculté de médecine.
Étant donné que le quart des DIM inscrits au programme de résidence en médecine familiale n’obtient pas les notes suffisantes pour obtenir un droit de pratique complet, et que l’admissibilité à un tel droit est un prédicteur d’une pratique au Canada, les résultats de l’étude démontrent la nécessité de comprendre et d’améliorer le rendement des DIM au sein des programmes de résidence en médecine familiale.
Footnotes
This article has been peer reviewed.
Cet article a fait l’objet d’une révision par des pairs.
Contributors
All authors contributed to the concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.
Competing interests
None declared
- Copyright© the College of Family Physicians of Canada