An objective structured clinical exam to measure intrinsic CanMEDS roles

BACKGROUND: The CanMEDS roles provide a comprehensive framework to organize competency-based curricula; however, there is a challenge in finding feasible, valid, and reliable assessment methods to measure intrinsic roles such as Communicator and Collaborator. The objective structured clinical exam (OSCE) is more commonly used in postgraduate medical education for the assessment of clinical skills beyond medical expertise. METHOD: We developed the CanMEDS In-Training Exam (CITE), a six-station OSCE designed to assess two different CanMEDS roles (one primary and one secondary) and general communication skills at each station. Correlation coefficients were computed for CanMEDS roles within and between stations, and for general communication, global rating, and total scores. One-way analysis of variance (ANOVA) was used to investigate differences between year of residency, sex, and the type of residency program. RESULTS: In total, 63 residents participated in the CITE; 40 residents (63%) were from internal medicine programs, whereas the remaining 23 (37%) were pursuing other specialties. There was satisfactory internal consistency for all stations, and the total scores of the stations were strongly correlated with the global scores r=0.86, p<0.05. Noninternal medicine residents scored higher in terms of the Professional competency overall, whereas internal medicine residents scored significantly higher in the Collaborator competency overall. DISCUSSION: The OSCE checklists developed for the assessment of intrinsic CanMEDS roles were functional, but the specific items within stations required more uniformity to be used between stations. More generic types of checklists may also improve correlations across stations. CONCLUSION: An OSCE measuring intrinsic competence is feasible; however, further development of our cases and checklists is needed. We provide a model of how to develop an OSCE to measure intrinsic CanMEDS roles that educators may adopt as residency programs move into competency-based medical education.