Assessing musculoskeletal examination skills and diagnostic reasoning of 4th year medical students using a novel objective structured clinical exam

BACKGROUND: Medical students have difficulty performing and interpreting musculoskeletal physical examinations and interpreting the findings. Research has focused on students' knowledge deficits, but there are few direct assessments of students' ability to perform a hypothesis-driven physical examination (HDPE). We developed a novel musculoskeletal Objective Structured Clinical Exam (OSCE) focusing on HDPE skills for disorders of the shoulder, back and knee, and used it to explore medical student diagnostic reasoning. METHODS: A multidisciplinary group of musculoskeletal specialists developed and gathered validity evidence for a three station OSCE focusing on the HDPE of the shoulder, back and knee, emphasizing the ability to anticipate (identify pre-encounter) expected physical exam findings, and subsequently perform discriminatory physical examination maneuvers. The OSCE was administered to 45 final year medical students. Trained faculty observed and scored students’ ability to anticipate exam findings and perform diagnostic examination maneuvers on simulated patients. Encounters were digitally recorded and scored again by another trained faculty member. Inter-rater reliability for each maneuver was estimated using type-2 intra-class correlations (ICC). Percentages of perfect scores for anticipation and performance were calculated. Pearson’s correlation between anticipation and performance scores was computed for each maneuver and their relationship to diagnostic accuracy was tested with logistic regression. RESULTS: Inter-rater reliability was good (ICC between .69 and .87) for six exam maneuvers. Maneuver performance was overall poor, with no discriminatory maneuver performed correctly by more than two thirds of students, and one maneuver only performed correctly by 4 % of students. For the shoulder and knee stations, students were able to anticipate necessary discriminatory exam findings better than they could actually perform relevant exam maneuvers. The ability to anticipate a discriminatory finding correlated with the ability to perform the associated maneuver correctly, with the exception of the ability to perform maneuvers needed to diagnose a torn anterior cruciate ligament of the knee. Neither the ability to anticipate or perform was predictive of identifying correct diagnoses for the different cases. CONCLUSIONS: A novel musculoskeletal OSCE, based on principles of the hypothesis-driven physical examination, was able to identify significant deficiencies in examination skills needed to diagnose common disorders of the shoulder, back and knee amongst graduating medical students. In addition, the OSCE demonstrated that accurate anticipation of discriminatory examination findings correlates with ability to perform the associated maneuver; however, the ability to anticipate exceeds the ability to perform. Students do not appear to be using the physical exam to inform their diagnostic reasoning. The findings of this study have implications for both assessment and teaching of the musculoskeletal exam. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12909-016-0780-4) contains supplementary material, which is available to authorized users.