Summative assessment of 5(th) year medical students’ clinical reasoning by script concordance test: requirements and challenges

BACKGROUND: The Script Concordance Test (SCT) has not been reported in summative assessment of students across the multiple domains of a medical curriculum. We report the steps used to build a test for summative assessment in a medical curriculum. METHODS: A 51 case, 158-question, multidisciplinary paper was constructed to assess clinical reasoning in 5(th)-year. 10–16 experts in each of 7 discipline-based reference panels answered questions on-line. A multidisciplinary group considered reference panel data and data from a volunteer group of 6(th) Years, who sat the same test, to determine the passing score for the 5(th) Years. RESULTS: The mean (SD) scores were 63.6 (7.6) and 68.6 (4.8) for the 6(th) Year (n = 23, alpha = 0.78) and and 5(th) Year (n = 132, alpha =0.62) groups (p < 0.05), respectively. The passing score was set at 4 SD from the expert mean. Four students failed. CONCLUSIONS: The SCT may be a useful method to assess clinical reasoning in medical students in multidisciplinary summative assessments. Substantial investment in training of faculty and students and in the development of questions is required.