Assessment in pediatrics clerkships: impact of strategies to solve item-sharing problems

INTRODUCTION: Reusing multiple-choice questions in different examinations may lead to item sharing between students. Our aim was to analyze and solve an item-sharing problem during assessment of medical students in pediatrics clerkship. METHODS: This is a 3 years prospective analysis of 5th year medical students, submitted to an examination at the end of their Pediatrics’ clerkship. In 2012, questions were reused across different clerkships. In 2013, no questions were reused in different clerkships. In 2014, no questions were reused and the review of the test was postponed to the end of the year, after all clerkships ended. RESULTS: In 2012, the mean score increased 1.36 points (in a scale of 0–20) per clerkship rotation, with the last clerkship having a difference of 9.5 points regarding the first one (P < .001). Fifty percent of this variation was due to the repetition of questions. In 2013, with a new question bank, the mean score increased 0.8 points per clerkship rotation, with a difference of 5.6 points between the last and the first rotations (P < .024). Finally, in 2014 there was no significant variation between clerkships. Tests’ scores had a significant moderate correlation with students’ average course grade (r = 0.39, r = 0.30, r = 0.48, for 2012, 2013, and 2014, respectively). The students’ average course grade, however, did not confound the increase in tests’ scores across different clerkships. CONCLUSION: The present work demonstrated an item-sharing problem among students during pediatric clerkships. An effective approach to correct this bias assessment was achieved by restricting the reuse of questions, by changing the time-point of test revision and by progressively adapting equating strategies.