Do students achieve the desired learning goals using open-book formative assessments?

OBJECTIVES: The present study aimed to examine whether medical students benefit from an open-book online formative assessment as a preparation for a practical course. METHODS: A between-subjects experimental design was used: participants – a whole cohort of second-year medical students (N=232) – were randomly assigned to either a formative assessment that covered the topic of a subsequent practical course (treatment condition) or a formative assessment that did not cover the topic of the subsequent course (control condition). Course-script-knowledge, as well as additional in-depth-knowledge, was assessed. RESULTS: Students in the treatment condition had better course-script knowledge, both at the beginning, t((212)) = 4.96, p < .01, d = 0.72., and in the end of the practical course , t((208)) = 4.80, p < .01, d = 0.68. Analyses of covariance show that this effect is stronger for those students who understood the feedback that was presented within the formative assessment, F((1, 213))=10.17, p<.01. Additionally, the gain of in-depth-knowledge was significantly higher for students in the treatment condition compared to students in the control condition, t((208)) = 3.68., p < .05, d = 0.72 (0.51). CONCLUSIONS: Students benefit from a formative assessment that is related to and takes place before a subsequent practical course. They have a better understanding of the topic and gain more in-depth-knowledge that goes beyond the content of the script. Moreover, the study points out the importance of feedback pages in formative assessments.