Assessment during Covid-19: quality assurance of an online open book formative examination for undergraduate medical students

BACKGROUND: The spread of COVID-19 pandemic in early 2020 compelled all the educational activities, including medical education to be shifted from face-to-face interaction to a virtual platform. This shift provided opportunities for exploring online assessment modalities. One such assessment method is an online open book exam which is a unique concept in medical education of Pakistan. Limited information is available in literature regarding open book exam for the basic science subjects. Hence, the objective of the study was to determine the quality of the open book exam administered as a pilot project to the first-year medical students. METHODS: It was a cross-sectional analytical study that included 99 students of first year MBBS. The students were administered an online unrestricted type of open book exam as a formative assessment. The exam consisted of 30 open-ended, short answer type questions. The scores of the exam were analyzed for psychometric quality. RESULTS: The mean score was 47.24 ± 15.30 SD %. The reliability of the exam was 0.79. The majority (66.6%) of items were found to be moderately difficult with their difficulty index ranging from 31 to 80%. The majority (86.6%) items were in the range of moderate to high discrimination. There were no questions with negative discrimination. CONCLUSIONS: The exam was found to be reliable and can be implemented with training of faculty and students. Online open book exam provides a good format for remote and formative assessment of students with minimum proctoring during times of constraints such as COVID-19 pandemic. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12909-022-03849-y.