Analysis of use of a single best answer format in an undergraduate medical examination

Examinations at the Faculty of Medicine of Mu'tah University are based on a single best answer multiple-choice questions (MCQs) format. However, the reliability of this examination format has not been determined. Objective: Using an examination of obstetrics and gynaecology as a model, this study aims to analyze the difficulty (facility) index, the discriminatory power and reliability of the examination format. Materials, Subjects and Methods: A prospective study on the psychometric performance was carried out for an undergraduate examination in obstetrics and gynaecology. The performance of the items was measured in terms of facility, discrimination and reliability. Two statistical tests were used to estimate the reliability of the exam: The Kuder-Richardson Formula 20 (KR-20) and the Cronbach alpha test. Results: The items scored well in facility with a significant portion (26%) achieving a positive point biserial of over 0.3 and a Cronbach alpha score of 0.947. However, 23% of the items had a negative point biserial and the Kuder-Richardson 20 (KR20) score was only 0.599. Conclusion: In order to improve the reliability of examinations, we recommend removing the items with negative point biserials and increasing the total number of items.