Knowledge, application and how about competence? Qualitative assessment of multiple-choice questions for dental students

Background: Many medical schools train their faculty members to construct high cognitive level multiple choice questions (MCQs) that demand a great deal of analytical and critical thinking, application, and competence. The purpose of this study is to determine the cognitive levels of MCQs by using Moore’s Expanded Outcomes Framework and to understand whether the quality of MCQs has an effect on students’ assessment performance. Methods: Four trained faculty members analysed 100 randomly selected questions developed at the University Clinic of Dentistry (UCD) and 100 questions developed by the National Board of Dental Examinations (NBDE). Moore’s framework was applied to assist the review process. Results: The majority of questions was at the level of declarative knowledge followed by questions at the level of procedural knowledge. The cognitive level of UCD questions from 2002 to 2009 was significantly lower than that of NBDE questions but increased in questions written from 2010 to 2018. The improvement of quality of MCQs had no impact on assessment performance of students. Conclusion: The enhanced cognitive levels of UCD MCQs written 2010–2018 coincides with the implementation of a faculty training program for writing high-ordered MCQs. In addition, this study shows that the use of Moore’s expanded framework is on par with other known taxonomies in supporting educators in writing items and reviewing the process. Abbreviations: MCQs: Multiple Choice Questions; UCD: University Clinic of Dentistry; NBDE: National Board of Dental Examinations