Goodness of fit of the items used in the 2nd cycle of evaluation and accreditation of medical schools by the Korea Institute of Medical Education and Evaluation based on the Rasch model

PURPOSE: Since 2004, the Korea Institute of Medical Education and Evaluation has been responsible for the evaluation and accreditation of medical schools in Korea. The 2nd cycle of evaluations was conducted from 2007 to 2011. The present study aimed at testing the goodness of fit of the items used in the 2nd cycle of evaluation and accreditation based on the Rasch model. METHODS: Dichotomous data on 40 medical schools were analyzed using Winsteps, a tool based on the Rasch model that includes goodness-of-fit testing. RESULTS: Two of the 109 items had an outfit mean square exceeding 2.0. The other 107 items showed a goodness of fit in the acceptable range for the outfit mean square. All items were in the acceptable range in terms of the infit mean square. Furthermore, 1 school had an outfit mean square exceeding 2.0, while all schools were in the acceptable range for the infit mean square. An outfit mean square value over 2.0 means that an item is a outlier. Therefore, 2 items showed an extreme response relative to the overall response. Meanwhile, the finding of an outfit mean square over 2.0 for 1 school means that it showed extraordinary responses to specific items, despite its excellent overall competency. CONCLUSION: The goodness of fit of the items used for evaluation and accreditation by the Korea Institute of Medical Education and Evaluation should be checked so that they can be revised appropriately. Furthermore, the outlier school should be investigated to determine why it showed such an inappropriate goodness of fit.