Using objective structured clinical examination (OSCE) for circulating and scrub skills of first-semester operating room technology students: Is it valid and reliable?

OBJECTIVES: This study aimed to determine the validity and reliability of an Objective Structured Clinical Examination (OSCE) developed for the summative assessment of the practical credit of the course titled “Principles and Techniques of Circulating and Scrub Persons’ Practice.” METHODS: This applied descriptive study was conducted on 63 first-semester undergraduate Operating Room (OR) technology students. Totally, 17 circulating and scrub skills were incorporated in four 7-min OSCE stations. The content validity of the examination was assessed by an expert panel (n = 14). Additionally, its concurrent validity was evaluated by Intra-Class Correlation (ICC) through measuring the correlation between the total score of OSCE and grade point average, grades of the practical and theoretical credits of the course that were obtained by multiple-choice tests, and the total grade of the course (achieved by the grades of the practical and theoretical credits tests). The Inter-Rater Reliability (IRR) was also calculated by the correlation between the scores recorded by two examiners at each OSCE station using ICC. Meanwhile, the internal consistency across stations (item-total correlation) and across items within a station was determined by Cronbach's alpha (α). RESULTS: The OSCE was deemed to fulfill the content validity criteria. The total score of OSCE showed a significant correlation with the concurrent validity criteria (ICC = 0.66–0.85, p < 0.001). Besides, the IRR was significant for the total score of OSCE as well as for the score of each station (ICC = 0.90–0.98, p < 0.001). Likewise, a good level of internal consistency was found across stations (α = 0.63–0.93) and across items within a station (α = 0.78–0.92). CONCLUSION: The developed OSCE had appropriate validity and reliability. Thus, it can be used to assess the circulating and scrub skills of first-semester OR technology students.