Developing a viva exam to assess clinical reasoning in pre-registration osteopathy students

BACKGROUND: Clinical reasoning (CR) is a core capability for health practitioners. Assessing CR requires a suite of tools to encompass a wide scope of contexts and cognitive abilities. The aim of this project was to develop an oral examination and grading rubric for the assessment of CR in osteopathy, trial it with senior students in three accredited university programs in Australia and New Zealand, and to evaluate its content and face validity. METHODS: Experienced osteopathic academics developed 20 cases and a grading rubric. Thirty senior students were recruited, 10 from each university. Twelve fourth year and 18 fifth year students participated. Three members of the research team were trained and examined students at an institution different from their own. Two cases were presented to each student participant in a series of vignettes. The rubric was constructed to follow a set of examiner questions that related to each attribute of CR. Data were analysed to explore differences in examiner marking, as well as relationships between cases, institutions, and different year levels. A non-examining member of the research team acted as an observer at each location. RESULTS: No statistical difference was found between the total and single question scores, nor for the total scores between examiners. Significant differences were found between 4(th) and 5(th) students on total score and a number of single questions. The rubric was found to be internally consistent. CONCLUSIONS: A viva examination of clinical reasoning, trialled with senior osteopathy students, showed face and content validity. Results suggested that the viva exam may also differentiate between 4(th) and 5(th) year students’ capabilities in CR. Further work is required to establish the reliability of assessment, to further refine the rubric, and to train examiners before it is implemented as a high-stakes assessment in accredited osteopathy programs. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/1472-6920-14-193) contains supplementary material, which is available to authorized users.