Simulation-based clinical assessment identifies threshold competence to practise physiotherapy in Australia: a crossover trial

BACKGROUND: Although evidence exists for the efficacy of high-fidelity simulation as an educational tool, there is limited evidence for its application in high-stakes professional threshold competency assessment. An alternative model of simulation-based assessment was developed by the Australian Physiotherapy Council (APC), using purpose-written standardised patients, mapped to the appropriate threshold level. The aim of this two-phase study was to investigate whether simulation-based clinical assessments resulted in equivalent outcomes to standard, real-life assessments for overseas-trained physiotherapists seeking registration to practice in Australia. METHODS: A randomised crossover trial comparing simulation-based assessment to real-life assessment was completed. Participants were internationally trained physiotherapists applying for registration to practice in Australia, voluntarily recruited from the Australian Physiotherapy Council (APC) assessment waiting list: study 1 n = 25, study 2 n = 144. Study 1 participants completed usual APC real-life assessments in 3 practice areas, completed on different days at APC partner healthcare facilities. Participants also underwent 3 practice area-matched simulation-based assessments, completed on the same day at purpose-designed simulation facilities. Study 2 participants completed 3 simulation-based assessments and 1 real-life assessment that was randomly allocated for order and practice area. Assessment of competency followed the standard APC procedure of 90-minute examinations using The Moderated Assessment Form (MAF). RESULTS: The overall pass rate was higher for real-life assessments in both studies: study 1, 50% versus 42.7%; study 2, 55.6% versus 44.4%. Chi-square analysis showed a high to moderate level of exact matching of pass/fail grades across all assessments: study 1, 73.4% (p < 0.001); study 2, 58.3% (p = 0.027). Binary logistic regression showed that the best predictors of real-life pass/fail grade were simulation-based MAF pass/fail grade (study 1, OR 7.86 p < 0.001; study 2, OR 2.037, p = 0.038) and simulation-based total MAF score (study 1, OR 1.464 p < 0.001; study 2, OR 1.234, p = 0.001). CONCLUSION: Simulation-based assessment is a significant predictor of clinical performance and can be used to successfully identify high stakes threshold competence to practice physiotherapy in Australia.