Assessment of US Paramedic Professionalism: A Psychometric Appraisal

INTRODUCTION: Professionalism is an essential behavior for paramedic students to demonstrate. In the United States, paramedic accreditation standards require educators to evaluate and document summative affective evaluation on each paramedic student before graduation. The 2009 Emergency Medical Services Education Standards identified the affective behaviors as one of the three learning domains and published a grading tool to help educators recognize professional behaviors. However, little attention was given to the validity or reliability of this tool. Therefore, the aim of this study was to evaluate the psychometric properties of the 5-point Paramedic Affective Domain Tool. METHODS: This was a retrospective study with educators that completed evaluations on paramedic students from May 2013 to January 2017. A total of 707 cases met inclusion criteria and 131 unique evaluators from 27 different paramedic programs. A Rasch Partial Credit Model was used to analyze the data. RESULTS: Almost 97% of the paramedic students received passing scores and 28.1% (n=199) received perfect scores. Only 3.5% (n=25) failed the evaluation. Scores ranged from 11 to 55 (M = 46, SD = 9.02) and α = 0.97. Evidence suggests that the tool is not valid and the clustering of scores suggests minimal information can be gleaned from the results. CONCLUSION: Serious consideration should be made in the continued use of this tool and future research should focus on developing a new tool that is both valid and reliable.