Assessment of Junior Doctor performance: a validation study

BACKGROUND: In recent years, Australia has developed a National Junior Doctor Curriculum Framework that sets out the expected standards and describes areas of performance for junior doctors and through this has allowed a national approach to junior doctor assessment to develop. Given the significance of the judgments made, in terms of patient safety, development of junior doctors, and preventing progression of junior doctors moving to the next stage of training, it is essential to develop and validate assessment tools as rigorously as possible. This paper reports on a validation study of the Junior Doctor Assessment Tool as used for PGY1 doctors to evaluate the psychometric properties of the instrument and to explore the effect of length of experience as a PGY1 on assessment scores. METHODS: This validation study of the Australian developed Junior Doctor Assessment Tool as it was used in three public and other associated hospitals in Western Australia for PGY1 across a two year period addressed two core aims, namely: (1) to evaluate the psychometric properties of the instrument; (2) to explore the effect of length of experience as a PGY1 on assessment scores. RESULTS: The highest mean scores were for professional behaviours, teamwork and interpersonal skills and the lowest were for procedures. Most junior doctors were assessed three or more times and scores were not different in the first rotation compared to subsequent rotations. While statistically significant, there appeared to be little practical influence on scores obtained by the number of times they were assessed. Principal component analysis identified two principal components of junior doctor performance are being assessed rather than the commonly reported three. A Cronbach Alpha of .883 was calculated for the 10 item scale. CONCLUSIONS: Now that the components of the tool have been analysed it will be more meaningful and potentially more influential to consider these factors on the potential educational impact of this assessment process for monitoring junior doctor development and progression.