Development and psychometric evaluation of scales to measure professional confidence in manual medicine: a Rasch measurement approach

BACKGROUND: Health professionals in athletic training, chiropractic, osteopathy, and physiotherapy fields, require high-level knowledge and skills in their assessment and management of patients. This is important when communicating with patients and applying a range of manual procedures. Prior to embarking on professional practice, it is imperative to acquire optimal situation-specific levels of self-confidence for a beginner practitioner in these areas. In order to foster this professional self-confidence within the higher education context, it is necessary to have valid and reliable scales that can measure and track levels and how they change. This study reports on the development and psychometric analysis of two new scales, Patient Communication Confidence Scale (PCCS) and the Clinical Skills Confidence Scale (CSCS), to measure confidence in these two areas for students in manual medicine programs. The Rasch measurement model was used to guide the development of the scales and establish their psychometric properties. METHODS: The responses to 269 returned questionnaires over two occasions were submitted to psychometric analysis, with various aspects of the scales examined including: item thresholds; item fit; Differential Item Functioning; targeting; item locations; item dependencies; and reliability. To provide further evidence of validity, scores were correlated with two existing valid scales. RESULTS: Analyses showed that the scales provided valid and reliable measures of confidence for this sample of persons. High Person Separation Indices (0.96 for PCCS; 0.93 for SCSC) provided statistical evidence of reliability, meaning the scales are able to discriminate amongst persons with different levels of confidence. For the PCCS, item categories were operating as required, and for the CSCS only two items’ thresholds were slightly disordered. Three tests of fit revealed good fit to the model (indicating the internal consistency of both scales) and results of the correlations with two existing valid scales were consistent with expectations. CONCLUSIONS: The importance of confidence cannot be overlooked in health education because students learning new information and skills, and dealing with challenging situations can be negatively impacted by a lack of confidence which can result in students disengaging from placements or leaving a program. Valid and reliable instruments are essential in tracking change in levels of confidence in specific skills over time and the examination of the degree of congruence between confidence and competence. Analysis of responses to the two confidence scales established that they are valid and reliable instruments.