Pragmatic Quality Assessment of Brief Health Behavior Change Interventions: Evidence for Criterion-Related Validity of the Behavior Change Counseling Index

BACKGROUND: Healthcare providers frequently engage patients in conversations about health behavior change and are encouraged to use patient-centered approaches, such as Motivational Interviewing. Training in and sustainment of these skills are known to require feedback based on actual or role-played patient encounters. The behavior change counseling index (BECCI) is a pragmatic measure to assess healthcare providers’ patient-centered behavior change counseling skills that was developed as an alternative to resource-intensive “gold standard” measures, which are difficult to use in routine practice. We are not aware of any studies that examine the criterion-related validity of this measure using an alternative gold standard measure. We examined the criterion-related validity of the BECCI as rated by a simulated patient actor immediately after a brief behavior change intervention role-play using objective ratings on the motivational interviewing treatment integrity (MITI) scale. METHODS: We conducted a secondary analysis of data from a 25-site clinical trial of screening and intervention for posttraumatic stress disorder and comorbidities with patients at level I trauma centers in the USA. Participants were 64 providers representing diverse professional roles trained to deliver a multi-component intervention with study patients. As part of the training, providers role-played counseling a patient to reduce risky alcohol use with a simulated patient actor. These 20-min role-plays were conducted by telephone and audio recorded. Immediately after the role-play, the simulated patient actor rated the quality of the providers’ patient-centered behavior change counseling skills using the BECCI. A third-party expert MITI rater later listened to the audio recordings of the role-plays and rated the quality of the providers’ patient-centered behavior change counseling skills using the MITI 3.1.1. RESULTS: All correlations observed were statistically significant. The overall BECCI score correlated strongly (≥ 0.50) with five of the six MITI scores and moderately (0.33) with MITI percent complex reflections. CONCLUSIONS: This study provides evidence of criterion-related validity of the BECCI with a sample of healthcare providers representing a range of professional roles. Simulated patient actor rating using the BECCI is a pragmatic approach to assessing the quality of brief behavior change interventions delivered by healthcare providers.