Development and Evaluation of a Pragmatic Measure of Adherence to Dialectical Behavior Therapy: The DBT Adherence Checklist for Individual Therapy

This paper presents two studies conducted to develop and evaluate a new pragmatic measure of therapist adherence to Dialectical Behavior Therapy (DBT): the DBT Adherence Checklist for Individual Therapy (DBT AC-I). Study 1 used item response analysis to select items from the gold standard DBT Adherence Coding Scale (DBT ACS) using archival data from 1271 DBT sessions. Items were then iteratively refined based on feedback from 33 target end-users to ensure relevance, usability, and understandability. Study 2 examined the psychometric properties of the DBT AC-I as a therapist self-report and observer-rated measure in 100 sessions from 50 therapist-client dyads, while also evaluating predictors of therapist accuracy in self-rated adherence. When used as a therapist self-report measure, concordance between therapist and observer ratings was at least moderate (AC(1) ≥ 0.41) for all DBT AC-I items but overall concordance (ICC = 0.09) as well as convergent (r = 0.05) and criterion validity (AUC = 0.54) with the DBT ACS were poor. Higher therapist accuracy was predicted by greater DBT knowledge and adherence as well as more severe client suicidal ideation. When used by trained observers, the DBT AC-I had excellent interrater reliability (ICC = 0.93), convergent validity (r = 0.90), and criterion validity (AUC = 0.94). While therapists’ self-rated adherence on the DBT AC-I should not be assumed to reflect their actual adherence, some therapists may self-rate accurately. The DBT AC-I offers an effective and relatively efficient method of evaluating adherence to DBT when used by trained observers. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s10488-023-01274-x.