Psychometric evaluation of a decision quality instrument for medication decisions for treatment of depression symptoms

BACKGROUND: A high quality treatment decision means patients are informed and receive treatment that matches their goals. This research examined the reliability and validity of the Depression Decision Quality Instrument (DQI), a survey to measure the extent to which patients are informed and received preferred treatment for depression. METHODS: Participants were aged 18 and older from 17 US cities who discussed medication or counseling with a physician in the past year, and physicians who treated patients with depression who practiced in the same cities. Participants were mailed a survey that included the Depression-DQI, a tool with 10 knowledge and 7 goal and concern items. Patients were randomly assigned to either receive a patient decision aid (DA) on treatment of depression or no DA. A matching score was created by comparing the patient’s preferred treatment to their self-reported treatment received. Concordant scores were considered matched, discordant were not. We examined the reliability and known group validity of the Depression-DQI. RESULTS: Most patients 405/504 (80%) responded, 79% (320/405) returned the retest survey, and 60% (114/187) of physicians returned the survey. Patients’ knowledge scores on the 10-item scale ranged from 14.6 to 100% with no evidence of floor or ceiling effects. Retest reliability for knowledge was moderate and for goals and concerns ranged from moderate to good. Mean knowledge scores differentiated between patients and physicians (M = 63 [SD = 15] vs. M = 81 [SD = 11], p < 0.001), and between patients who did and didn’t receive a DA (M = 64 [SD = 16] vs. M = 61 [SD = 14], p = 0.041). 60.5% of participants received treatment that matched their preference. Based on the multivariate logistic regression, ‘avoiding taking anti-depressants’ was the only goal that was predictive of taking mediation (OR = 0.73 [0.66, 0.80], p < 0.01). Shared Decision Making Process scores were similar for those who matched their preference and those who didn’t (M = 2.18 [SD = 0.97] vs. M = 2.06 [SD = 1.07]; t(320) = − 1.06, p = 0.29). Those who matched had lower regret scores (matched M = 1.72 [SD = 0.74] vs. unmatched M = 2.32 [SD = 0.8]; t(301) = − 6.6, p < .001). CONCLUSIONS: The Depression DQI demonstrated modest reliability and validity. More work is needed to establish validity of the method to determine concordance. Trial registration: NCT01152307. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12911-021-01611-w.