Agreement between self-reports and statutory health insurance claims data on healthcare utilization in patients with mental disorders

BACKGROUND: Data on resource use are frequently required for healthcare assessments. Studies on healthcare utilization (HCU) in individuals with mental disorders have analyzed both self-reports and administrative data. Source of data may affect the quality of analysis and compromise the accuracy of results. We sought to ascertain the degree of agreement between self-reports and statutory health insurance (SHI) fund claims data from patients with mental disorders. METHODS: Claims data from six German SHI and self-reports were obtained along with a cost-effectiveness analysis performed as a part of a controlled prospective multicenter cohort study conducted in 18 psychiatric hospitals in Germany (PsychCare), including patients with pre-defined psychiatric disorders. Self-reports were collected using the German adaption of the Client Sociodemographic and Service Receipt Inventory (CSSRI) questionnaire with a 6-month recall period. Data linkage was performed using a unique pseudonymized identifier. Missing responses were coded as non-use for all analyses. HCU was calculated for inpatient and outpatient care, day-care services, home treatment, and pharmaceuticals. Concordance was measured using Cohen’s Kappa (κ) and intraclass correlation coefficient (ICC). Regression approaches were used to investigate the effect of independent variables on the agreements. RESULTS: In total 274 participants (mean age 47.8 [SD = 14.2] years; 47.08% women) were included in the analysis. No significant differences were observed between the linked and unlinked patients in terms of baseline characteristics. Total agreements values were 63.9% (κ = 0.03; PABAK = 0.28) for outpatient contacts, 69.3% (κ = 0.25; PABAK = 0.39) for medication use, 81.0% (κ = 0.56; PABAK = 0.62) for inpatient days and 86.1% (κ = 0.67; PABAK = 0.72) for day-care services. There was varied quantitative agreement between data sources, with the poorest agreement for outpatient care (ICC [95% CI] = 0.22 [0.10–0.33]) and the best for psychiatric day-care services (ICC [95% CI] = 0.72 [0.66–0.78]). Marital status and time since first treatment positively affected the chance of agreement on utilization of outpatient services. CONCLUSIONS: Although there were high levels of absolute agreement, the measures of concordance between administrative records and self-reports were generally minimal to moderate. Healthcare investigations should consider using linked or at least different data sources to estimate HCU for specific utilization areas, where unbiased information can be expected. TRIAL REGISTRATION: This study was part of the multi-center controlled PsychCare trial (German Clinical Trials Register No. DRKS00022535; Date of registration: 2020–10-02). SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12913-023-10175-6.