External Validation of the eRADAR Risk Score for Detecting Undiagnosed Dementia in Two Real-World Healthcare Systems

BACKGROUND: Fifty percent of people living with dementia are undiagnosed. The electronic health record (EHR) Risk of Alzheimer’s and Dementia Assessment Rule (eRADAR) was developed to identify older adults at risk of having undiagnosed dementia using routinely collected clinical data. OBJECTIVE: To externally validate eRADAR in two real-world healthcare systems, including examining performance over time and by race/ethnicity. DESIGN: Retrospective cohort study PARTICIPANTS: 129,315 members of Kaiser Permanente Washington (KPWA), an integrated health system providing insurance coverage and medical care, and 13,444 primary care patients at University of California San Francisco Health (UCSF), an academic medical system, aged 65 years or older without prior EHR documentation of dementia diagnosis or medication. MAIN MEASURES: Performance of eRADAR scores, calculated annually from EHR data (including vital signs, diagnoses, medications, and utilization in the prior 2 years), for predicting EHR documentation of incident dementia diagnosis within 12 months. KEY RESULTS: A total of 7631 dementia diagnoses were observed at KPWA (11.1 per 1000 person-years) and 216 at UCSF (4.6 per 1000 person-years). The area under the curve was 0.84 (95% confidence interval: 0.84–0.85) at KPWA and 0.79 (0.76–0.82) at UCSF. Using the 90(th) percentile as the cut point for identifying high-risk patients, sensitivity was 54% (53–56%) at KPWA and 44% (38–51%) at UCSF. Performance was similar over time, including across the transition from International Classification of Diseases, version 9 (ICD-9) to ICD-10 codes, and across racial/ethnic groups (though small samples limited precision in some groups). CONCLUSIONS: eRADAR showed strong external validity for detecting undiagnosed dementia in two health systems with different patient populations and differential availability of external healthcare data for risk calculations. In this study, eRADAR demonstrated generalizability from a research sample to real-world clinical populations, transportability across health systems, robustness to temporal changes in healthcare, and similar performance across larger racial/ethnic groups. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s11606-022-07736-6.