Digital Phenotyping Data to Predict Symptom Improvement and Mental Health App Personalization in College Students: Prospective Validation of a Predictive Model

BACKGROUND: Mental health apps offer a transformative means to increase access to scalable evidence-based care for college students. Yet low rates of engagement currently preclude the effectiveness of these apps. One promising solution is to make these apps more responsive and personalized through digital phenotyping methods able to predict symptoms and offer tailored interventions. OBJECTIVE: Following our protocol and using the exact model shared in that paper, our primary aim in this study is to assess the prospective validity of mental health symptom prediction using the mindLAMP app through a replication study. We also explored secondary aims around app intervention personalization and correlations of engagement with the Technology Acceptance Model (TAM) and Digital Working Alliance Inventory scale in the context of automating the study. METHODS: The study was 28 days in duration and followed the published protocol, with participants collecting digital phenotyping data and being offered optional scheduled and algorithm-recommended app interventions. Study compensation was tied to the completion of weekly surveys and was not otherwise tied to engagement or use of the app. RESULTS: The data from 67 participants were used in this analysis. The area under the curve values for the symptom prediction model ranged from 0.58 for the UCLA Loneliness Scale to 0.71 for the Patient Health Questionnaire-9. Engagement with the scheduled app interventions was high, with a study mean of 73%, but few participants engaged with the optional recommended interventions. The perceived utility of the app in the TAM was higher (P=.01) among those completing at least one recommended intervention. CONCLUSIONS: Our results suggest how digital phenotyping methods can be used to create generalizable models that may help create more personalized and engaging mental health apps. Automating studies is feasible, and our results suggest targets to increase engagement in future studies. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/37954