Digital Phenotyping Models of Symptom Improvement in College Mental Health: Generalizability Across Two Cohorts

Smartphones can be used to gain insight into mental health conditions through the collection of survey and sensor data. However, the external validity of this digital phenotyping data is still being explored, and there is a need to assess if predictive models derived from this data are generalizable. The first dataset (V1) of 632 college students was collected between December 2020 and May 2021. The second dataset (V2) was collected using the same app between November and December 2021 and included 66 students. Students in V1 could enroll in V2. The main difference between the V1 and V2 studies was that we focused on protocol methods in V2 to ensure digital phenotyping data had a lower degree of missing data than in the V1 dataset. We compared survey response counts and sensor data coverage across the two datasets. Additionally, we explored whether models trained to predict symptom survey improvement could generalize across datasets. Design changes in V2, such as a run-in period and data quality checks, resulted in significantly higher engagement and sensor data coverage. The best-performing model was able to predict a 50% change in mood with 28 days of data, and models were able to generalize across datasets. The similarities between the features in V1 and V2 suggest that our features are valid across time. In addition, models must be able to generalize to new populations to be used in practice, so our experiments provide an encouraging result toward the potential of personalized digital mental health care.