Cargando…

Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach

BACKGROUND: Mental health disorders affect multiple aspects of patients’ lives, including mood, cognition, and behavior. eHealth and mobile health (mHealth) technologies enable rich sets of information to be collected noninvasively, representing a promising opportunity to construct behavioral marker...

Descripción completa

Detalles Bibliográficos
Autores principales: Sükei, Emese, Norbury, Agnes, Perez-Rodriguez, M Mercedes, Olmos, Pablo M, Artés, Antonio
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8088855/
https://www.ncbi.nlm.nih.gov/pubmed/33749612
http://dx.doi.org/10.2196/24465
_version_ 1783686923930304512
author Sükei, Emese
Norbury, Agnes
Perez-Rodriguez, M Mercedes
Olmos, Pablo M
Artés, Antonio
author_facet Sükei, Emese
Norbury, Agnes
Perez-Rodriguez, M Mercedes
Olmos, Pablo M
Artés, Antonio
author_sort Sükei, Emese
collection PubMed
description BACKGROUND: Mental health disorders affect multiple aspects of patients’ lives, including mood, cognition, and behavior. eHealth and mobile health (mHealth) technologies enable rich sets of information to be collected noninvasively, representing a promising opportunity to construct behavioral markers of mental health. Combining such data with self-reported information about psychological symptoms may provide a more comprehensive and contextualized view of a patient’s mental state than questionnaire data alone. However, mobile sensed data are usually noisy and incomplete, with significant amounts of missing observations. Therefore, recognizing the clinical potential of mHealth tools depends critically on developing methods to cope with such data issues. OBJECTIVE: This study aims to present a machine learning–based approach for emotional state prediction that uses passively collected data from mobile phones and wearable devices and self-reported emotions. The proposed methods must cope with high-dimensional and heterogeneous time-series data with a large percentage of missing observations. METHODS: Passively sensed behavior and self-reported emotional state data from a cohort of 943 individuals (outpatients recruited from community clinics) were available for analysis. All patients had at least 30 days’ worth of naturally occurring behavior observations, including information about physical activity, geolocation, sleep, and smartphone app use. These regularly sampled but frequently missing and heterogeneous time series were analyzed with the following probabilistic latent variable models for data averaging and feature extraction: mixture model (MM) and hidden Markov model (HMM). The extracted features were then combined with a classifier to predict emotional state. A variety of classical machine learning methods and recurrent neural networks were compared. Finally, a personalized Bayesian model was proposed to improve performance by considering the individual differences in the data and applying a different classifier bias term for each patient. RESULTS: Probabilistic generative models proved to be good preprocessing and feature extractor tools for data with large percentages of missing observations. Models that took into account the posterior probabilities of the MM and HMM latent states outperformed those that did not by more than 20%, suggesting that the underlying behavioral patterns identified were meaningful for individuals’ overall emotional state. The best performing generalized models achieved a 0.81 area under the curve of the receiver operating characteristic and 0.71 area under the precision-recall curve when predicting self-reported emotional valence from behavior in held-out test data. Moreover, the proposed personalized models demonstrated that accounting for individual differences through a simple hierarchical model can substantially improve emotional state prediction performance without relying on previous days’ data. CONCLUSIONS: These findings demonstrate the feasibility of designing machine learning models for predicting emotional states from mobile sensing data capable of dealing with heterogeneous data with large numbers of missing observations. Such models may represent valuable tools for clinicians to monitor patients’ mood states.
format Online
Article
Text
id pubmed-8088855
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-80888552021-05-07 Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach Sükei, Emese Norbury, Agnes Perez-Rodriguez, M Mercedes Olmos, Pablo M Artés, Antonio JMIR Mhealth Uhealth Original Paper BACKGROUND: Mental health disorders affect multiple aspects of patients’ lives, including mood, cognition, and behavior. eHealth and mobile health (mHealth) technologies enable rich sets of information to be collected noninvasively, representing a promising opportunity to construct behavioral markers of mental health. Combining such data with self-reported information about psychological symptoms may provide a more comprehensive and contextualized view of a patient’s mental state than questionnaire data alone. However, mobile sensed data are usually noisy and incomplete, with significant amounts of missing observations. Therefore, recognizing the clinical potential of mHealth tools depends critically on developing methods to cope with such data issues. OBJECTIVE: This study aims to present a machine learning–based approach for emotional state prediction that uses passively collected data from mobile phones and wearable devices and self-reported emotions. The proposed methods must cope with high-dimensional and heterogeneous time-series data with a large percentage of missing observations. METHODS: Passively sensed behavior and self-reported emotional state data from a cohort of 943 individuals (outpatients recruited from community clinics) were available for analysis. All patients had at least 30 days’ worth of naturally occurring behavior observations, including information about physical activity, geolocation, sleep, and smartphone app use. These regularly sampled but frequently missing and heterogeneous time series were analyzed with the following probabilistic latent variable models for data averaging and feature extraction: mixture model (MM) and hidden Markov model (HMM). The extracted features were then combined with a classifier to predict emotional state. A variety of classical machine learning methods and recurrent neural networks were compared. Finally, a personalized Bayesian model was proposed to improve performance by considering the individual differences in the data and applying a different classifier bias term for each patient. RESULTS: Probabilistic generative models proved to be good preprocessing and feature extractor tools for data with large percentages of missing observations. Models that took into account the posterior probabilities of the MM and HMM latent states outperformed those that did not by more than 20%, suggesting that the underlying behavioral patterns identified were meaningful for individuals’ overall emotional state. The best performing generalized models achieved a 0.81 area under the curve of the receiver operating characteristic and 0.71 area under the precision-recall curve when predicting self-reported emotional valence from behavior in held-out test data. Moreover, the proposed personalized models demonstrated that accounting for individual differences through a simple hierarchical model can substantially improve emotional state prediction performance without relying on previous days’ data. CONCLUSIONS: These findings demonstrate the feasibility of designing machine learning models for predicting emotional states from mobile sensing data capable of dealing with heterogeneous data with large numbers of missing observations. Such models may represent valuable tools for clinicians to monitor patients’ mood states. JMIR Publications 2021-03-22 /pmc/articles/PMC8088855/ /pubmed/33749612 http://dx.doi.org/10.2196/24465 Text en ©Emese Sükei, Agnes Norbury, M Mercedes Perez-Rodriguez, Pablo M Olmos, Antonio Artés. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 22.03.2021. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Sükei, Emese
Norbury, Agnes
Perez-Rodriguez, M Mercedes
Olmos, Pablo M
Artés, Antonio
Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach
title Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach
title_full Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach
title_fullStr Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach
title_full_unstemmed Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach
title_short Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach
title_sort predicting emotional states using behavioral markers derived from passively sensed data: data-driven machine learning approach
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8088855/
https://www.ncbi.nlm.nih.gov/pubmed/33749612
http://dx.doi.org/10.2196/24465
work_keys_str_mv AT sukeiemese predictingemotionalstatesusingbehavioralmarkersderivedfrompassivelysenseddatadatadrivenmachinelearningapproach
AT norburyagnes predictingemotionalstatesusingbehavioralmarkersderivedfrompassivelysenseddatadatadrivenmachinelearningapproach
AT perezrodriguezmmercedes predictingemotionalstatesusingbehavioralmarkersderivedfrompassivelysenseddatadatadrivenmachinelearningapproach
AT olmospablom predictingemotionalstatesusingbehavioralmarkersderivedfrompassivelysenseddatadatadrivenmachinelearningapproach
AT artesantonio predictingemotionalstatesusingbehavioralmarkersderivedfrompassivelysenseddatadatadrivenmachinelearningapproach