Cargando…

Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement

COVID-19 has severely impacted mental health in vulnerable demographics, in particular older adults, who face unprecedented isolation. Consequences, while globally severe, are acutely pronounced in low- and middle-income countries (LMICs) confronting pronounced gaps in resources and clinician access...

Descripción completa

Detalles Bibliográficos
Autores principales: Lima, Maria R., Wairagkar, Maitreyee, Natarajan, Nirupama, Vaitheswaran, Sridhar, Vaidyanathan, Ravi
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8014955/
https://www.ncbi.nlm.nih.gov/pubmed/33816568
http://dx.doi.org/10.3389/frobt.2021.618866
_version_ 1783673590162391040
author Lima, Maria R.
Wairagkar, Maitreyee
Natarajan, Nirupama
Vaitheswaran, Sridhar
Vaidyanathan, Ravi
author_facet Lima, Maria R.
Wairagkar, Maitreyee
Natarajan, Nirupama
Vaitheswaran, Sridhar
Vaidyanathan, Ravi
author_sort Lima, Maria R.
collection PubMed
description COVID-19 has severely impacted mental health in vulnerable demographics, in particular older adults, who face unprecedented isolation. Consequences, while globally severe, are acutely pronounced in low- and middle-income countries (LMICs) confronting pronounced gaps in resources and clinician accessibility. Social robots are well-recognized for their potential to support mental health, yet user compliance (i.e., trust) demands seamless affective human-robot interactions; natural ‘human-like’ conversations are required in simple, inexpensive, deployable platforms. We present the design, development, and pilot testing of a multimodal robotic framework fusing verbal (contextual speech) and nonverbal (facial expressions) social cues, aimed to improve engagement in human-robot interaction and ultimately facilitate mental health telemedicine during and beyond the COVID-19 pandemic. We report the design optimization of a hybrid face robot, which combines digital facial expressions based on mathematical affect space mapping with static 3D facial features. We further introduce a contextual virtual assistant with integrated cloud-based AI coupled to the robot’s facial representation of emotions, such that the robot adapts its emotional response to users’ speech in real-time. Experiments with healthy participants demonstrate emotion recognition exceeding 90% for happy, tired, sad, angry, surprised and stern/disgusted robotic emotions. When separated, stern and disgusted are occasionally transposed (70%+ accuracy overall) but are easily distinguishable from other emotions. A qualitative user experience analysis indicates overall enthusiastic and engaging reception to human-robot multimodal interaction with the new framework. The robot has been modified to enable clinical telemedicine for cognitive engagement with older adults and people with dementia (PwD) in LMICs. The mechanically simple and low-cost social robot has been deployed in pilot tests to support older individuals and PwD at the Schizophrenia Research Foundation (SCARF) in Chennai, India. A procedure for deployment addressing challenges in cultural acceptance, end-user acclimatization and resource allocation is further introduced. Results indicate strong promise to stimulate human-robot psychosocial interaction through the hybrid-face robotic system. Future work is targeting deployment for telemedicine to mitigate the mental health impact of COVID-19 on older adults and PwD in both LMICs and higher income regions.
format Online
Article
Text
id pubmed-8014955
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-80149552021-04-02 Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement Lima, Maria R. Wairagkar, Maitreyee Natarajan, Nirupama Vaitheswaran, Sridhar Vaidyanathan, Ravi Front Robot AI Robotics and AI COVID-19 has severely impacted mental health in vulnerable demographics, in particular older adults, who face unprecedented isolation. Consequences, while globally severe, are acutely pronounced in low- and middle-income countries (LMICs) confronting pronounced gaps in resources and clinician accessibility. Social robots are well-recognized for their potential to support mental health, yet user compliance (i.e., trust) demands seamless affective human-robot interactions; natural ‘human-like’ conversations are required in simple, inexpensive, deployable platforms. We present the design, development, and pilot testing of a multimodal robotic framework fusing verbal (contextual speech) and nonverbal (facial expressions) social cues, aimed to improve engagement in human-robot interaction and ultimately facilitate mental health telemedicine during and beyond the COVID-19 pandemic. We report the design optimization of a hybrid face robot, which combines digital facial expressions based on mathematical affect space mapping with static 3D facial features. We further introduce a contextual virtual assistant with integrated cloud-based AI coupled to the robot’s facial representation of emotions, such that the robot adapts its emotional response to users’ speech in real-time. Experiments with healthy participants demonstrate emotion recognition exceeding 90% for happy, tired, sad, angry, surprised and stern/disgusted robotic emotions. When separated, stern and disgusted are occasionally transposed (70%+ accuracy overall) but are easily distinguishable from other emotions. A qualitative user experience analysis indicates overall enthusiastic and engaging reception to human-robot multimodal interaction with the new framework. The robot has been modified to enable clinical telemedicine for cognitive engagement with older adults and people with dementia (PwD) in LMICs. The mechanically simple and low-cost social robot has been deployed in pilot tests to support older individuals and PwD at the Schizophrenia Research Foundation (SCARF) in Chennai, India. A procedure for deployment addressing challenges in cultural acceptance, end-user acclimatization and resource allocation is further introduced. Results indicate strong promise to stimulate human-robot psychosocial interaction through the hybrid-face robotic system. Future work is targeting deployment for telemedicine to mitigate the mental health impact of COVID-19 on older adults and PwD in both LMICs and higher income regions. Frontiers Media S.A. 2021-03-18 /pmc/articles/PMC8014955/ /pubmed/33816568 http://dx.doi.org/10.3389/frobt.2021.618866 Text en Copyright © 2021 Lima, Wairagkar, Natarajan, Vaitheswaran and Vaidyanathan. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Robotics and AI
Lima, Maria R.
Wairagkar, Maitreyee
Natarajan, Nirupama
Vaitheswaran, Sridhar
Vaidyanathan, Ravi
Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement
title Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement
title_full Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement
title_fullStr Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement
title_full_unstemmed Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement
title_short Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement
title_sort robotic telemedicine for mental health: a multimodal approach to improve human-robot engagement
topic Robotics and AI
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8014955/
https://www.ncbi.nlm.nih.gov/pubmed/33816568
http://dx.doi.org/10.3389/frobt.2021.618866
work_keys_str_mv AT limamariar robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement
AT wairagkarmaitreyee robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement
AT natarajannirupama robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement
AT vaitheswaransridhar robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement
AT vaidyanathanravi robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement