Cargando…
Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement
COVID-19 has severely impacted mental health in vulnerable demographics, in particular older adults, who face unprecedented isolation. Consequences, while globally severe, are acutely pronounced in low- and middle-income countries (LMICs) confronting pronounced gaps in resources and clinician access...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8014955/ https://www.ncbi.nlm.nih.gov/pubmed/33816568 http://dx.doi.org/10.3389/frobt.2021.618866 |
_version_ | 1783673590162391040 |
---|---|
author | Lima, Maria R. Wairagkar, Maitreyee Natarajan, Nirupama Vaitheswaran, Sridhar Vaidyanathan, Ravi |
author_facet | Lima, Maria R. Wairagkar, Maitreyee Natarajan, Nirupama Vaitheswaran, Sridhar Vaidyanathan, Ravi |
author_sort | Lima, Maria R. |
collection | PubMed |
description | COVID-19 has severely impacted mental health in vulnerable demographics, in particular older adults, who face unprecedented isolation. Consequences, while globally severe, are acutely pronounced in low- and middle-income countries (LMICs) confronting pronounced gaps in resources and clinician accessibility. Social robots are well-recognized for their potential to support mental health, yet user compliance (i.e., trust) demands seamless affective human-robot interactions; natural ‘human-like’ conversations are required in simple, inexpensive, deployable platforms. We present the design, development, and pilot testing of a multimodal robotic framework fusing verbal (contextual speech) and nonverbal (facial expressions) social cues, aimed to improve engagement in human-robot interaction and ultimately facilitate mental health telemedicine during and beyond the COVID-19 pandemic. We report the design optimization of a hybrid face robot, which combines digital facial expressions based on mathematical affect space mapping with static 3D facial features. We further introduce a contextual virtual assistant with integrated cloud-based AI coupled to the robot’s facial representation of emotions, such that the robot adapts its emotional response to users’ speech in real-time. Experiments with healthy participants demonstrate emotion recognition exceeding 90% for happy, tired, sad, angry, surprised and stern/disgusted robotic emotions. When separated, stern and disgusted are occasionally transposed (70%+ accuracy overall) but are easily distinguishable from other emotions. A qualitative user experience analysis indicates overall enthusiastic and engaging reception to human-robot multimodal interaction with the new framework. The robot has been modified to enable clinical telemedicine for cognitive engagement with older adults and people with dementia (PwD) in LMICs. The mechanically simple and low-cost social robot has been deployed in pilot tests to support older individuals and PwD at the Schizophrenia Research Foundation (SCARF) in Chennai, India. A procedure for deployment addressing challenges in cultural acceptance, end-user acclimatization and resource allocation is further introduced. Results indicate strong promise to stimulate human-robot psychosocial interaction through the hybrid-face robotic system. Future work is targeting deployment for telemedicine to mitigate the mental health impact of COVID-19 on older adults and PwD in both LMICs and higher income regions. |
format | Online Article Text |
id | pubmed-8014955 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-80149552021-04-02 Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement Lima, Maria R. Wairagkar, Maitreyee Natarajan, Nirupama Vaitheswaran, Sridhar Vaidyanathan, Ravi Front Robot AI Robotics and AI COVID-19 has severely impacted mental health in vulnerable demographics, in particular older adults, who face unprecedented isolation. Consequences, while globally severe, are acutely pronounced in low- and middle-income countries (LMICs) confronting pronounced gaps in resources and clinician accessibility. Social robots are well-recognized for their potential to support mental health, yet user compliance (i.e., trust) demands seamless affective human-robot interactions; natural ‘human-like’ conversations are required in simple, inexpensive, deployable platforms. We present the design, development, and pilot testing of a multimodal robotic framework fusing verbal (contextual speech) and nonverbal (facial expressions) social cues, aimed to improve engagement in human-robot interaction and ultimately facilitate mental health telemedicine during and beyond the COVID-19 pandemic. We report the design optimization of a hybrid face robot, which combines digital facial expressions based on mathematical affect space mapping with static 3D facial features. We further introduce a contextual virtual assistant with integrated cloud-based AI coupled to the robot’s facial representation of emotions, such that the robot adapts its emotional response to users’ speech in real-time. Experiments with healthy participants demonstrate emotion recognition exceeding 90% for happy, tired, sad, angry, surprised and stern/disgusted robotic emotions. When separated, stern and disgusted are occasionally transposed (70%+ accuracy overall) but are easily distinguishable from other emotions. A qualitative user experience analysis indicates overall enthusiastic and engaging reception to human-robot multimodal interaction with the new framework. The robot has been modified to enable clinical telemedicine for cognitive engagement with older adults and people with dementia (PwD) in LMICs. The mechanically simple and low-cost social robot has been deployed in pilot tests to support older individuals and PwD at the Schizophrenia Research Foundation (SCARF) in Chennai, India. A procedure for deployment addressing challenges in cultural acceptance, end-user acclimatization and resource allocation is further introduced. Results indicate strong promise to stimulate human-robot psychosocial interaction through the hybrid-face robotic system. Future work is targeting deployment for telemedicine to mitigate the mental health impact of COVID-19 on older adults and PwD in both LMICs and higher income regions. Frontiers Media S.A. 2021-03-18 /pmc/articles/PMC8014955/ /pubmed/33816568 http://dx.doi.org/10.3389/frobt.2021.618866 Text en Copyright © 2021 Lima, Wairagkar, Natarajan, Vaitheswaran and Vaidyanathan. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Robotics and AI Lima, Maria R. Wairagkar, Maitreyee Natarajan, Nirupama Vaitheswaran, Sridhar Vaidyanathan, Ravi Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement |
title | Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement |
title_full | Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement |
title_fullStr | Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement |
title_full_unstemmed | Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement |
title_short | Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement |
title_sort | robotic telemedicine for mental health: a multimodal approach to improve human-robot engagement |
topic | Robotics and AI |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8014955/ https://www.ncbi.nlm.nih.gov/pubmed/33816568 http://dx.doi.org/10.3389/frobt.2021.618866 |
work_keys_str_mv | AT limamariar robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement AT wairagkarmaitreyee robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement AT natarajannirupama robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement AT vaitheswaransridhar robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement AT vaidyanathanravi robotictelemedicineformentalhealthamultimodalapproachtoimprovehumanrobotengagement |