Cargando…

A Multimodal Approach for Real Time Recognition of Engagement towards Adaptive Serious Games for Health

In this article, an unobtrusive and affordable sensor-based multimodal approach for real time recognition of engagement in serious games (SGs) for health is presented. This approach aims to achieve individualization in SGs that promote self-health management. The feasibility of the proposed approach...

Descripción completa

Detalles Bibliográficos
Autores principales: Mitsis, Konstantinos, Zarkogianni, Konstantia, Kalafatis, Eleftherios, Dalakleidi, Kalliopi, Jaafar, Amyn, Mourkousis, Georgios, Nikita, Konstantina S.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9002748/
https://www.ncbi.nlm.nih.gov/pubmed/35408088
http://dx.doi.org/10.3390/s22072472
_version_ 1784685964263686144
author Mitsis, Konstantinos
Zarkogianni, Konstantia
Kalafatis, Eleftherios
Dalakleidi, Kalliopi
Jaafar, Amyn
Mourkousis, Georgios
Nikita, Konstantina S.
author_facet Mitsis, Konstantinos
Zarkogianni, Konstantia
Kalafatis, Eleftherios
Dalakleidi, Kalliopi
Jaafar, Amyn
Mourkousis, Georgios
Nikita, Konstantina S.
author_sort Mitsis, Konstantinos
collection PubMed
description In this article, an unobtrusive and affordable sensor-based multimodal approach for real time recognition of engagement in serious games (SGs) for health is presented. This approach aims to achieve individualization in SGs that promote self-health management. The feasibility of the proposed approach was investigated by designing and implementing an experimental process focusing on real time recognition of engagement. Twenty-six participants were recruited and engaged in sessions with a SG that promotes food and nutrition literacy. Data were collected during play from a heart rate sensor, a smart chair, and in-game metrics. Perceived engagement, as an approximation to the ground truth, was annotated continuously by participants. An additional group of six participants were recruited for smart chair calibration purposes. The analysis was conducted in two directions, firstly investigating associations between identified sitting postures and perceived engagement, and secondly evaluating the predictive capacity of features extracted from the multitude of sources towards the ground truth. The results demonstrate significant associations and predictive capacity from all investigated sources, with a multimodal feature combination displaying superiority over unimodal features. These results advocate for the feasibility of real time recognition of engagement in adaptive serious games for health by using the presented approach.
format Online
Article
Text
id pubmed-9002748
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-90027482022-04-13 A Multimodal Approach for Real Time Recognition of Engagement towards Adaptive Serious Games for Health Mitsis, Konstantinos Zarkogianni, Konstantia Kalafatis, Eleftherios Dalakleidi, Kalliopi Jaafar, Amyn Mourkousis, Georgios Nikita, Konstantina S. Sensors (Basel) Article In this article, an unobtrusive and affordable sensor-based multimodal approach for real time recognition of engagement in serious games (SGs) for health is presented. This approach aims to achieve individualization in SGs that promote self-health management. The feasibility of the proposed approach was investigated by designing and implementing an experimental process focusing on real time recognition of engagement. Twenty-six participants were recruited and engaged in sessions with a SG that promotes food and nutrition literacy. Data were collected during play from a heart rate sensor, a smart chair, and in-game metrics. Perceived engagement, as an approximation to the ground truth, was annotated continuously by participants. An additional group of six participants were recruited for smart chair calibration purposes. The analysis was conducted in two directions, firstly investigating associations between identified sitting postures and perceived engagement, and secondly evaluating the predictive capacity of features extracted from the multitude of sources towards the ground truth. The results demonstrate significant associations and predictive capacity from all investigated sources, with a multimodal feature combination displaying superiority over unimodal features. These results advocate for the feasibility of real time recognition of engagement in adaptive serious games for health by using the presented approach. MDPI 2022-03-23 /pmc/articles/PMC9002748/ /pubmed/35408088 http://dx.doi.org/10.3390/s22072472 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Mitsis, Konstantinos
Zarkogianni, Konstantia
Kalafatis, Eleftherios
Dalakleidi, Kalliopi
Jaafar, Amyn
Mourkousis, Georgios
Nikita, Konstantina S.
A Multimodal Approach for Real Time Recognition of Engagement towards Adaptive Serious Games for Health
title A Multimodal Approach for Real Time Recognition of Engagement towards Adaptive Serious Games for Health
title_full A Multimodal Approach for Real Time Recognition of Engagement towards Adaptive Serious Games for Health
title_fullStr A Multimodal Approach for Real Time Recognition of Engagement towards Adaptive Serious Games for Health
title_full_unstemmed A Multimodal Approach for Real Time Recognition of Engagement towards Adaptive Serious Games for Health
title_short A Multimodal Approach for Real Time Recognition of Engagement towards Adaptive Serious Games for Health
title_sort multimodal approach for real time recognition of engagement towards adaptive serious games for health
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9002748/
https://www.ncbi.nlm.nih.gov/pubmed/35408088
http://dx.doi.org/10.3390/s22072472
work_keys_str_mv AT mitsiskonstantinos amultimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT zarkogiannikonstantia amultimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT kalafatiseleftherios amultimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT dalakleidikalliopi amultimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT jaafaramyn amultimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT mourkousisgeorgios amultimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT nikitakonstantinas amultimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT mitsiskonstantinos multimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT zarkogiannikonstantia multimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT kalafatiseleftherios multimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT dalakleidikalliopi multimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT jaafaramyn multimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT mourkousisgeorgios multimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth
AT nikitakonstantinas multimodalapproachforrealtimerecognitionofengagementtowardsadaptiveseriousgamesforhealth