Cargando…
Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models
The dramatic impact of the COVID-19 pandemic has resulted in the closure of physical classrooms and teaching methods being shifted to the online medium.To make the online learning environment more interactive, just like traditional offline classrooms, it is essential to ensure the proper engagement...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Springer US
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9461440/ https://www.ncbi.nlm.nih.gov/pubmed/36105662 http://dx.doi.org/10.1007/s11042-022-13558-9 |
_version_ | 1784786972914483200 |
---|---|
author | Gupta, Swadha Kumar, Parteek Tekchandani, Raj Kumar |
author_facet | Gupta, Swadha Kumar, Parteek Tekchandani, Raj Kumar |
author_sort | Gupta, Swadha |
collection | PubMed |
description | The dramatic impact of the COVID-19 pandemic has resulted in the closure of physical classrooms and teaching methods being shifted to the online medium.To make the online learning environment more interactive, just like traditional offline classrooms, it is essential to ensure the proper engagement of students during online learning sessions.This paper proposes a deep learning-based approach using facial emotions to detect the real-time engagement of online learners. This is done by analysing the students’ facial expressions to classify their emotions throughout the online learning session. The facial emotion recognition information is used to calculate the engagement index (EI) to predict two engagement states “Engaged” and “Disengaged”. Different deep learning models such as Inception-V3, VGG19 and ResNet-50 are evaluated and compared to get the best predictive classification model for real-time engagement detection. Varied benchmarked datasets such as FER-2013, CK+ and RAF-DB are used to gauge the overall performance and accuracy of the proposed system. Experimental results showed that the proposed system achieves an accuracy of 89.11%, 90.14% and 92.32% for Inception-V3, VGG19 and ResNet-50, respectively, on benchmarked datasets and our own created dataset. ResNet-50 outperforms all others with an accuracy of 92.3% for facial emotions classification in real-time learning scenarios. |
format | Online Article Text |
id | pubmed-9461440 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Springer US |
record_format | MEDLINE/PubMed |
spelling | pubmed-94614402022-09-10 Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models Gupta, Swadha Kumar, Parteek Tekchandani, Raj Kumar Multimed Tools Appl 1226: Deep-Patterns Emotion Recognition in the Wild The dramatic impact of the COVID-19 pandemic has resulted in the closure of physical classrooms and teaching methods being shifted to the online medium.To make the online learning environment more interactive, just like traditional offline classrooms, it is essential to ensure the proper engagement of students during online learning sessions.This paper proposes a deep learning-based approach using facial emotions to detect the real-time engagement of online learners. This is done by analysing the students’ facial expressions to classify their emotions throughout the online learning session. The facial emotion recognition information is used to calculate the engagement index (EI) to predict two engagement states “Engaged” and “Disengaged”. Different deep learning models such as Inception-V3, VGG19 and ResNet-50 are evaluated and compared to get the best predictive classification model for real-time engagement detection. Varied benchmarked datasets such as FER-2013, CK+ and RAF-DB are used to gauge the overall performance and accuracy of the proposed system. Experimental results showed that the proposed system achieves an accuracy of 89.11%, 90.14% and 92.32% for Inception-V3, VGG19 and ResNet-50, respectively, on benchmarked datasets and our own created dataset. ResNet-50 outperforms all others with an accuracy of 92.3% for facial emotions classification in real-time learning scenarios. Springer US 2022-09-09 2023 /pmc/articles/PMC9461440/ /pubmed/36105662 http://dx.doi.org/10.1007/s11042-022-13558-9 Text en © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022, Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic. |
spellingShingle | 1226: Deep-Patterns Emotion Recognition in the Wild Gupta, Swadha Kumar, Parteek Tekchandani, Raj Kumar Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models |
title | Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models |
title_full | Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models |
title_fullStr | Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models |
title_full_unstemmed | Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models |
title_short | Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models |
title_sort | facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models |
topic | 1226: Deep-Patterns Emotion Recognition in the Wild |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9461440/ https://www.ncbi.nlm.nih.gov/pubmed/36105662 http://dx.doi.org/10.1007/s11042-022-13558-9 |
work_keys_str_mv | AT guptaswadha facialemotionrecognitionbasedrealtimelearnerengagementdetectionsysteminonlinelearningcontextusingdeeplearningmodels AT kumarparteek facialemotionrecognitionbasedrealtimelearnerengagementdetectionsysteminonlinelearningcontextusingdeeplearningmodels AT tekchandanirajkumar facialemotionrecognitionbasedrealtimelearnerengagementdetectionsysteminonlinelearningcontextusingdeeplearningmodels |