A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments

A multimodal human-computer interaction system is composed of the comprehensive usage of various input and output channels. For the information input, apart from the traditional keyboard typing, mouse clicking, screen touching, the latest speech and face recognition technology can be used. For the o...

Descripción completa

Detalles Bibliográficos
Autores principales: Jia, Jiyou, He, Yunfan, Le, Huixiao
Formato: Online Artículo Texto
Lenguaje:English
Publicado: 2020
Materias:
Article
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7366450/
http://dx.doi.org/10.1007/978-3-030-51968-1_1
_version_ 1783560224280412160
author Jia, Jiyou
He, Yunfan
Le, Huixiao
author_facet Jia, Jiyou
He, Yunfan
Le, Huixiao
author_sort Jia, Jiyou
collection PubMed
description A multimodal human-computer interaction system is composed of the comprehensive usage of various input and output channels. For the information input, apart from the traditional keyboard typing, mouse clicking, screen touching, the latest speech and face recognition technology can be used. For the output, the traditional screen display, the latest speech and facial expression synthesis and gesture generation can be used. After literature review of related works, this paper at first presents such a system, MMISE (Multimodal Interaction System for Education), about its architecture and working mechanism, POOOIIM (Pedagogical Objective Oriented Output, Input and Implementation Mechanism) illustrated with practical examples. Then this paper introduces this system’s pilot applications in the epidemic time of novel coronavirus in 2020.
format Online
Article
Text
id pubmed-7366450
institution National Center for Biotechnology Information
language English
publishDate 2020
record_format MEDLINE/PubMed
spelling pubmed-73664502020-07-17 A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments Jia, Jiyou He, Yunfan Le, Huixiao Blended Learning. Education in a Smart Learning Environment Article A multimodal human-computer interaction system is composed of the comprehensive usage of various input and output channels. For the information input, apart from the traditional keyboard typing, mouse clicking, screen touching, the latest speech and face recognition technology can be used. For the output, the traditional screen display, the latest speech and facial expression synthesis and gesture generation can be used. After literature review of related works, this paper at first presents such a system, MMISE (Multimodal Interaction System for Education), about its architecture and working mechanism, POOOIIM (Pedagogical Objective Oriented Output, Input and Implementation Mechanism) illustrated with practical examples. Then this paper introduces this system’s pilot applications in the epidemic time of novel coronavirus in 2020. 2020-06-05 /pmc/articles/PMC7366450/ http://dx.doi.org/10.1007/978-3-030-51968-1_1 Text en © Springer Nature Switzerland AG 2020 This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.
spellingShingle Article
Jia, Jiyou
He, Yunfan
Le, Huixiao
A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments
title A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments
title_full A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments
title_fullStr A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments
title_full_unstemmed A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments
title_short A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments
title_sort multimodal human-computer interaction system and its application in smart learning environments
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7366450/
http://dx.doi.org/10.1007/978-3-030-51968-1_1
work_keys_str_mv AT jiajiyou amultimodalhumancomputerinteractionsystemanditsapplicationinsmartlearningenvironments
AT heyunfan amultimodalhumancomputerinteractionsystemanditsapplicationinsmartlearningenvironments
AT lehuixiao amultimodalhumancomputerinteractionsystemanditsapplicationinsmartlearningenvironments
AT jiajiyou multimodalhumancomputerinteractionsystemanditsapplicationinsmartlearningenvironments
AT heyunfan multimodalhumancomputerinteractionsystemanditsapplicationinsmartlearningenvironments
AT lehuixiao multimodalhumancomputerinteractionsystemanditsapplicationinsmartlearningenvironments

Ejemplares similares