Cargando…

Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain

Two multimodal classification models aimed at enhancing object classification through the integration of semantically congruent unimodal stimuli are introduced. The feature-integrating model, inspired by multisensory integration in the subcortical superior colliculus, combines unimodal features whic...

Descripción completa

Detalles Bibliográficos
Autores principales: Amerineni, Rajesh, Gupta, Resh S., Gupta, Lalit
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6356735/
https://www.ncbi.nlm.nih.gov/pubmed/30609705
http://dx.doi.org/10.3390/brainsci9010003
_version_ 1783391621904072704
author Amerineni, Rajesh
Gupta, Resh S.
Gupta, Lalit
author_facet Amerineni, Rajesh
Gupta, Resh S.
Gupta, Lalit
author_sort Amerineni, Rajesh
collection PubMed
description Two multimodal classification models aimed at enhancing object classification through the integration of semantically congruent unimodal stimuli are introduced. The feature-integrating model, inspired by multisensory integration in the subcortical superior colliculus, combines unimodal features which are subsequently classified by a multimodal classifier. The decision-integrating model, inspired by integration in primary cortical areas, classifies unimodal stimuli independently using unimodal classifiers and classifies the combined decisions using a multimodal classifier. The multimodal classifier models are implemented using multilayer perceptrons and multivariate statistical classifiers. Experiments involving the classification of noisy and attenuated auditory and visual representations of ten digits are designed to demonstrate the properties of the multimodal classifiers and to compare the performances of multimodal and unimodal classifiers. The experimental results show that the multimodal classification systems exhibit an important aspect of the “inverse effectiveness principle” by yielding significantly higher classification accuracies when compared with those of the unimodal classifiers. Furthermore, the flexibility offered by the generalized models enables the simulations and evaluations of various combinations of multimodal stimuli and classifiers under varying uncertainty conditions.
format Online
Article
Text
id pubmed-6356735
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-63567352019-02-05 Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain Amerineni, Rajesh Gupta, Resh S. Gupta, Lalit Brain Sci Article Two multimodal classification models aimed at enhancing object classification through the integration of semantically congruent unimodal stimuli are introduced. The feature-integrating model, inspired by multisensory integration in the subcortical superior colliculus, combines unimodal features which are subsequently classified by a multimodal classifier. The decision-integrating model, inspired by integration in primary cortical areas, classifies unimodal stimuli independently using unimodal classifiers and classifies the combined decisions using a multimodal classifier. The multimodal classifier models are implemented using multilayer perceptrons and multivariate statistical classifiers. Experiments involving the classification of noisy and attenuated auditory and visual representations of ten digits are designed to demonstrate the properties of the multimodal classifiers and to compare the performances of multimodal and unimodal classifiers. The experimental results show that the multimodal classification systems exhibit an important aspect of the “inverse effectiveness principle” by yielding significantly higher classification accuracies when compared with those of the unimodal classifiers. Furthermore, the flexibility offered by the generalized models enables the simulations and evaluations of various combinations of multimodal stimuli and classifiers under varying uncertainty conditions. MDPI 2019-01-02 /pmc/articles/PMC6356735/ /pubmed/30609705 http://dx.doi.org/10.3390/brainsci9010003 Text en © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Amerineni, Rajesh
Gupta, Resh S.
Gupta, Lalit
Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain
title Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain
title_full Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain
title_fullStr Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain
title_full_unstemmed Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain
title_short Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain
title_sort multimodal object classification models inspired by multisensory integration in the brain
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6356735/
https://www.ncbi.nlm.nih.gov/pubmed/30609705
http://dx.doi.org/10.3390/brainsci9010003
work_keys_str_mv AT amerinenirajesh multimodalobjectclassificationmodelsinspiredbymultisensoryintegrationinthebrain
AT guptareshs multimodalobjectclassificationmodelsinspiredbymultisensoryintegrationinthebrain
AT guptalalit multimodalobjectclassificationmodelsinspiredbymultisensoryintegrationinthebrain