Cargando…
Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain
Two multimodal classification models aimed at enhancing object classification through the integration of semantically congruent unimodal stimuli are introduced. The feature-integrating model, inspired by multisensory integration in the subcortical superior colliculus, combines unimodal features whic...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6356735/ https://www.ncbi.nlm.nih.gov/pubmed/30609705 http://dx.doi.org/10.3390/brainsci9010003 |
_version_ | 1783391621904072704 |
---|---|
author | Amerineni, Rajesh Gupta, Resh S. Gupta, Lalit |
author_facet | Amerineni, Rajesh Gupta, Resh S. Gupta, Lalit |
author_sort | Amerineni, Rajesh |
collection | PubMed |
description | Two multimodal classification models aimed at enhancing object classification through the integration of semantically congruent unimodal stimuli are introduced. The feature-integrating model, inspired by multisensory integration in the subcortical superior colliculus, combines unimodal features which are subsequently classified by a multimodal classifier. The decision-integrating model, inspired by integration in primary cortical areas, classifies unimodal stimuli independently using unimodal classifiers and classifies the combined decisions using a multimodal classifier. The multimodal classifier models are implemented using multilayer perceptrons and multivariate statistical classifiers. Experiments involving the classification of noisy and attenuated auditory and visual representations of ten digits are designed to demonstrate the properties of the multimodal classifiers and to compare the performances of multimodal and unimodal classifiers. The experimental results show that the multimodal classification systems exhibit an important aspect of the “inverse effectiveness principle” by yielding significantly higher classification accuracies when compared with those of the unimodal classifiers. Furthermore, the flexibility offered by the generalized models enables the simulations and evaluations of various combinations of multimodal stimuli and classifiers under varying uncertainty conditions. |
format | Online Article Text |
id | pubmed-6356735 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-63567352019-02-05 Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain Amerineni, Rajesh Gupta, Resh S. Gupta, Lalit Brain Sci Article Two multimodal classification models aimed at enhancing object classification through the integration of semantically congruent unimodal stimuli are introduced. The feature-integrating model, inspired by multisensory integration in the subcortical superior colliculus, combines unimodal features which are subsequently classified by a multimodal classifier. The decision-integrating model, inspired by integration in primary cortical areas, classifies unimodal stimuli independently using unimodal classifiers and classifies the combined decisions using a multimodal classifier. The multimodal classifier models are implemented using multilayer perceptrons and multivariate statistical classifiers. Experiments involving the classification of noisy and attenuated auditory and visual representations of ten digits are designed to demonstrate the properties of the multimodal classifiers and to compare the performances of multimodal and unimodal classifiers. The experimental results show that the multimodal classification systems exhibit an important aspect of the “inverse effectiveness principle” by yielding significantly higher classification accuracies when compared with those of the unimodal classifiers. Furthermore, the flexibility offered by the generalized models enables the simulations and evaluations of various combinations of multimodal stimuli and classifiers under varying uncertainty conditions. MDPI 2019-01-02 /pmc/articles/PMC6356735/ /pubmed/30609705 http://dx.doi.org/10.3390/brainsci9010003 Text en © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Amerineni, Rajesh Gupta, Resh S. Gupta, Lalit Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain |
title | Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain |
title_full | Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain |
title_fullStr | Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain |
title_full_unstemmed | Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain |
title_short | Multimodal Object Classification Models Inspired by Multisensory Integration in the Brain |
title_sort | multimodal object classification models inspired by multisensory integration in the brain |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6356735/ https://www.ncbi.nlm.nih.gov/pubmed/30609705 http://dx.doi.org/10.3390/brainsci9010003 |
work_keys_str_mv | AT amerinenirajesh multimodalobjectclassificationmodelsinspiredbymultisensoryintegrationinthebrain AT guptareshs multimodalobjectclassificationmodelsinspiredbymultisensoryintegrationinthebrain AT guptalalit multimodalobjectclassificationmodelsinspiredbymultisensoryintegrationinthebrain |