Cargando…
MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization
In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2015
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4619362/ https://www.ncbi.nlm.nih.gov/pubmed/26489021 http://dx.doi.org/10.1371/journal.pone.0140732 |
_version_ | 1782397084973924352 |
---|---|
author | Mangin, Olivier Filliat, David ten Bosch, Louis Oudeyer, Pierre-Yves |
author_facet | Mangin, Olivier Filliat, David ten Bosch, Louis Oudeyer, Pierre-Yves |
author_sort | Mangin, Olivier |
collection | PubMed |
description | In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We propose this computational model as an answer to the question of how some class of concepts can be learnt. In addition, the model provides a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to reduce the ambiguity of learnt concepts as well as to communicate about them through speech. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts and associated experimental data to reproduce the experiments are publicly available. |
format | Online Article Text |
id | pubmed-4619362 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2015 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-46193622015-10-29 MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization Mangin, Olivier Filliat, David ten Bosch, Louis Oudeyer, Pierre-Yves PLoS One Research Article In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We propose this computational model as an answer to the question of how some class of concepts can be learnt. In addition, the model provides a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to reduce the ambiguity of learnt concepts as well as to communicate about them through speech. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts and associated experimental data to reproduce the experiments are publicly available. Public Library of Science 2015-10-21 /pmc/articles/PMC4619362/ /pubmed/26489021 http://dx.doi.org/10.1371/journal.pone.0140732 Text en © 2015 Mangin et al http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly credited. |
spellingShingle | Research Article Mangin, Olivier Filliat, David ten Bosch, Louis Oudeyer, Pierre-Yves MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization |
title | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization |
title_full | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization |
title_fullStr | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization |
title_full_unstemmed | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization |
title_short | MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization |
title_sort | mca-nmf: multimodal concept acquisition with non-negative matrix factorization |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4619362/ https://www.ncbi.nlm.nih.gov/pubmed/26489021 http://dx.doi.org/10.1371/journal.pone.0140732 |
work_keys_str_mv | AT manginolivier mcanmfmultimodalconceptacquisitionwithnonnegativematrixfactorization AT filliatdavid mcanmfmultimodalconceptacquisitionwithnonnegativematrixfactorization AT tenboschlouis mcanmfmultimodalconceptacquisitionwithnonnegativematrixfactorization AT oudeyerpierreyves mcanmfmultimodalconceptacquisitionwithnonnegativematrixfactorization |