Cargando…
A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension
Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10079236/ https://www.ncbi.nlm.nih.gov/pubmed/36947552 http://dx.doi.org/10.1371/journal.pbio.3002046 |
_version_ | 1785020690433310720 |
---|---|
author | Su, Yaqing MacGregor, Lucy J. Olasagasti, Itsaso Giraud, Anne-Lise |
author_facet | Su, Yaqing MacGregor, Lucy J. Olasagasti, Itsaso Giraud, Anne-Lise |
author_sort | Su, Yaqing |
collection | PubMed |
description | Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating among multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of the neurophysiological responses to speech compared to using lexical predictions generated by a neural network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model, we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input. |
format | Online Article Text |
id | pubmed-10079236 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-100792362023-04-07 A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension Su, Yaqing MacGregor, Lucy J. Olasagasti, Itsaso Giraud, Anne-Lise PLoS Biol Research Article Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating among multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of the neurophysiological responses to speech compared to using lexical predictions generated by a neural network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model, we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input. Public Library of Science 2023-03-22 /pmc/articles/PMC10079236/ /pubmed/36947552 http://dx.doi.org/10.1371/journal.pbio.3002046 Text en © 2023 Su et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Su, Yaqing MacGregor, Lucy J. Olasagasti, Itsaso Giraud, Anne-Lise A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension |
title | A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension |
title_full | A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension |
title_fullStr | A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension |
title_full_unstemmed | A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension |
title_short | A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension |
title_sort | deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10079236/ https://www.ncbi.nlm.nih.gov/pubmed/36947552 http://dx.doi.org/10.1371/journal.pbio.3002046 |
work_keys_str_mv | AT suyaqing adeephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension AT macgregorlucyj adeephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension AT olasagastiitsaso adeephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension AT giraudannelise adeephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension AT suyaqing deephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension AT macgregorlucyj deephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension AT olasagastiitsaso deephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension AT giraudannelise deephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension |