Cargando…

A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension

Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple...

Descripción completa

Detalles Bibliográficos
Autores principales: Su, Yaqing, MacGregor, Lucy J., Olasagasti, Itsaso, Giraud, Anne-Lise
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10079236/
https://www.ncbi.nlm.nih.gov/pubmed/36947552
http://dx.doi.org/10.1371/journal.pbio.3002046
_version_ 1785020690433310720
author Su, Yaqing
MacGregor, Lucy J.
Olasagasti, Itsaso
Giraud, Anne-Lise
author_facet Su, Yaqing
MacGregor, Lucy J.
Olasagasti, Itsaso
Giraud, Anne-Lise
author_sort Su, Yaqing
collection PubMed
description Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating among multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of the neurophysiological responses to speech compared to using lexical predictions generated by a neural network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model, we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input.
format Online
Article
Text
id pubmed-10079236
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-100792362023-04-07 A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension Su, Yaqing MacGregor, Lucy J. Olasagasti, Itsaso Giraud, Anne-Lise PLoS Biol Research Article Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating among multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of the neurophysiological responses to speech compared to using lexical predictions generated by a neural network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model, we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input. Public Library of Science 2023-03-22 /pmc/articles/PMC10079236/ /pubmed/36947552 http://dx.doi.org/10.1371/journal.pbio.3002046 Text en © 2023 Su et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Su, Yaqing
MacGregor, Lucy J.
Olasagasti, Itsaso
Giraud, Anne-Lise
A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension
title A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension
title_full A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension
title_fullStr A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension
title_full_unstemmed A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension
title_short A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension
title_sort deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10079236/
https://www.ncbi.nlm.nih.gov/pubmed/36947552
http://dx.doi.org/10.1371/journal.pbio.3002046
work_keys_str_mv AT suyaqing adeephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension
AT macgregorlucyj adeephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension
AT olasagastiitsaso adeephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension
AT giraudannelise adeephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension
AT suyaqing deephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension
AT macgregorlucyj deephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension
AT olasagastiitsaso deephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension
AT giraudannelise deephierarchyofpredictionsenablesonlinemeaningextractioninacomputationalmodelofhumanspeechcomprehension