Cargando…
Localizing category-related information in speech with multi-scale analyses
Measurements of the physical outputs of speech—vocal tract geometry and acoustic energy—are high-dimensional, but linguistic theories posit a low-dimensional set of categories such as phonemes and phrase types. How can it be determined when and where in high-dimensional articulatory and acoustic sig...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8486085/ https://www.ncbi.nlm.nih.gov/pubmed/34597350 http://dx.doi.org/10.1371/journal.pone.0258178 |
_version_ | 1784577668006543360 |
---|---|
author | Tilsen, Sam Kim, Seung-Eun Wang, Claire |
author_facet | Tilsen, Sam Kim, Seung-Eun Wang, Claire |
author_sort | Tilsen, Sam |
collection | PubMed |
description | Measurements of the physical outputs of speech—vocal tract geometry and acoustic energy—are high-dimensional, but linguistic theories posit a low-dimensional set of categories such as phonemes and phrase types. How can it be determined when and where in high-dimensional articulatory and acoustic signals there is information related to theoretical categories? For a variety of reasons, it is problematic to directly quantify mutual information between hypothesized categories and signals. To address this issue, a multi-scale analysis method is proposed for localizing category-related information in an ensemble of speech signals using machine learning algorithms. By analyzing how classification accuracy on unseen data varies as the temporal extent of training input is systematically restricted, inferences can be drawn regarding the temporal distribution of category-related information. The method can also be used to investigate redundancy between subsets of signal dimensions. Two types of theoretical categories are examined in this paper: phonemic/gestural categories and syntactic relative clause categories. Moreover, two different machine learning algorithms were examined: linear discriminant analysis and neural networks with long short-term memory units. Both algorithms detected category-related information earlier and later in signals than would be expected given standard theoretical assumptions about when linguistic categories should influence speech. The neural network algorithm was able to identify category-related information to a greater extent than the discriminant analyses. |
format | Online Article Text |
id | pubmed-8486085 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-84860852021-10-02 Localizing category-related information in speech with multi-scale analyses Tilsen, Sam Kim, Seung-Eun Wang, Claire PLoS One Research Article Measurements of the physical outputs of speech—vocal tract geometry and acoustic energy—are high-dimensional, but linguistic theories posit a low-dimensional set of categories such as phonemes and phrase types. How can it be determined when and where in high-dimensional articulatory and acoustic signals there is information related to theoretical categories? For a variety of reasons, it is problematic to directly quantify mutual information between hypothesized categories and signals. To address this issue, a multi-scale analysis method is proposed for localizing category-related information in an ensemble of speech signals using machine learning algorithms. By analyzing how classification accuracy on unseen data varies as the temporal extent of training input is systematically restricted, inferences can be drawn regarding the temporal distribution of category-related information. The method can also be used to investigate redundancy between subsets of signal dimensions. Two types of theoretical categories are examined in this paper: phonemic/gestural categories and syntactic relative clause categories. Moreover, two different machine learning algorithms were examined: linear discriminant analysis and neural networks with long short-term memory units. Both algorithms detected category-related information earlier and later in signals than would be expected given standard theoretical assumptions about when linguistic categories should influence speech. The neural network algorithm was able to identify category-related information to a greater extent than the discriminant analyses. Public Library of Science 2021-10-01 /pmc/articles/PMC8486085/ /pubmed/34597350 http://dx.doi.org/10.1371/journal.pone.0258178 Text en © 2021 Tilsen et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Tilsen, Sam Kim, Seung-Eun Wang, Claire Localizing category-related information in speech with multi-scale analyses |
title | Localizing category-related information in speech with multi-scale analyses |
title_full | Localizing category-related information in speech with multi-scale analyses |
title_fullStr | Localizing category-related information in speech with multi-scale analyses |
title_full_unstemmed | Localizing category-related information in speech with multi-scale analyses |
title_short | Localizing category-related information in speech with multi-scale analyses |
title_sort | localizing category-related information in speech with multi-scale analyses |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8486085/ https://www.ncbi.nlm.nih.gov/pubmed/34597350 http://dx.doi.org/10.1371/journal.pone.0258178 |
work_keys_str_mv | AT tilsensam localizingcategoryrelatedinformationinspeechwithmultiscaleanalyses AT kimseungeun localizingcategoryrelatedinformationinspeechwithmultiscaleanalyses AT wangclaire localizingcategoryrelatedinformationinspeechwithmultiscaleanalyses |