Cargando…

Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs

Content and temporal cues have been shown to interact during audio-visual (AV) speech identification. Typically, the most reliable unimodal cue is used more strongly to identify specific speech features; however, visual cues are only used if the AV stimuli are presented within a certain temporal win...

Descripción completa

Detalles Bibliográficos
Autores principales: ten Oever, Sanne, Sack, Alexander T., Wheat, Katherine L., Bien, Nina, van Atteveldt, Nienke
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2013
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3693065/
https://www.ncbi.nlm.nih.gov/pubmed/23805110
http://dx.doi.org/10.3389/fpsyg.2013.00331
_version_ 1782274698968563712
author ten Oever, Sanne
Sack, Alexander T.
Wheat, Katherine L.
Bien, Nina
van Atteveldt, Nienke
author_facet ten Oever, Sanne
Sack, Alexander T.
Wheat, Katherine L.
Bien, Nina
van Atteveldt, Nienke
author_sort ten Oever, Sanne
collection PubMed
description Content and temporal cues have been shown to interact during audio-visual (AV) speech identification. Typically, the most reliable unimodal cue is used more strongly to identify specific speech features; however, visual cues are only used if the AV stimuli are presented within a certain temporal window of integration (TWI). This suggests that temporal cues denote whether unimodal stimuli belong together, that is, whether they should be integrated. It is not known whether temporal cues also provide information about the identity of a syllable. Since spoken syllables have naturally varying AV onset asynchronies, we hypothesize that for suboptimal AV cues presented within the TWI, information about the natural AV onset differences can aid in speech identification. To test this, we presented low-intensity auditory syllables concurrently with visual speech signals, and varied the stimulus onset asynchronies (SOA) of the AV pair, while participants were instructed to identify the auditory syllables. We revealed that specific speech features (e.g., voicing) were identified by relying primarily on one modality (e.g., auditory). Additionally, we showed a wide window in which visual information influenced auditory perception, that seemed even wider for congruent stimulus pairs. Finally, we found a specific response pattern across the SOA range for syllables that were not reliably identified by the unimodal cues, which we explained as the result of the use of natural onset differences between AV speech signals. This indicates that temporal cues not only provide information about the temporal integration of AV stimuli, but additionally convey information about the identity of AV pairs. These results provide a detailed behavioral basis for further neuro-imaging and stimulation studies to unravel the neurofunctional mechanisms of the audio-visual-temporal interplay within speech perception.
format Online
Article
Text
id pubmed-3693065
institution National Center for Biotechnology Information
language English
publishDate 2013
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-36930652013-06-26 Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs ten Oever, Sanne Sack, Alexander T. Wheat, Katherine L. Bien, Nina van Atteveldt, Nienke Front Psychol Psychology Content and temporal cues have been shown to interact during audio-visual (AV) speech identification. Typically, the most reliable unimodal cue is used more strongly to identify specific speech features; however, visual cues are only used if the AV stimuli are presented within a certain temporal window of integration (TWI). This suggests that temporal cues denote whether unimodal stimuli belong together, that is, whether they should be integrated. It is not known whether temporal cues also provide information about the identity of a syllable. Since spoken syllables have naturally varying AV onset asynchronies, we hypothesize that for suboptimal AV cues presented within the TWI, information about the natural AV onset differences can aid in speech identification. To test this, we presented low-intensity auditory syllables concurrently with visual speech signals, and varied the stimulus onset asynchronies (SOA) of the AV pair, while participants were instructed to identify the auditory syllables. We revealed that specific speech features (e.g., voicing) were identified by relying primarily on one modality (e.g., auditory). Additionally, we showed a wide window in which visual information influenced auditory perception, that seemed even wider for congruent stimulus pairs. Finally, we found a specific response pattern across the SOA range for syllables that were not reliably identified by the unimodal cues, which we explained as the result of the use of natural onset differences between AV speech signals. This indicates that temporal cues not only provide information about the temporal integration of AV stimuli, but additionally convey information about the identity of AV pairs. These results provide a detailed behavioral basis for further neuro-imaging and stimulation studies to unravel the neurofunctional mechanisms of the audio-visual-temporal interplay within speech perception. Frontiers Media S.A. 2013-06-26 /pmc/articles/PMC3693065/ /pubmed/23805110 http://dx.doi.org/10.3389/fpsyg.2013.00331 Text en Copyright © 2013 ten Oever, Sack, Wheat, Bien and van Atteveldt. http://creativecommons.org/licenses/by/3.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.
spellingShingle Psychology
ten Oever, Sanne
Sack, Alexander T.
Wheat, Katherine L.
Bien, Nina
van Atteveldt, Nienke
Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs
title Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs
title_full Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs
title_fullStr Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs
title_full_unstemmed Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs
title_short Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs
title_sort audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs
topic Psychology
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3693065/
https://www.ncbi.nlm.nih.gov/pubmed/23805110
http://dx.doi.org/10.3389/fpsyg.2013.00331
work_keys_str_mv AT tenoeversanne audiovisualonsetdifferencesareusedtodeterminesyllableidentityforambiguousaudiovisualstimuluspairs
AT sackalexandert audiovisualonsetdifferencesareusedtodeterminesyllableidentityforambiguousaudiovisualstimuluspairs
AT wheatkatherinel audiovisualonsetdifferencesareusedtodeterminesyllableidentityforambiguousaudiovisualstimuluspairs
AT biennina audiovisualonsetdifferencesareusedtodeterminesyllableidentityforambiguousaudiovisualstimuluspairs
AT vanatteveldtnienke audiovisualonsetdifferencesareusedtodeterminesyllableidentityforambiguousaudiovisualstimuluspairs