Cargando…
Cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones
Audio-visual correlation is a common phenomenon in real life. In this article, aiming at analyzing the correlation between multiple colors and combined tones, we comprehensively used experimental methods and technologies such as experimental psychology methods, audio-visual information processing te...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9763593/ https://www.ncbi.nlm.nih.gov/pubmed/36562072 http://dx.doi.org/10.3389/fpsyg.2022.970219 |
_version_ | 1784853093787107328 |
---|---|
author | Wang, Shuang Liu, Jingyu Lan, Xuedan Hu, Qihang Jiang, Jian Zhang, Jingjing |
author_facet | Wang, Shuang Liu, Jingyu Lan, Xuedan Hu, Qihang Jiang, Jian Zhang, Jingjing |
author_sort | Wang, Shuang |
collection | PubMed |
description | Audio-visual correlation is a common phenomenon in real life. In this article, aiming at analyzing the correlation between multiple colors and combined tones, we comprehensively used experimental methods and technologies such as experimental psychology methods, audio-visual information processing technology, and machine learning algorithms to study the correlation mechanism between the multi-color perceptual attributes and the interval consonance attribute of musical sounds, so as to construct an audio-visual cross-modal matching models. Specifically, in the first, this article constructed the multi-color perceptual attribute dataset through the subjective evaluation experiment, namely “cold/warm,” “soft/hard,” “transparent/turbid,” “far/near,” “weak/strong,” pleasure, arousal, and dominance; and constructed the interval consonance attribute dataset based on calculating the audio objective parameters. Secondly, a subjective evaluation experiment of cross-modal matching was designed and carried out for analyzing the audio-visual correlation, so as to obtain the cross-modal matched and mismatched data between the audio-visual perceptual attributes. On this basis, through visual processing and correlation analysis of the matched and mismatched data, this article proved that there is a certain correlation between multicolor and combined tones from the perspective of perceptual attributes. Finally, this article used linear and non-linear machine learning algorithms to construct audio-visual cross-modal matching models, so as to realize the mutual prediction between the audio-visual perceptual attributes, and the highest prediction accuracy is up to 79.1%. The contributions of our research are: (1) The cross-modal matched and mismatched dataset can provide basic data support for audio-visual cross-modal research; (2) The constructed audio-visual cross-modal matching models can provide a theoretical basis for audio-visual interaction technology; (3) In addition, the research method of audio-visual cross-modal matching proposed in this article can provide new research ideas for related research. |
format | Online Article Text |
id | pubmed-9763593 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-97635932022-12-21 Cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones Wang, Shuang Liu, Jingyu Lan, Xuedan Hu, Qihang Jiang, Jian Zhang, Jingjing Front Psychol Psychology Audio-visual correlation is a common phenomenon in real life. In this article, aiming at analyzing the correlation between multiple colors and combined tones, we comprehensively used experimental methods and technologies such as experimental psychology methods, audio-visual information processing technology, and machine learning algorithms to study the correlation mechanism between the multi-color perceptual attributes and the interval consonance attribute of musical sounds, so as to construct an audio-visual cross-modal matching models. Specifically, in the first, this article constructed the multi-color perceptual attribute dataset through the subjective evaluation experiment, namely “cold/warm,” “soft/hard,” “transparent/turbid,” “far/near,” “weak/strong,” pleasure, arousal, and dominance; and constructed the interval consonance attribute dataset based on calculating the audio objective parameters. Secondly, a subjective evaluation experiment of cross-modal matching was designed and carried out for analyzing the audio-visual correlation, so as to obtain the cross-modal matched and mismatched data between the audio-visual perceptual attributes. On this basis, through visual processing and correlation analysis of the matched and mismatched data, this article proved that there is a certain correlation between multicolor and combined tones from the perspective of perceptual attributes. Finally, this article used linear and non-linear machine learning algorithms to construct audio-visual cross-modal matching models, so as to realize the mutual prediction between the audio-visual perceptual attributes, and the highest prediction accuracy is up to 79.1%. The contributions of our research are: (1) The cross-modal matched and mismatched dataset can provide basic data support for audio-visual cross-modal research; (2) The constructed audio-visual cross-modal matching models can provide a theoretical basis for audio-visual interaction technology; (3) In addition, the research method of audio-visual cross-modal matching proposed in this article can provide new research ideas for related research. Frontiers Media S.A. 2022-12-06 /pmc/articles/PMC9763593/ /pubmed/36562072 http://dx.doi.org/10.3389/fpsyg.2022.970219 Text en Copyright © 2022 Wang, Liu, Lan, Hu, Jiang and Zhang. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Psychology Wang, Shuang Liu, Jingyu Lan, Xuedan Hu, Qihang Jiang, Jian Zhang, Jingjing Cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones |
title | Cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones |
title_full | Cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones |
title_fullStr | Cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones |
title_full_unstemmed | Cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones |
title_short | Cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones |
title_sort | cross-modal association analysis and matching model construction of perceptual attributes of multiple colors and combined tones |
topic | Psychology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9763593/ https://www.ncbi.nlm.nih.gov/pubmed/36562072 http://dx.doi.org/10.3389/fpsyg.2022.970219 |
work_keys_str_mv | AT wangshuang crossmodalassociationanalysisandmatchingmodelconstructionofperceptualattributesofmultiplecolorsandcombinedtones AT liujingyu crossmodalassociationanalysisandmatchingmodelconstructionofperceptualattributesofmultiplecolorsandcombinedtones AT lanxuedan crossmodalassociationanalysisandmatchingmodelconstructionofperceptualattributesofmultiplecolorsandcombinedtones AT huqihang crossmodalassociationanalysisandmatchingmodelconstructionofperceptualattributesofmultiplecolorsandcombinedtones AT jiangjian crossmodalassociationanalysisandmatchingmodelconstructionofperceptualattributesofmultiplecolorsandcombinedtones AT zhangjingjing crossmodalassociationanalysisandmatchingmodelconstructionofperceptualattributesofmultiplecolorsandcombinedtones |