Cargando…
A Pseudo-Value Approach to Analyze the Semantic Similarity of the Speech of Children With and Without Autism Spectrum Disorder
Conversational impairments are well known among people with autism spectrum disorder (ASD), but their measurement requires time-consuming manual annotation of language samples. Natural language processing (NLP) has shown promise in identifying semantic difficulties when compared to clinician-annotat...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8335559/ https://www.ncbi.nlm.nih.gov/pubmed/34366986 http://dx.doi.org/10.3389/fpsyg.2021.668344 |
_version_ | 1783733135022751744 |
---|---|
author | Adams, Joel R. Salem, Alexandra C. MacFarlane, Heather Ingham, Rosemary Bedrick, Steven D. Fombonne, Eric Dolata, Jill K. Hill, Alison Presmanes van Santen, Jan |
author_facet | Adams, Joel R. Salem, Alexandra C. MacFarlane, Heather Ingham, Rosemary Bedrick, Steven D. Fombonne, Eric Dolata, Jill K. Hill, Alison Presmanes van Santen, Jan |
author_sort | Adams, Joel R. |
collection | PubMed |
description | Conversational impairments are well known among people with autism spectrum disorder (ASD), but their measurement requires time-consuming manual annotation of language samples. Natural language processing (NLP) has shown promise in identifying semantic difficulties when compared to clinician-annotated reference transcripts. Our goal was to develop a novel measure of lexico-semantic similarity – based on recent work in natural language processing (NLP) and recent applications of pseudo-value analysis – which could be applied to transcripts of children’s conversational language, without recourse to some ground-truth reference document. We hypothesized that: (a) semantic coherence, as measured by this method, would discriminate between children with and without ASD and (b) more variability would be found in the group with ASD. We used data from 70 4- to 8-year-old males with ASD (N = 38) or typically developing (TD; N = 32) enrolled in a language study. Participants were administered a battery of standardized diagnostic tests, including the Autism Diagnostic Observation Schedule (ADOS). ADOS was recorded and transcribed, and we analyzed children’s language output during the conversation/interview ADOS tasks. Transcripts were converted to vectors via a word2vec model trained on the Google News Corpus. Pairwise similarity across all subjects and a sample grand mean were calculated. Using a leave-one-out algorithm, a pseudo-value, detailed below, representing each subject’s contribution to the grand mean was generated. Means of pseudo-values were compared between the two groups. Analyses were co-varied for nonverbal IQ, mean length of utterance, and number of distinct word roots (NDR). Statistically significant differences were observed in means of pseudo-values between TD and ASD groups (p = 0.007). TD subjects had higher pseudo-value scores suggesting that similarity scores of TD subjects were more similar to the overall group mean. Variance of pseudo-values was greater in the ASD group. Nonverbal IQ, mean length of utterance, or NDR did not account for between group differences. The findings suggest that our pseudo-value-based method can be effectively used to identify specific semantic difficulties that characterize children with ASD without requiring a reference transcript. |
format | Online Article Text |
id | pubmed-8335559 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-83355592021-08-05 A Pseudo-Value Approach to Analyze the Semantic Similarity of the Speech of Children With and Without Autism Spectrum Disorder Adams, Joel R. Salem, Alexandra C. MacFarlane, Heather Ingham, Rosemary Bedrick, Steven D. Fombonne, Eric Dolata, Jill K. Hill, Alison Presmanes van Santen, Jan Front Psychol Psychology Conversational impairments are well known among people with autism spectrum disorder (ASD), but their measurement requires time-consuming manual annotation of language samples. Natural language processing (NLP) has shown promise in identifying semantic difficulties when compared to clinician-annotated reference transcripts. Our goal was to develop a novel measure of lexico-semantic similarity – based on recent work in natural language processing (NLP) and recent applications of pseudo-value analysis – which could be applied to transcripts of children’s conversational language, without recourse to some ground-truth reference document. We hypothesized that: (a) semantic coherence, as measured by this method, would discriminate between children with and without ASD and (b) more variability would be found in the group with ASD. We used data from 70 4- to 8-year-old males with ASD (N = 38) or typically developing (TD; N = 32) enrolled in a language study. Participants were administered a battery of standardized diagnostic tests, including the Autism Diagnostic Observation Schedule (ADOS). ADOS was recorded and transcribed, and we analyzed children’s language output during the conversation/interview ADOS tasks. Transcripts were converted to vectors via a word2vec model trained on the Google News Corpus. Pairwise similarity across all subjects and a sample grand mean were calculated. Using a leave-one-out algorithm, a pseudo-value, detailed below, representing each subject’s contribution to the grand mean was generated. Means of pseudo-values were compared between the two groups. Analyses were co-varied for nonverbal IQ, mean length of utterance, and number of distinct word roots (NDR). Statistically significant differences were observed in means of pseudo-values between TD and ASD groups (p = 0.007). TD subjects had higher pseudo-value scores suggesting that similarity scores of TD subjects were more similar to the overall group mean. Variance of pseudo-values was greater in the ASD group. Nonverbal IQ, mean length of utterance, or NDR did not account for between group differences. The findings suggest that our pseudo-value-based method can be effectively used to identify specific semantic difficulties that characterize children with ASD without requiring a reference transcript. Frontiers Media S.A. 2021-07-21 /pmc/articles/PMC8335559/ /pubmed/34366986 http://dx.doi.org/10.3389/fpsyg.2021.668344 Text en Copyright © 2021 Adams, Salem, MacFarlane, Ingham, Bedrick, Fombonne, Dolata, Hill and van Santen. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Psychology Adams, Joel R. Salem, Alexandra C. MacFarlane, Heather Ingham, Rosemary Bedrick, Steven D. Fombonne, Eric Dolata, Jill K. Hill, Alison Presmanes van Santen, Jan A Pseudo-Value Approach to Analyze the Semantic Similarity of the Speech of Children With and Without Autism Spectrum Disorder |
title | A Pseudo-Value Approach to Analyze the Semantic Similarity of the Speech of Children With and Without Autism Spectrum Disorder |
title_full | A Pseudo-Value Approach to Analyze the Semantic Similarity of the Speech of Children With and Without Autism Spectrum Disorder |
title_fullStr | A Pseudo-Value Approach to Analyze the Semantic Similarity of the Speech of Children With and Without Autism Spectrum Disorder |
title_full_unstemmed | A Pseudo-Value Approach to Analyze the Semantic Similarity of the Speech of Children With and Without Autism Spectrum Disorder |
title_short | A Pseudo-Value Approach to Analyze the Semantic Similarity of the Speech of Children With and Without Autism Spectrum Disorder |
title_sort | pseudo-value approach to analyze the semantic similarity of the speech of children with and without autism spectrum disorder |
topic | Psychology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8335559/ https://www.ncbi.nlm.nih.gov/pubmed/34366986 http://dx.doi.org/10.3389/fpsyg.2021.668344 |
work_keys_str_mv | AT adamsjoelr apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT salemalexandrac apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT macfarlaneheather apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT inghamrosemary apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT bedrickstevend apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT fombonneeric apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT dolatajillk apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT hillalisonpresmanes apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT vansantenjan apseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT adamsjoelr pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT salemalexandrac pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT macfarlaneheather pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT inghamrosemary pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT bedrickstevend pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT fombonneeric pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT dolatajillk pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT hillalisonpresmanes pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder AT vansantenjan pseudovalueapproachtoanalyzethesemanticsimilarityofthespeechofchildrenwithandwithoutautismspectrumdisorder |