Cargando…

Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder

The recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognit...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhang, Yunkai, Tian, Yinghong, Wu, Pingyi, Chen, Dongfan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7827022/
https://www.ncbi.nlm.nih.gov/pubmed/33430118
http://dx.doi.org/10.3390/s21020411
_version_ 1783640660953268224
author Zhang, Yunkai
Tian, Yinghong
Wu, Pingyi
Chen, Dongfan
author_facet Zhang, Yunkai
Tian, Yinghong
Wu, Pingyi
Chen, Dongfan
author_sort Zhang, Yunkai
collection PubMed
description The recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognition process of stereotyped actions, a method based on skeleton data and Long Short-Term Memory (LSTM) is proposed in this paper. In the first stage of our method, the OpenPose algorithm is used to obtain the initial skeleton data from the video of ASD children. Furthermore, four denoising methods are proposed to eliminate the noise of the initial skeleton data. In the second stage, we track multiple ASD children in the same scene by matching distance between current skeletons and previous skeletons. In the last stage, the neural network based on LSTM is proposed to classify the ASD children’s actions. The performed experiments show that our proposed method is effective for ASD children’s action recognition. Compared to the previous traditional schemes, our scheme has higher accuracy and is almost non-invasive for ASD children.
format Online
Article
Text
id pubmed-7827022
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-78270222021-01-25 Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder Zhang, Yunkai Tian, Yinghong Wu, Pingyi Chen, Dongfan Sensors (Basel) Article The recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognition process of stereotyped actions, a method based on skeleton data and Long Short-Term Memory (LSTM) is proposed in this paper. In the first stage of our method, the OpenPose algorithm is used to obtain the initial skeleton data from the video of ASD children. Furthermore, four denoising methods are proposed to eliminate the noise of the initial skeleton data. In the second stage, we track multiple ASD children in the same scene by matching distance between current skeletons and previous skeletons. In the last stage, the neural network based on LSTM is proposed to classify the ASD children’s actions. The performed experiments show that our proposed method is effective for ASD children’s action recognition. Compared to the previous traditional schemes, our scheme has higher accuracy and is almost non-invasive for ASD children. MDPI 2021-01-08 /pmc/articles/PMC7827022/ /pubmed/33430118 http://dx.doi.org/10.3390/s21020411 Text en © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Zhang, Yunkai
Tian, Yinghong
Wu, Pingyi
Chen, Dongfan
Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder
title Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder
title_full Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder
title_fullStr Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder
title_full_unstemmed Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder
title_short Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder
title_sort application of skeleton data and long short-term memory in action recognition of children with autism spectrum disorder
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7827022/
https://www.ncbi.nlm.nih.gov/pubmed/33430118
http://dx.doi.org/10.3390/s21020411
work_keys_str_mv AT zhangyunkai applicationofskeletondataandlongshorttermmemoryinactionrecognitionofchildrenwithautismspectrumdisorder
AT tianyinghong applicationofskeletondataandlongshorttermmemoryinactionrecognitionofchildrenwithautismspectrumdisorder
AT wupingyi applicationofskeletondataandlongshorttermmemoryinactionrecognitionofchildrenwithautismspectrumdisorder
AT chendongfan applicationofskeletondataandlongshorttermmemoryinactionrecognitionofchildrenwithautismspectrumdisorder