Cargando…

Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses

BACKGROUND: Several studies have shown that facial attention differs in children with autism. Measuring eye gaze and emotion recognition in children with autism is challenging, as standard clinical assessments must be delivered in clinical settings by a trained clinician. Wearable technologies may b...

Descripción completa

Detalles Bibliográficos
Autores principales: Nag, Anish, Haber, Nick, Voss, Catalin, Tamura, Serena, Daniels, Jena, Ma, Jeffrey, Chiang, Bryan, Ramachandran, Shasta, Schwartz, Jessey, Winograd, Terry, Feinstein, Carl, Wall, Dennis P
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7203617/
https://www.ncbi.nlm.nih.gov/pubmed/32319961
http://dx.doi.org/10.2196/13810
_version_ 1783529904342564864
author Nag, Anish
Haber, Nick
Voss, Catalin
Tamura, Serena
Daniels, Jena
Ma, Jeffrey
Chiang, Bryan
Ramachandran, Shasta
Schwartz, Jessey
Winograd, Terry
Feinstein, Carl
Wall, Dennis P
author_facet Nag, Anish
Haber, Nick
Voss, Catalin
Tamura, Serena
Daniels, Jena
Ma, Jeffrey
Chiang, Bryan
Ramachandran, Shasta
Schwartz, Jessey
Winograd, Terry
Feinstein, Carl
Wall, Dennis P
author_sort Nag, Anish
collection PubMed
description BACKGROUND: Several studies have shown that facial attention differs in children with autism. Measuring eye gaze and emotion recognition in children with autism is challenging, as standard clinical assessments must be delivered in clinical settings by a trained clinician. Wearable technologies may be able to bring eye gaze and emotion recognition into natural social interactions and settings. OBJECTIVE: This study aimed to test: (1) the feasibility of tracking gaze using wearable smart glasses during a facial expression recognition task and (2) the ability of these gaze-tracking data, together with facial expression recognition responses, to distinguish children with autism from neurotypical controls (NCs). METHODS: We compared the eye gaze and emotion recognition patterns of 16 children with autism spectrum disorder (ASD) and 17 children without ASD via wearable smart glasses fitted with a custom eye tracker. Children identified static facial expressions of images presented on a computer screen along with nonsocial distractors while wearing Google Glass and the eye tracker. Faces were presented in three trials, during one of which children received feedback in the form of the correct classification. We employed hybrid human-labeling and computer vision–enabled methods for pupil tracking and world–gaze translation calibration. We analyzed the impact of gaze and emotion recognition features in a prediction task aiming to distinguish children with ASD from NC participants. RESULTS: Gaze and emotion recognition patterns enabled the training of a classifier that distinguished ASD and NC groups. However, it was unable to significantly outperform other classifiers that used only age and gender features, suggesting that further work is necessary to disentangle these effects. CONCLUSIONS: Although wearable smart glasses show promise in identifying subtle differences in gaze tracking and emotion recognition patterns in children with and without ASD, the present form factor and data do not allow for these differences to be reliably exploited by machine learning systems. Resolving these challenges will be an important step toward continuous tracking of the ASD phenotype.
format Online
Article
Text
id pubmed-7203617
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-72036172020-05-08 Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses Nag, Anish Haber, Nick Voss, Catalin Tamura, Serena Daniels, Jena Ma, Jeffrey Chiang, Bryan Ramachandran, Shasta Schwartz, Jessey Winograd, Terry Feinstein, Carl Wall, Dennis P J Med Internet Res Original Paper BACKGROUND: Several studies have shown that facial attention differs in children with autism. Measuring eye gaze and emotion recognition in children with autism is challenging, as standard clinical assessments must be delivered in clinical settings by a trained clinician. Wearable technologies may be able to bring eye gaze and emotion recognition into natural social interactions and settings. OBJECTIVE: This study aimed to test: (1) the feasibility of tracking gaze using wearable smart glasses during a facial expression recognition task and (2) the ability of these gaze-tracking data, together with facial expression recognition responses, to distinguish children with autism from neurotypical controls (NCs). METHODS: We compared the eye gaze and emotion recognition patterns of 16 children with autism spectrum disorder (ASD) and 17 children without ASD via wearable smart glasses fitted with a custom eye tracker. Children identified static facial expressions of images presented on a computer screen along with nonsocial distractors while wearing Google Glass and the eye tracker. Faces were presented in three trials, during one of which children received feedback in the form of the correct classification. We employed hybrid human-labeling and computer vision–enabled methods for pupil tracking and world–gaze translation calibration. We analyzed the impact of gaze and emotion recognition features in a prediction task aiming to distinguish children with ASD from NC participants. RESULTS: Gaze and emotion recognition patterns enabled the training of a classifier that distinguished ASD and NC groups. However, it was unable to significantly outperform other classifiers that used only age and gender features, suggesting that further work is necessary to disentangle these effects. CONCLUSIONS: Although wearable smart glasses show promise in identifying subtle differences in gaze tracking and emotion recognition patterns in children with and without ASD, the present form factor and data do not allow for these differences to be reliably exploited by machine learning systems. Resolving these challenges will be an important step toward continuous tracking of the ASD phenotype. JMIR Publications 2020-04-22 /pmc/articles/PMC7203617/ /pubmed/32319961 http://dx.doi.org/10.2196/13810 Text en ©Anish Nag, Nick Haber, Catalin Voss, Serena Tamura, Jena Daniels, Jeffrey Ma, Bryan Chiang, Shasta Ramachandran, Jessey Schwartz, Terry Winograd, Carl Feinstein, Dennis P Wall. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 22.04.2020. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Nag, Anish
Haber, Nick
Voss, Catalin
Tamura, Serena
Daniels, Jena
Ma, Jeffrey
Chiang, Bryan
Ramachandran, Shasta
Schwartz, Jessey
Winograd, Terry
Feinstein, Carl
Wall, Dennis P
Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses
title Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses
title_full Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses
title_fullStr Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses
title_full_unstemmed Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses
title_short Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses
title_sort toward continuous social phenotyping: analyzing gaze patterns in an emotion recognition task for children with autism through wearable smart glasses
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7203617/
https://www.ncbi.nlm.nih.gov/pubmed/32319961
http://dx.doi.org/10.2196/13810
work_keys_str_mv AT naganish towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT habernick towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT vosscatalin towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT tamuraserena towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT danielsjena towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT majeffrey towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT chiangbryan towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT ramachandranshasta towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT schwartzjessey towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT winogradterry towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT feinsteincarl towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses
AT walldennisp towardcontinuoussocialphenotypinganalyzinggazepatternsinanemotionrecognitiontaskforchildrenwithautismthroughwearablesmartglasses