Cargando…

Imaging Time Series of Eye Tracking Data to Classify Attentional States

It has been shown that conclusions about the human mental state can be drawn from eye gaze behavior by several previous studies. For this reason, eye tracking recordings are suitable as input data for attentional state classifiers. In current state-of-the-art studies, the extracted eye tracking feat...

Descripción completa

Detalles Bibliográficos
Autores principales: Vortmann, Lisa-Marie, Knychalla, Jannes, Annerer-Walcher, Sonja, Benedek, Mathias, Putze, Felix
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8193942/
https://www.ncbi.nlm.nih.gov/pubmed/34121994
http://dx.doi.org/10.3389/fnins.2021.664490
_version_ 1783706323769098240
author Vortmann, Lisa-Marie
Knychalla, Jannes
Annerer-Walcher, Sonja
Benedek, Mathias
Putze, Felix
author_facet Vortmann, Lisa-Marie
Knychalla, Jannes
Annerer-Walcher, Sonja
Benedek, Mathias
Putze, Felix
author_sort Vortmann, Lisa-Marie
collection PubMed
description It has been shown that conclusions about the human mental state can be drawn from eye gaze behavior by several previous studies. For this reason, eye tracking recordings are suitable as input data for attentional state classifiers. In current state-of-the-art studies, the extracted eye tracking feature set usually consists of descriptive statistics about specific eye movement characteristics (i.e., fixations, saccades, blinks, vergence, and pupil dilation). We suggest an Imaging Time Series approach for eye tracking data followed by classification using a convolutional neural net to improve the classification accuracy. We compared multiple algorithms that used the one-dimensional statistical summary feature set as input with two different implementations of the newly suggested method for three different data sets that target different aspects of attention. The results show that our two-dimensional image features with the convolutional neural net outperform the classical classifiers for most analyses, especially regarding generalization over participants and tasks. We conclude that current attentional state classifiers that are based on eye tracking can be optimized by adjusting the feature set while requiring less feature engineering and our future work will focus on a more detailed and suited investigation of this approach for other scenarios and data sets.
format Online
Article
Text
id pubmed-8193942
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-81939422021-06-12 Imaging Time Series of Eye Tracking Data to Classify Attentional States Vortmann, Lisa-Marie Knychalla, Jannes Annerer-Walcher, Sonja Benedek, Mathias Putze, Felix Front Neurosci Neuroscience It has been shown that conclusions about the human mental state can be drawn from eye gaze behavior by several previous studies. For this reason, eye tracking recordings are suitable as input data for attentional state classifiers. In current state-of-the-art studies, the extracted eye tracking feature set usually consists of descriptive statistics about specific eye movement characteristics (i.e., fixations, saccades, blinks, vergence, and pupil dilation). We suggest an Imaging Time Series approach for eye tracking data followed by classification using a convolutional neural net to improve the classification accuracy. We compared multiple algorithms that used the one-dimensional statistical summary feature set as input with two different implementations of the newly suggested method for three different data sets that target different aspects of attention. The results show that our two-dimensional image features with the convolutional neural net outperform the classical classifiers for most analyses, especially regarding generalization over participants and tasks. We conclude that current attentional state classifiers that are based on eye tracking can be optimized by adjusting the feature set while requiring less feature engineering and our future work will focus on a more detailed and suited investigation of this approach for other scenarios and data sets. Frontiers Media S.A. 2021-05-28 /pmc/articles/PMC8193942/ /pubmed/34121994 http://dx.doi.org/10.3389/fnins.2021.664490 Text en Copyright © 2021 Vortmann, Knychalla, Annerer-Walcher, Benedek and Putze. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Vortmann, Lisa-Marie
Knychalla, Jannes
Annerer-Walcher, Sonja
Benedek, Mathias
Putze, Felix
Imaging Time Series of Eye Tracking Data to Classify Attentional States
title Imaging Time Series of Eye Tracking Data to Classify Attentional States
title_full Imaging Time Series of Eye Tracking Data to Classify Attentional States
title_fullStr Imaging Time Series of Eye Tracking Data to Classify Attentional States
title_full_unstemmed Imaging Time Series of Eye Tracking Data to Classify Attentional States
title_short Imaging Time Series of Eye Tracking Data to Classify Attentional States
title_sort imaging time series of eye tracking data to classify attentional states
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8193942/
https://www.ncbi.nlm.nih.gov/pubmed/34121994
http://dx.doi.org/10.3389/fnins.2021.664490
work_keys_str_mv AT vortmannlisamarie imagingtimeseriesofeyetrackingdatatoclassifyattentionalstates
AT knychallajannes imagingtimeseriesofeyetrackingdatatoclassifyattentionalstates
AT annererwalchersonja imagingtimeseriesofeyetrackingdatatoclassifyattentionalstates
AT benedekmathias imagingtimeseriesofeyetrackingdatatoclassifyattentionalstates
AT putzefelix imagingtimeseriesofeyetrackingdatatoclassifyattentionalstates