Cargando…

Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images

Gaze is an excellent indicator and has utility in that it can express interest or intention and the condition of an object. Recent deep-learning methods are mainly appearance-based methods that estimate gaze based on a simple regression from entire face and eye images. However, sometimes, this metho...

Descripción completa

Detalles Bibliográficos
Autores principales: Oh, Jaekwang, Lee, Youngkeun, Yoo, Jisang, Kwon, Soonchul
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9183137/
https://www.ncbi.nlm.nih.gov/pubmed/35684647
http://dx.doi.org/10.3390/s22114026
_version_ 1784724215949164544
author Oh, Jaekwang
Lee, Youngkeun
Yoo, Jisang
Kwon, Soonchul
author_facet Oh, Jaekwang
Lee, Youngkeun
Yoo, Jisang
Kwon, Soonchul
author_sort Oh, Jaekwang
collection PubMed
description Gaze is an excellent indicator and has utility in that it can express interest or intention and the condition of an object. Recent deep-learning methods are mainly appearance-based methods that estimate gaze based on a simple regression from entire face and eye images. However, sometimes, this method does not give satisfactory results for gaze estimations in low-resolution and noisy images obtained in unconstrained real-world settings (e.g., places with severe lighting changes). In this study, we propose a method that estimates gaze by detecting eye region landmarks through a single eye image; and this approach is shown to be competitive with recent appearance-based methods. Our approach acquires rich information by extracting more landmarks and including iris and eye edges, similar to the existing feature-based methods. To acquire strong features even at low resolutions, we used the HRNet backbone network to learn representations of images at various resolutions. Furthermore, we used the self-attention module CBAM to obtain a refined feature map with better spatial information, which enhanced the robustness to noisy inputs, thereby yielding a performance of a 3.18% landmark localization error, a 4% improvement over the existing error and A large number of landmarks were acquired and used as inputs for a lightweight neural network to estimate the gaze. We conducted a within-datasets evaluation on the MPIIGaze, which was obtained in a natural environment and achieved a state-of-the-art performance of 4.32 degrees, a 6% improvement over the existing performance.
format Online
Article
Text
id pubmed-9183137
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-91831372022-06-10 Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images Oh, Jaekwang Lee, Youngkeun Yoo, Jisang Kwon, Soonchul Sensors (Basel) Article Gaze is an excellent indicator and has utility in that it can express interest or intention and the condition of an object. Recent deep-learning methods are mainly appearance-based methods that estimate gaze based on a simple regression from entire face and eye images. However, sometimes, this method does not give satisfactory results for gaze estimations in low-resolution and noisy images obtained in unconstrained real-world settings (e.g., places with severe lighting changes). In this study, we propose a method that estimates gaze by detecting eye region landmarks through a single eye image; and this approach is shown to be competitive with recent appearance-based methods. Our approach acquires rich information by extracting more landmarks and including iris and eye edges, similar to the existing feature-based methods. To acquire strong features even at low resolutions, we used the HRNet backbone network to learn representations of images at various resolutions. Furthermore, we used the self-attention module CBAM to obtain a refined feature map with better spatial information, which enhanced the robustness to noisy inputs, thereby yielding a performance of a 3.18% landmark localization error, a 4% improvement over the existing error and A large number of landmarks were acquired and used as inputs for a lightweight neural network to estimate the gaze. We conducted a within-datasets evaluation on the MPIIGaze, which was obtained in a natural environment and achieved a state-of-the-art performance of 4.32 degrees, a 6% improvement over the existing performance. MDPI 2022-05-26 /pmc/articles/PMC9183137/ /pubmed/35684647 http://dx.doi.org/10.3390/s22114026 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Oh, Jaekwang
Lee, Youngkeun
Yoo, Jisang
Kwon, Soonchul
Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images
title Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images
title_full Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images
title_fullStr Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images
title_full_unstemmed Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images
title_short Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images
title_sort improved feature-based gaze estimation using self-attention module and synthetic eye images
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9183137/
https://www.ncbi.nlm.nih.gov/pubmed/35684647
http://dx.doi.org/10.3390/s22114026
work_keys_str_mv AT ohjaekwang improvedfeaturebasedgazeestimationusingselfattentionmoduleandsyntheticeyeimages
AT leeyoungkeun improvedfeaturebasedgazeestimationusingselfattentionmoduleandsyntheticeyeimages
AT yoojisang improvedfeaturebasedgazeestimationusingselfattentionmoduleandsyntheticeyeimages
AT kwonsoonchul improvedfeaturebasedgazeestimationusingselfattentionmoduleandsyntheticeyeimages