Cargando…
Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images
Gaze is an excellent indicator and has utility in that it can express interest or intention and the condition of an object. Recent deep-learning methods are mainly appearance-based methods that estimate gaze based on a simple regression from entire face and eye images. However, sometimes, this metho...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9183137/ https://www.ncbi.nlm.nih.gov/pubmed/35684647 http://dx.doi.org/10.3390/s22114026 |
_version_ | 1784724215949164544 |
---|---|
author | Oh, Jaekwang Lee, Youngkeun Yoo, Jisang Kwon, Soonchul |
author_facet | Oh, Jaekwang Lee, Youngkeun Yoo, Jisang Kwon, Soonchul |
author_sort | Oh, Jaekwang |
collection | PubMed |
description | Gaze is an excellent indicator and has utility in that it can express interest or intention and the condition of an object. Recent deep-learning methods are mainly appearance-based methods that estimate gaze based on a simple regression from entire face and eye images. However, sometimes, this method does not give satisfactory results for gaze estimations in low-resolution and noisy images obtained in unconstrained real-world settings (e.g., places with severe lighting changes). In this study, we propose a method that estimates gaze by detecting eye region landmarks through a single eye image; and this approach is shown to be competitive with recent appearance-based methods. Our approach acquires rich information by extracting more landmarks and including iris and eye edges, similar to the existing feature-based methods. To acquire strong features even at low resolutions, we used the HRNet backbone network to learn representations of images at various resolutions. Furthermore, we used the self-attention module CBAM to obtain a refined feature map with better spatial information, which enhanced the robustness to noisy inputs, thereby yielding a performance of a 3.18% landmark localization error, a 4% improvement over the existing error and A large number of landmarks were acquired and used as inputs for a lightweight neural network to estimate the gaze. We conducted a within-datasets evaluation on the MPIIGaze, which was obtained in a natural environment and achieved a state-of-the-art performance of 4.32 degrees, a 6% improvement over the existing performance. |
format | Online Article Text |
id | pubmed-9183137 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-91831372022-06-10 Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images Oh, Jaekwang Lee, Youngkeun Yoo, Jisang Kwon, Soonchul Sensors (Basel) Article Gaze is an excellent indicator and has utility in that it can express interest or intention and the condition of an object. Recent deep-learning methods are mainly appearance-based methods that estimate gaze based on a simple regression from entire face and eye images. However, sometimes, this method does not give satisfactory results for gaze estimations in low-resolution and noisy images obtained in unconstrained real-world settings (e.g., places with severe lighting changes). In this study, we propose a method that estimates gaze by detecting eye region landmarks through a single eye image; and this approach is shown to be competitive with recent appearance-based methods. Our approach acquires rich information by extracting more landmarks and including iris and eye edges, similar to the existing feature-based methods. To acquire strong features even at low resolutions, we used the HRNet backbone network to learn representations of images at various resolutions. Furthermore, we used the self-attention module CBAM to obtain a refined feature map with better spatial information, which enhanced the robustness to noisy inputs, thereby yielding a performance of a 3.18% landmark localization error, a 4% improvement over the existing error and A large number of landmarks were acquired and used as inputs for a lightweight neural network to estimate the gaze. We conducted a within-datasets evaluation on the MPIIGaze, which was obtained in a natural environment and achieved a state-of-the-art performance of 4.32 degrees, a 6% improvement over the existing performance. MDPI 2022-05-26 /pmc/articles/PMC9183137/ /pubmed/35684647 http://dx.doi.org/10.3390/s22114026 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Oh, Jaekwang Lee, Youngkeun Yoo, Jisang Kwon, Soonchul Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images |
title | Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images |
title_full | Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images |
title_fullStr | Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images |
title_full_unstemmed | Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images |
title_short | Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images |
title_sort | improved feature-based gaze estimation using self-attention module and synthetic eye images |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9183137/ https://www.ncbi.nlm.nih.gov/pubmed/35684647 http://dx.doi.org/10.3390/s22114026 |
work_keys_str_mv | AT ohjaekwang improvedfeaturebasedgazeestimationusingselfattentionmoduleandsyntheticeyeimages AT leeyoungkeun improvedfeaturebasedgazeestimationusingselfattentionmoduleandsyntheticeyeimages AT yoojisang improvedfeaturebasedgazeestimationusingselfattentionmoduleandsyntheticeyeimages AT kwonsoonchul improvedfeaturebasedgazeestimationusingselfattentionmoduleandsyntheticeyeimages |