Cargando…

Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks

The monitoring of vital signs and increasing patient comfort are cornerstones of modern neonatal intensive care. Commonly used monitoring methods are based on skin contact which can cause irritations and discomfort in preterm neonates. Therefore, non-contact approaches are the subject of current res...

Descripción completa

Detalles Bibliográficos
Autores principales: Gleichauf, Johanna, Hennemann, Lukas, Fahlbusch, Fabian B., Hofmann, Oliver, Niebler, Christine, Koelpin, Alexander
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10223875/
https://www.ncbi.nlm.nih.gov/pubmed/37430829
http://dx.doi.org/10.3390/s23104910
_version_ 1785050045360373760
author Gleichauf, Johanna
Hennemann, Lukas
Fahlbusch, Fabian B.
Hofmann, Oliver
Niebler, Christine
Koelpin, Alexander
author_facet Gleichauf, Johanna
Hennemann, Lukas
Fahlbusch, Fabian B.
Hofmann, Oliver
Niebler, Christine
Koelpin, Alexander
author_sort Gleichauf, Johanna
collection PubMed
description The monitoring of vital signs and increasing patient comfort are cornerstones of modern neonatal intensive care. Commonly used monitoring methods are based on skin contact which can cause irritations and discomfort in preterm neonates. Therefore, non-contact approaches are the subject of current research aiming to resolve this dichotomy. Robust neonatal face detection is essential for the reliable detection of heart rate, respiratory rate and body temperature. While solutions for adult face detection are established, the unique neonatal proportions require a tailored approach. Additionally, sufficient open-source data of neonates on the NICU is lacking. We set out to train neural networks with the thermal-RGB-fusion data of neonates. We propose a novel indirect fusion approach including the sensor fusion of a thermal and RGB camera based on a 3D time-of-flight (ToF) camera. Unlike other approaches, this method is tailored for close distances encountered in neonatal incubators. Two neural networks were used with the fusion data and compared to RGB and thermal networks. For the class “head” we reached average precision values of 0.9958 (RetinaNet) and 0.9455 (YOLOv3) for the fusion data. Compared with the literature, similar precision was achieved, but we are the first to train a neural network with fusion data of neonates. The advantage of this approach is in calculating the detection area directly from the fusion image for the RGB and thermal modality. This increases data efficiency by 66%. Our results will facilitate the future development of non-contact monitoring to further improve the standard of care for preterm neonates.
format Online
Article
Text
id pubmed-10223875
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-102238752023-05-28 Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks Gleichauf, Johanna Hennemann, Lukas Fahlbusch, Fabian B. Hofmann, Oliver Niebler, Christine Koelpin, Alexander Sensors (Basel) Article The monitoring of vital signs and increasing patient comfort are cornerstones of modern neonatal intensive care. Commonly used monitoring methods are based on skin contact which can cause irritations and discomfort in preterm neonates. Therefore, non-contact approaches are the subject of current research aiming to resolve this dichotomy. Robust neonatal face detection is essential for the reliable detection of heart rate, respiratory rate and body temperature. While solutions for adult face detection are established, the unique neonatal proportions require a tailored approach. Additionally, sufficient open-source data of neonates on the NICU is lacking. We set out to train neural networks with the thermal-RGB-fusion data of neonates. We propose a novel indirect fusion approach including the sensor fusion of a thermal and RGB camera based on a 3D time-of-flight (ToF) camera. Unlike other approaches, this method is tailored for close distances encountered in neonatal incubators. Two neural networks were used with the fusion data and compared to RGB and thermal networks. For the class “head” we reached average precision values of 0.9958 (RetinaNet) and 0.9455 (YOLOv3) for the fusion data. Compared with the literature, similar precision was achieved, but we are the first to train a neural network with fusion data of neonates. The advantage of this approach is in calculating the detection area directly from the fusion image for the RGB and thermal modality. This increases data efficiency by 66%. Our results will facilitate the future development of non-contact monitoring to further improve the standard of care for preterm neonates. MDPI 2023-05-19 /pmc/articles/PMC10223875/ /pubmed/37430829 http://dx.doi.org/10.3390/s23104910 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Gleichauf, Johanna
Hennemann, Lukas
Fahlbusch, Fabian B.
Hofmann, Oliver
Niebler, Christine
Koelpin, Alexander
Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks
title Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks
title_full Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks
title_fullStr Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks
title_full_unstemmed Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks
title_short Sensor Fusion for the Robust Detection of Facial Regions of Neonates Using Neural Networks
title_sort sensor fusion for the robust detection of facial regions of neonates using neural networks
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10223875/
https://www.ncbi.nlm.nih.gov/pubmed/37430829
http://dx.doi.org/10.3390/s23104910
work_keys_str_mv AT gleichaufjohanna sensorfusionfortherobustdetectionoffacialregionsofneonatesusingneuralnetworks
AT hennemannlukas sensorfusionfortherobustdetectionoffacialregionsofneonatesusingneuralnetworks
AT fahlbuschfabianb sensorfusionfortherobustdetectionoffacialregionsofneonatesusingneuralnetworks
AT hofmannoliver sensorfusionfortherobustdetectionoffacialregionsofneonatesusingneuralnetworks
AT nieblerchristine sensorfusionfortherobustdetectionoffacialregionsofneonatesusingneuralnetworks
AT koelpinalexander sensorfusionfortherobustdetectionoffacialregionsofneonatesusingneuralnetworks