Cargando…
Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism
Remote photoplethysmography (rPPG) is a video-based non-contact heart rate measurement technology. It is a fact that most existing rPPG methods fail to deal with the spatiotemporal features of the video, which is significant for the extraction of the rPPG signal. In this paper, we propose a 3D centr...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8781886/ https://www.ncbi.nlm.nih.gov/pubmed/35062649 http://dx.doi.org/10.3390/s22020688 |
_version_ | 1784638188072992768 |
---|---|
author | Liu, Xinhua Wei, Wenqian Kuang, Hailan Ma, Xiaolin |
author_facet | Liu, Xinhua Wei, Wenqian Kuang, Hailan Ma, Xiaolin |
author_sort | Liu, Xinhua |
collection | PubMed |
description | Remote photoplethysmography (rPPG) is a video-based non-contact heart rate measurement technology. It is a fact that most existing rPPG methods fail to deal with the spatiotemporal features of the video, which is significant for the extraction of the rPPG signal. In this paper, we propose a 3D central difference convolutional network (CDCA-rPPGNet) to measure heart rate, with an attention mechanism to combine spatial and temporal features. First, we crop and stitch the region of interest together through facial landmarks. Next, the high-quality regions of interest are fed to CDCA-rPPGNet based on a central difference convolution, which can enhance the spatiotemporal representation and capture rich relevant time contexts by collecting time difference information. In addition, we integrate the attention module into the neural network, aiming to strengthen the ability of the neural network to extract video channels and spatial features, so as to obtain more accurate rPPG signals. In summary, the three main contributions of this paper are as follows: (1) the proposed network base on central difference convolution could better capture the subtle color changes to recover the rPPG signals; (2) the proposed ROI extraction method provides high-quality input to the network; (3) the attention module is used to strengthen the ability of the network to extract features. Extensive experiments are conducted on two public datasets—the PURE dataset and the UBFC-rPPG dataset. In terms of the experiment results, our proposed method achieves 0.46 MAE (bpm), 0.90 RMSE (bpm) and 0.99 R value of Pearson’s correlation coefficient on the PURE dataset, and 0.60 MAE (bpm), 1.38 RMSE (bpm) and 0.99 R value of Pearson’s correlation coefficient on the UBFC dataset, which proves the effectiveness of our proposed approach. |
format | Online Article Text |
id | pubmed-8781886 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-87818862022-01-22 Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism Liu, Xinhua Wei, Wenqian Kuang, Hailan Ma, Xiaolin Sensors (Basel) Article Remote photoplethysmography (rPPG) is a video-based non-contact heart rate measurement technology. It is a fact that most existing rPPG methods fail to deal with the spatiotemporal features of the video, which is significant for the extraction of the rPPG signal. In this paper, we propose a 3D central difference convolutional network (CDCA-rPPGNet) to measure heart rate, with an attention mechanism to combine spatial and temporal features. First, we crop and stitch the region of interest together through facial landmarks. Next, the high-quality regions of interest are fed to CDCA-rPPGNet based on a central difference convolution, which can enhance the spatiotemporal representation and capture rich relevant time contexts by collecting time difference information. In addition, we integrate the attention module into the neural network, aiming to strengthen the ability of the neural network to extract video channels and spatial features, so as to obtain more accurate rPPG signals. In summary, the three main contributions of this paper are as follows: (1) the proposed network base on central difference convolution could better capture the subtle color changes to recover the rPPG signals; (2) the proposed ROI extraction method provides high-quality input to the network; (3) the attention module is used to strengthen the ability of the network to extract features. Extensive experiments are conducted on two public datasets—the PURE dataset and the UBFC-rPPG dataset. In terms of the experiment results, our proposed method achieves 0.46 MAE (bpm), 0.90 RMSE (bpm) and 0.99 R value of Pearson’s correlation coefficient on the PURE dataset, and 0.60 MAE (bpm), 1.38 RMSE (bpm) and 0.99 R value of Pearson’s correlation coefficient on the UBFC dataset, which proves the effectiveness of our proposed approach. MDPI 2022-01-17 /pmc/articles/PMC8781886/ /pubmed/35062649 http://dx.doi.org/10.3390/s22020688 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Liu, Xinhua Wei, Wenqian Kuang, Hailan Ma, Xiaolin Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism |
title | Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism |
title_full | Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism |
title_fullStr | Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism |
title_full_unstemmed | Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism |
title_short | Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism |
title_sort | heart rate measurement based on 3d central difference convolution with attention mechanism |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8781886/ https://www.ncbi.nlm.nih.gov/pubmed/35062649 http://dx.doi.org/10.3390/s22020688 |
work_keys_str_mv | AT liuxinhua heartratemeasurementbasedon3dcentraldifferenceconvolutionwithattentionmechanism AT weiwenqian heartratemeasurementbasedon3dcentraldifferenceconvolutionwithattentionmechanism AT kuanghailan heartratemeasurementbasedon3dcentraldifferenceconvolutionwithattentionmechanism AT maxiaolin heartratemeasurementbasedon3dcentraldifferenceconvolutionwithattentionmechanism |