Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks

Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications. The emergence of novel coronavirus pneumonia COVID-19 has attracted worldwide attentions. Contact photoplethysmography (cPPG) methods need to contact the detection equipment with the patient, which may accelerate the spread of the epidemic. In the future, the non-contact heart rate detection will be an urgent need. However, existing heart rate measuring methods from facial videos are vulnerable to the less-constrained scenarios (e.g., with head movement and wearing a mask). In this paper, we proposed a method of heart rate detection based on eye location of region of interest (ROI) to solve the problem of missing information when wearing masks. Besides, a model to filter outliers based on residual network was conceived first by us and the better heart rate measurement accuracy was generated. To validate our method, we also created a mask dataset. The results demonstrated that after using our method for correcting the heart rate (HR) value measured with the traditional method, the accuracy reaches 4.65 bpm, which is 0.42 bpm higher than that without correction.