Automated vs. manual pain coding and heart rate estimations based on videos of older adults with and without dementia

INTRODUCTION: Technological advances have allowed for the estimation of physiological indicators from video data. FaceReader™ is an automated facial analysis software that has been used widely in studies of facial expressions of emotion and was recently updated to allow for the estimation of heart rate (HR) using remote photoplethysmography (rPPG). We investigated FaceReader™-based heart rate and pain expression estimations in older adults in relation to manual coding by experts. METHODS: Using a video dataset of older adult patients with and without dementia, we assessed the relationship between FaceReader’s™ HR estimations against a well-established Video Magnification (VM) algorithm during baseline and pain conditions. Furthermore, we examined the correspondence between the Facial Action Coding System (FACS)-based pain scores obtained through FaceReader™ and manual coding. RESULTS: FaceReader’s™ HR estimations were correlated with VM algorithm in baseline and pain conditions. Non-verbal FaceReader™ pain scores and manual coding were also highly correlated despite discrepancies between the FaceReader™ and manual coding in the absolute value of scores based on pain-related facial action coding of the events preceding and following the pain response. CONCLUSIONS: Compared to expert manual FACS coding and optimized VM algorithm, FaceReader™ showed good results in estimating HR values and non-verbal pain scores.