Multi-target video-based face recognition and gesture recognition based on enhanced detection and multi-trajectory incremental learning

BACKGROUND: Video-based face recognition (VFR) is one of the frontier topics in the domain of computer vision, which aims to automatically track and recognize facial regions of interests (ROIs) in video sequences. OBJECTIVE: In videos with multiple faces, the trajectories of individuals are incredibly complex. This is less studied than videos with a single face per frame. METHODS: In this paper, we present a multi-trajectory incremental learning (MTIL) algorithm, which categorizes trajectories using a Euclidean distance-based greedy algorithm and estimates the most likely labels for each trajectory by incremental learning to correct their classification and improve the accuracy of recognition. Furthermore, this study proposes an enhanced detection method that combines face detection with a robust tracking-learning-detection (TLD) algorithm to improve the performance of face detection in video. The method can also be extended for medical video recognition applications such as gesture recognition control based medical system. RESULTS: Experiments on Honda/UCSD and BMP (seq_mb) database demonstrate that our method can improve the face detection and face recognition (single or multiple) performance. The method also performs well on the gesture recognition system. CONCLUSION: The proposed MTIL algorithm can significantly improve the performance of the VFR system and the gesture recognition system.