Autonomous face mask detection using single shot multibox detector, and ResNet-50 with identity retrieval through face matching using deep siamese neural network

The COVID-19 pandemic poses a global health challenge. The World Health Organization states that face masks are proven to be effective, especially in public areas. Real-time monitoring of face masks is challenging and exhaustive for humans. To reduce human effort and to provide an enforcement mechanism, an autonomous system has been proposed to detect non-masked people and retrieve their identity using computer vision. The proposed method introduces a novel and efficient method that involves fine-tuning the pre-trained ResNet-50 model with a new head layer for classification between masked and non-masked people. The classifier is trained using adaptive momentum optimization algorithm with decaying learning rate and binary cross-entropy loss. Data augmentation and dropout regularization are employed to achieve best convergence. During real-time application of our classifier on videos, a Caffe face detector model based on Single Shot MultiBox Detector is used to extract the face regions of interest from each frame, on which the trained classifier is applied for detecting the non-masked people. The faces of these people are then captured, which is passed on to a deep siamese neural network, based on VGG-Face model for face matching. The captured faces are compared with the reference images from the database, by extracting the features and calculating cosine distance. If the faces match, the details of that person are retrieved from the database and displayed on the web application. The proposed method has secured best results where the trained classifier has achieved 99.74% accuracy, and the identity retrieval model achieved 98.24% accuracy.