Recognizing schizophrenia using facial expressions based on convolutional neural network

OBJECTIVE: Facial expressions have been served as clinical symptoms to convey mental conditions in psychiatry. This paper proposes to recognize patients with schizophrenia (SCZ) using their facial images based on deep learning algorithm, and to investigate objective differences in facial expressions between SCZ patients and healthy controls using deep learning algorithm and statistical analyses. METHODS: The study consists of two parts. The first part recruited 106 SCZ patients and 101 healthy controls, and videotaped their facial expressions through a fixed experimental paradigm. The video data were randomly divided into two sets, one for training a convolutional neural network (CNN) with the classification of “healthy control” or “SCZ patient” as output and the other for evaluating the classification result of the trained CNN. In the second part, all facial images of the recruited participants were put into another CNN separately, which was priorly trained with a facial expression database and will output the most likely facial expressions of the recruited participants. Statistical analyses were performed on the obtained facial expressions to find out the objective differences in facial expressions between the two recruited groups. RESULTS: The trained CNN achieved an overall accuracy of 95.18% for classifying “healthy control” or “SCZ patient.” Statistical results on the obtained facial expressions demonstrated that the objective differences between the two recruited groups were statistically significant (p < .05). CONCLUSIONS: Facial expressions hold great promise as SCZ clues with the help of deep learning algorithm. The proposed approach would be potentially applied to mobile devices for autorecognizing SCZ in the context of clinical and daily life.