Automatic Schizophrenia Detection Using Multimodality Media via a Text Reading Task

Schizophrenia is a crippling chronic mental disease that affects people worldwide. In this work, an automatic schizophrenia detection algorithm is proposed based on the reading deficit of schizophrenic patients. From speech and video modalities, the automatic schizophrenia detection algorithm illustrates abnormal speech, head movement, and reading fluency during the reading task. In the speech modality, an acoustic model of speech emotional flatness in schizophrenia is established to reflect the emotional expression flatness of schizophrenic speech from the perspective of speech production and perception. In the video modality, the head-movement-related features are proposed to illustrate the spontaneous head movement caused by repeated reading and unconscious movement, and the reading-fluency-related features are proposed to convey the damaged degree of schizophrenic patients' reading fluency. The experimental data of this work are 160 segments of speech and video data recorded by 40 participants (20 schizophrenic patients and 20 normal controls). Combined with support vector machines and random forest, the accuracy of the proposed acoustic model, the head-movement-related features, and the reading-fluency-related features range from 94.38 to 96.50%, 73.38 to 83.38%, and 79.50 to 83.63%, respectively. The average accuracy of the proposed automatic schizophrenia detection algorithm reaches 97.50%. The experimental results indicate the effectiveness of the proposed automatic detection algorithm as an auxiliary diagnostic method for schizophrenia.