Classification of fissured tongue images using deep neural networks

BACKGROUND: Tongue inspection is vital in traditional Chinese medicine. Fissured tongue is an important feature in tongue diagnosis, and primarily corresponds to three Chinese medicine syndromes: syndrome-related hotness, blood deficiency, and insufficiency of the spleen. Diagnosis of the syndrome is significantly affected by the experience of clinicians, and it is difficult for young doctors to perform accurate diagnoses. OBJECTIVE: The syndrome not only depends on the local features based on fissured regions but also on the global features of the whole tongue; therefore, a syndrome diagnosis framework combining the global and local features of a fissured tongue image was developed in the present study to achieve a quantitative and objective diagnosis. METHODS: First, we detected the fissured region of a tongue image using a single-shot multibox detector. Second, we extracted the global and local features from a whole tongue image and a fissured region using TongueNet (developed in-house). Third, we developed a classifier to determine the final syndrome. RESULTS: Based on an experiment involving 721 fissured tongue images, we discovered that TongueNet affords better feature extraction. The accuracy of TongueNet was 4% ([Formula: see text] 0.05) and 3% ([Formula: see text] 0.05) higher than that of InceptionV3 and ResNet18, respectively, for whole tongue images. Meanwhile, at local fissured regions, the accuracy of TongueNet was 3% ([Formula: see text] 0.05) higher than that of InceptionV3 and equal to that of ResNet18. Finally, the fusion features outperformed the global and local features with a 78% accuracy. CONCLUSIONS: Our findings indicate that TongueNet designed with batch normalization and dropout is more suitable for uncomplicated images than InceptionV3 and ResNet18. In addition, compared with the global features, the fusion features supplement the detailed information of the fissures and improve classification accuracy.