Breast Tumor Classification in Ultrasound Images Using Combined Deep and Handcrafted Features

This study aims to enable effective breast ultrasound image classification by combining deep features with conventional handcrafted features to classify the tumors. In particular, the deep features are extracted from a pre-trained convolutional neural network model, namely the VGG19 model, at six different extraction levels. The deep features extracted at each level are analyzed using a features selection algorithm to identify the deep feature combination that achieves the highest classification performance. Furthermore, the extracted deep features are combined with handcrafted texture and morphological features and processed using features selection to investigate the possibility of improving the classification performance. The cross-validation analysis, which is performed using 380 breast ultrasound images, shows that the best combination of deep features is obtained using a feature set, denoted by [Formula: see text] features that include convolution features extracted from all convolution blocks of the VGG19 model. In particular, the [Formula: see text] features achieved mean accuracy, sensitivity, and specificity values of [Formula: see text] , [Formula: see text] , and [Formula: see text] , respectively. The analysis also shows that the performance of the [Formula: see text] features degrades substantially when the features selection algorithm is not applied. The classification performance of the [Formula: see text] features is improved by combining these features with handcrafted morphological features to achieve mean accuracy, sensitivity, and specificity values of [Formula: see text] , [Formula: see text] , and [Formula: see text] , respectively. Furthermore, the cross-validation analysis demonstrates that the [Formula: see text] features and the combined [Formula: see text] and morphological features outperform the handcrafted texture and morphological features as well as the fine-tuned VGG19 model. The generalization performance of the [Formula: see text] features and the combined [Formula: see text] and morphological features is demonstrated by performing the training using the 380 breast ultrasound images and the testing using another dataset that includes 163 images. The results suggest that the combined [Formula: see text] and morphological features can achieve effective breast ultrasound image classifications that increase the capability of detecting malignant tumors and reduce the potential of misclassifying benign tumors.