CerviCell-detector: An object detection approach for identifying the cancerous cells in pap smear images of cervical cancer

Cervical cancer is the second most commonly seen cancer in women. It affects the cervix portion of the vagina. The most preferred diagnostic test required for screening for cervical cancer is the pap smear test. Pap smear is a time-consuming test as it requires detailed analysis by expert cytologists. Cytologists can screen around 100 to 1000 slides depending upon the availability of advanced equipment. It requires substantial time and effort to carefully examine each slide, identify and classify cells, and make accurate diagnoses. Prolonged periods of visual inspection can increase the likelihood of human errors, such as overlooking abnormalities or misclassifying cells. The sheer volume of slides to be screened can exacerbate fatigue and impact diagnostic accuracy. Due to this reason Artificial intelligence (AI) based computer-aided diagnosis system for the classification and detection of pap smear images is needed. There are some AI-based solutions proposed in the literature, still, an effective and accurate system is under research. In this paper, we implement a state-of-the-art object detection model with a newly available CRIC dataset which follows the Bethesda system for nomenclature. Object detection models implemented are YOLOv5 which uses the CSPNet backbone, Faster R–CNN which has Region Proposal Network (RPN) and Detectron2 framework created by Facebook AI Research (FAIR) Group. ResNext model is implemented among the available models from Detectron2. The CRIC dataset is preprocessed and augmented using Roboflow tool. The performance measures of Average Precision and mean Average precision over the Intersection over Union (IoU) are used to evaluate the effectiveness of the models. The models performed better for two classes namely Normal and Abnormal compared to six classes from the Bethesda system. The highest mean Average Precision (mAP) is observed on the augmented dataset for YOLOv5 models for binary classification with 83 % mAP with IoU in the range of 0.50–0.95.