Assessing kidney stone composition using smartphone microscopy and deep neural networks

OBJECTIVES: To propose a point‐of‐care image recognition system for kidney stone composition classification using smartphone microscopy and deep convolutional neural networks. MATERIALS AND METHODS: A total of 37 surgically extracted human kidney stones consisting of calcium oxalate (CaOx), cystine, uric acid (UA) and struvite stones were included in the study. All of the stones were fragmented from percutaneous nephrolithotomy (PCNL). The stones were classified using Fourier transform infrared spectroscopy (FTIR) analysis before obtaining smartphone microscope images. The size of the stones ranged from 5 to 10 mm in diameter. Nurugo 400× smartphone microscope (Nurugo, Seoul, Republic of Korea) was functionalized to acquire microscopic images (magnification = 25×) of dry kidney stones using iPhone 6s+ (Apple, Cupertino, CA, USA). Each kidney stone was imaged in six different locations. In total, 222 images were captured from 37 stones. A novel convolutional neural network architecture was built for classification, and the model was assessed using accuracy, positive predictive value, sensitivity and F1 scores. RESULTS: We achieved an overall and weighted accuracy of 88% and 87%, respectively, with an average F1 score of 0.84. The positive predictive value, sensitivity and F1 score for each stone type were respectively reported as follows: CaOx (0.82, 0.83, 0.82), cystine (0.80, 0.88, 0.84), UA (0.92, 0.77, 0.85) and struvite (0.86, 0.84, 0.85). CONCLUSION: We demonstrate a rapid and accurate point of care diagnostics method for classifying the four types of kidney stones. In the future, diagnostic tools that combine smartphone microscopy with artificial intelligence (AI) can provide accessible health care that can support physicians in their decision‐making process.