Identifying and mapping individual medicinal plant Lamiophlomis rotata at high elevations by using unmanned aerial vehicles and deep learning

BACKGROUND: The identification and enumeration of medicinal plants at high elevations is an important part of accurate yield calculations. However, the current assessment of medicinal plant reserves continues to rely on field sampling surveys, which are cumbersome and time-consuming. Recently, unmanned aerial vehicle (UAV) remote sensing and deep learning (DL) have provided ultrahigh-resolution imagery and high-accuracy object recognition techniques, respectively, providing an excellent opportunity to improve the current manual surveying of plants. However, accurate segmentation of individual plants from drone images remains a significant challenge due to the large variation in size, geometry, and distribution of medicinal plants. RESULTS: In this study, we proposed a new pipeline for wild medicinal plant detection and yield assessment based on UAV and DL that was specifically designed for detecting wild medicinal plants in an orthomosaic. We used a drone to collect panoramic images of Lamioplomis rotata Kudo (LR) in high-altitude areas. Then, we annotated and cropped these images into equally sized sub-images and used a DL model Mask R-CNN for object detection and segmentation of LR. Finally, on the basis of the segmentation results, we accurately counted the number and yield of LRs. The results showed that the Mask R-CNN model based on the ResNet-101 backbone network was superior to ResNet-50 in all evaluation indicators. The average identification precision of LR by Mask R-CNN based on the ResNet-101 backbone network was 89.34%, while that of ResNet-50 was 88.32%. The cross-validation results showed that the average accuracy of ResNet-101 was 78.73%, while that of ResNet-50 was 71.25%. According to the orthomosaic, the average number and yield of LR in the two sample sites were 19,376 plants and 57.93 kg and 19,129 plants and 73.5 kg respectively. CONCLUSIONS: The combination of DL and UAV remote sensing reveals significant promise in medicinal plant detection, counting, and yield prediction, which will benefit the monitoring of their populations for conservation assessment and management, among other applications. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13007-023-01015-z.