Monitoring Endangered and Rare Wildlife in the Field: A Foundation Deep Learning Model Integrating Human Knowledge for Incremental Recognition with Few Data and Low Cost

SIMPLE SUMMARY: Intelligent monitoring of endangered and rare wildlife using deep learning is important for biodiversity conservation. The present study aims to train deep learning recognition models using the limited animal samples and computational resources available in real-world field monitoring scenarios. Inspired by the ability of human zoologists to quickly identify endangered and rare wildlife species based on only one single picture, we incorporate expert knowledge to create an excellent recognition model with these practical constraints. The empirical evidence discovered in this study indicates that the insightful incorporation of specialist knowledge can meaningfully improve an algorithm’s accuracy, even with limited sample sizes. Additionally, our experimental results validate the usefulness and efficacy of our discoveries in identifying previously unknown species. This work presents the first exploration of a practical solution for endangered and rare wildlife monitoring using a foundational deep learning model. Automated monitoring capability for unidentified species has the potential to facilitate breakthroughs in the fields of zoology and biodiversity research. ABSTRACT: Intelligent monitoring of endangered and rare wildlife is important for biodiversity conservation. In practical monitoring, few animal data are available to train recognition algorithms. The system must, therefore, achieve high accuracy with limited resources. Simultaneously, zoologists expect the system to be able to discover unknown species to make significant discoveries. To date, none of the current algorithms have these abilities. Therefore, this paper proposed a KI-CLIP method. Firstly, by first introducing CLIP, a foundation deep learning model that has not yet been applied in animal fields, the powerful recognition capability with few training resources is exploited with an additional shallow network. Secondly, inspired by the single-image recognition abilities of zoologists, we incorporate easily accessible expert description texts to improve performance with few samples. Finally, a simple incremental learning module is designed to detect unknown species. We conducted extensive comparative experiments, ablation experiments, and case studies on 12 datasets containing real data. The results validate the effectiveness of KI-CLIP, which can be trained on multiple real scenarios in seconds, achieving in our study over 90% recognition accuracy with only 8 training samples, and over 97% with 16 training samples. In conclusion, KI-CLIP is suitable for practical animal monitoring.