Cargando…

Deep-learning-based in-field citrus fruit detection and tracking

Fruit yield estimation is crucial for establishing fruit harvest and marketing strategies. Recently, computer vision and deep learning techniques have been used to estimate citrus fruit yield and have exhibited notable fruit detection ability. However, computer-vision-based citrus fruit counting has...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhang, Wenli, Wang, Jiaqi, Liu, Yuxin, Chen, Kaizhen, Li, Huibin, Duan, Yulin, Wu, Wenbin, Shi, Yun, Guo, Wei
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Oxford University Press 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9113225/
https://www.ncbi.nlm.nih.gov/pubmed/35147157
http://dx.doi.org/10.1093/hr/uhac003
_version_ 1784709546599514112
author Zhang, Wenli
Wang, Jiaqi
Liu, Yuxin
Chen, Kaizhen
Li, Huibin
Duan, Yulin
Wu, Wenbin
Shi, Yun
Guo, Wei
author_facet Zhang, Wenli
Wang, Jiaqi
Liu, Yuxin
Chen, Kaizhen
Li, Huibin
Duan, Yulin
Wu, Wenbin
Shi, Yun
Guo, Wei
author_sort Zhang, Wenli
collection PubMed
description Fruit yield estimation is crucial for establishing fruit harvest and marketing strategies. Recently, computer vision and deep learning techniques have been used to estimate citrus fruit yield and have exhibited notable fruit detection ability. However, computer-vision-based citrus fruit counting has two key limitations: inconsistent fruit detection accuracy and double-counting of the same fruit. Using oranges as the experimental material, this paper proposes a deep-learning-based orange counting algorithm using video sequences to help overcome these problems. The algorithm consists of two sub-algorithms, OrangeYolo for fruit detection and OrangeSort for fruit tracking. The OrangeYolo backbone network is partially based on the YOLOv3 algorithm, which has been improved upon to detect small objects (fruits) at multiple scales. The network structure was adjusted to detect small-scale targets while enabling multiscale target detection. A channel attention and spatial attention multiscale fusion module was introduced to fuse the semantic features of the deep network with the shallow textural detail features. OrangeYolo can achieve mean Average Precision (mAP) values of 0.957 in the citrus dataset, higher than the 0.905, 0.911, and 0.917 achieved with the YOLOv3, YOLOv4, and YOLOv5 algorithms. OrangeSort was designed to alleviate the double-counting problem associated with occluded fruits. A specific tracking region counting strategy and tracking algorithm based on motion displacement estimation were established. Six video sequences taken from two fields containing 22 trees were used as the validation dataset. The proposed method showed better performance (Mean Absolute Error (MAE) = 0.081, Standard Deviation (SD) = 0.08) than video-based manual counting and produced more accurate results than the existing standards Sort and DeepSort (MAE = 0.45 and 1.212; SD = 0.4741 and 1.3975).
format Online
Article
Text
id pubmed-9113225
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Oxford University Press
record_format MEDLINE/PubMed
spelling pubmed-91132252022-05-18 Deep-learning-based in-field citrus fruit detection and tracking Zhang, Wenli Wang, Jiaqi Liu, Yuxin Chen, Kaizhen Li, Huibin Duan, Yulin Wu, Wenbin Shi, Yun Guo, Wei Hortic Res Article Fruit yield estimation is crucial for establishing fruit harvest and marketing strategies. Recently, computer vision and deep learning techniques have been used to estimate citrus fruit yield and have exhibited notable fruit detection ability. However, computer-vision-based citrus fruit counting has two key limitations: inconsistent fruit detection accuracy and double-counting of the same fruit. Using oranges as the experimental material, this paper proposes a deep-learning-based orange counting algorithm using video sequences to help overcome these problems. The algorithm consists of two sub-algorithms, OrangeYolo for fruit detection and OrangeSort for fruit tracking. The OrangeYolo backbone network is partially based on the YOLOv3 algorithm, which has been improved upon to detect small objects (fruits) at multiple scales. The network structure was adjusted to detect small-scale targets while enabling multiscale target detection. A channel attention and spatial attention multiscale fusion module was introduced to fuse the semantic features of the deep network with the shallow textural detail features. OrangeYolo can achieve mean Average Precision (mAP) values of 0.957 in the citrus dataset, higher than the 0.905, 0.911, and 0.917 achieved with the YOLOv3, YOLOv4, and YOLOv5 algorithms. OrangeSort was designed to alleviate the double-counting problem associated with occluded fruits. A specific tracking region counting strategy and tracking algorithm based on motion displacement estimation were established. Six video sequences taken from two fields containing 22 trees were used as the validation dataset. The proposed method showed better performance (Mean Absolute Error (MAE) = 0.081, Standard Deviation (SD) = 0.08) than video-based manual counting and produced more accurate results than the existing standards Sort and DeepSort (MAE = 0.45 and 1.212; SD = 0.4741 and 1.3975). Oxford University Press 2022-02-11 /pmc/articles/PMC9113225/ /pubmed/35147157 http://dx.doi.org/10.1093/hr/uhac003 Text en © The Author(s) 2022. Published by Oxford University Press on behalf of Nanjing Agricultural University https://creativecommons.org/licenses/by/4.0/This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
spellingShingle Article
Zhang, Wenli
Wang, Jiaqi
Liu, Yuxin
Chen, Kaizhen
Li, Huibin
Duan, Yulin
Wu, Wenbin
Shi, Yun
Guo, Wei
Deep-learning-based in-field citrus fruit detection and tracking
title Deep-learning-based in-field citrus fruit detection and tracking
title_full Deep-learning-based in-field citrus fruit detection and tracking
title_fullStr Deep-learning-based in-field citrus fruit detection and tracking
title_full_unstemmed Deep-learning-based in-field citrus fruit detection and tracking
title_short Deep-learning-based in-field citrus fruit detection and tracking
title_sort deep-learning-based in-field citrus fruit detection and tracking
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9113225/
https://www.ncbi.nlm.nih.gov/pubmed/35147157
http://dx.doi.org/10.1093/hr/uhac003
work_keys_str_mv AT zhangwenli deeplearningbasedinfieldcitrusfruitdetectionandtracking
AT wangjiaqi deeplearningbasedinfieldcitrusfruitdetectionandtracking
AT liuyuxin deeplearningbasedinfieldcitrusfruitdetectionandtracking
AT chenkaizhen deeplearningbasedinfieldcitrusfruitdetectionandtracking
AT lihuibin deeplearningbasedinfieldcitrusfruitdetectionandtracking
AT duanyulin deeplearningbasedinfieldcitrusfruitdetectionandtracking
AT wuwenbin deeplearningbasedinfieldcitrusfruitdetectionandtracking
AT shiyun deeplearningbasedinfieldcitrusfruitdetectionandtracking
AT guowei deeplearningbasedinfieldcitrusfruitdetectionandtracking