Cargando…

“Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots

Blueberries are grown worldwide because of their high nutritional value; however, manual picking is difficult, and expert pickers are scarce. To meet the real needs of the market, picking robots that can identify the ripeness of blueberries are increasingly being used to replace manual operators. Ho...

Descripción completa

Detalles Bibliográficos
Autores principales: Liu, Yan, Zheng, Hongtao, Zhang, Yonghua, Zhang, Qiujie, Chen, Hongli, Xu, Xueyong, Wang, Gaoyang
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10289036/
https://www.ncbi.nlm.nih.gov/pubmed/37360727
http://dx.doi.org/10.3389/fpls.2023.1198650
_version_ 1785062198181101568
author Liu, Yan
Zheng, Hongtao
Zhang, Yonghua
Zhang, Qiujie
Chen, Hongli
Xu, Xueyong
Wang, Gaoyang
author_facet Liu, Yan
Zheng, Hongtao
Zhang, Yonghua
Zhang, Qiujie
Chen, Hongli
Xu, Xueyong
Wang, Gaoyang
author_sort Liu, Yan
collection PubMed
description Blueberries are grown worldwide because of their high nutritional value; however, manual picking is difficult, and expert pickers are scarce. To meet the real needs of the market, picking robots that can identify the ripeness of blueberries are increasingly being used to replace manual operators. However, they struggle to accurately identify the ripeness of blueberries because of the heavy shading between the fruits and the small size of the fruit. This makes it difficult to obtain sufficient information on characteristics; and the disturbances caused by environmental changes remain unsolved. Additionally, the picking robot has limited computational power for running complex algorithms. To address these issues, we propose a new YOLO-based algorithm to detect the ripeness of blueberry fruits. The algorithm improves the structure of YOLOv5x. We replaced the fully connected layer with a one-dimensional convolution and also replaced the high-latitude convolution with a null convolution based on the structure of CBAM, and finally obtained a lightweight CBAM structure with efficient attention-guiding capability (Little-CBAM), which we embedded into MobileNetv3 while replacing the original backbone structure with the improved MobileNetv3. We expanded the original three-layer neck path by one to create a larger-scale detection layer leading from the backbone network. We added a multi-scale fusion module to the channel attention mechanism to build a multi-method feature extractor (MSSENet) and then embedded the designed channel attention module into the head network, which can significantly enhance the feature representation capability of the small target detection network and the anti-interference capability of the algorithm. Considering that these improvements will significantly extend the training time of the algorithm, we used EIOU_Loss instead of CIOU_Loss, whereas the k-means++ algorithm was used to cluster the detection frames such that the generated predefined anchor frames are better adapted to the scale of the blueberries. The algorithm in this study achieved a final mAP of 78.3% on the PC terminal, which was 9% higher than that of YOLOv5x, and the FPS was 2.1 times higher than that of YOLOv5x. By translating the algorithm into a picking robot, the algorithm in this study ran at 47 FPS and achieved real-time detection well beyond that achieved manually.
format Online
Article
Text
id pubmed-10289036
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-102890362023-06-24 “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots Liu, Yan Zheng, Hongtao Zhang, Yonghua Zhang, Qiujie Chen, Hongli Xu, Xueyong Wang, Gaoyang Front Plant Sci Plant Science Blueberries are grown worldwide because of their high nutritional value; however, manual picking is difficult, and expert pickers are scarce. To meet the real needs of the market, picking robots that can identify the ripeness of blueberries are increasingly being used to replace manual operators. However, they struggle to accurately identify the ripeness of blueberries because of the heavy shading between the fruits and the small size of the fruit. This makes it difficult to obtain sufficient information on characteristics; and the disturbances caused by environmental changes remain unsolved. Additionally, the picking robot has limited computational power for running complex algorithms. To address these issues, we propose a new YOLO-based algorithm to detect the ripeness of blueberry fruits. The algorithm improves the structure of YOLOv5x. We replaced the fully connected layer with a one-dimensional convolution and also replaced the high-latitude convolution with a null convolution based on the structure of CBAM, and finally obtained a lightweight CBAM structure with efficient attention-guiding capability (Little-CBAM), which we embedded into MobileNetv3 while replacing the original backbone structure with the improved MobileNetv3. We expanded the original three-layer neck path by one to create a larger-scale detection layer leading from the backbone network. We added a multi-scale fusion module to the channel attention mechanism to build a multi-method feature extractor (MSSENet) and then embedded the designed channel attention module into the head network, which can significantly enhance the feature representation capability of the small target detection network and the anti-interference capability of the algorithm. Considering that these improvements will significantly extend the training time of the algorithm, we used EIOU_Loss instead of CIOU_Loss, whereas the k-means++ algorithm was used to cluster the detection frames such that the generated predefined anchor frames are better adapted to the scale of the blueberries. The algorithm in this study achieved a final mAP of 78.3% on the PC terminal, which was 9% higher than that of YOLOv5x, and the FPS was 2.1 times higher than that of YOLOv5x. By translating the algorithm into a picking robot, the algorithm in this study ran at 47 FPS and achieved real-time detection well beyond that achieved manually. Frontiers Media S.A. 2023-06-09 /pmc/articles/PMC10289036/ /pubmed/37360727 http://dx.doi.org/10.3389/fpls.2023.1198650 Text en Copyright © 2023 Liu, Zheng, Zhang, Zhang, Chen, Xu and Wang https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Plant Science
Liu, Yan
Zheng, Hongtao
Zhang, Yonghua
Zhang, Qiujie
Chen, Hongli
Xu, Xueyong
Wang, Gaoyang
“Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots
title “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots
title_full “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots
title_fullStr “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots
title_full_unstemmed “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots
title_short “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots
title_sort “is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots
topic Plant Science
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10289036/
https://www.ncbi.nlm.nih.gov/pubmed/37360727
http://dx.doi.org/10.3389/fpls.2023.1198650
work_keys_str_mv AT liuyan isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots
AT zhenghongtao isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots
AT zhangyonghua isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots
AT zhangqiujie isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots
AT chenhongli isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots
AT xuxueyong isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots
AT wanggaoyang isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots