Cargando…

Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning

Citrus Huanglongbing (HLB), also named citrus greening disease, occurs worldwide and is known as a citrus cancer without an effective treatment. The symptoms of HLB are similar to those of nutritional deficiency or other disease. The methods based on single-source information, such as RGB images or...

Descripción completa

Detalles Bibliográficos
Autores principales: Yang, Dongzi, Wang, Fengcheng, Hu, Yuqi, Lan, Yubin, Deng, Xiaoling
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8751206/
https://www.ncbi.nlm.nih.gov/pubmed/35027917
http://dx.doi.org/10.3389/fpls.2021.809506
_version_ 1784631636295417856
author Yang, Dongzi
Wang, Fengcheng
Hu, Yuqi
Lan, Yubin
Deng, Xiaoling
author_facet Yang, Dongzi
Wang, Fengcheng
Hu, Yuqi
Lan, Yubin
Deng, Xiaoling
author_sort Yang, Dongzi
collection PubMed
description Citrus Huanglongbing (HLB), also named citrus greening disease, occurs worldwide and is known as a citrus cancer without an effective treatment. The symptoms of HLB are similar to those of nutritional deficiency or other disease. The methods based on single-source information, such as RGB images or hyperspectral data, are not able to achieve great detection performance. In this study, a multi-modal feature fusion network, combining a RGB image network and hyperspectral band extraction network, was proposed to recognize HLB from four categories (HLB, suspected HLB, Zn-deficient, and healthy). Three contributions including a dimension-reduction scheme for hyperspectral data based on a soft attention mechanism, a feature fusion proposal based on a bilinear fusion method, and auxiliary classifiers to extract more useful information are introduced in this manuscript. The multi-modal feature fusion network can effectively classify the above four types of citrus leaves and is better than single-modal classifiers. In experiments, the highest accuracy of multi-modal network recognition was 97.89% when the amount of data was not very abundant (1,325 images of the four aforementioned types and 1,325 pieces of hyperspectral data), while the single-modal network with RGB images only achieved 87.98% recognition and the single-modal network using hyperspectral information only 89%. Results show that the proposed multi-modal network implementing the concept of multi-source information fusion provides a better way to detect citrus HLB and citrus deficiency.
format Online
Article
Text
id pubmed-8751206
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-87512062022-01-12 Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning Yang, Dongzi Wang, Fengcheng Hu, Yuqi Lan, Yubin Deng, Xiaoling Front Plant Sci Plant Science Citrus Huanglongbing (HLB), also named citrus greening disease, occurs worldwide and is known as a citrus cancer without an effective treatment. The symptoms of HLB are similar to those of nutritional deficiency or other disease. The methods based on single-source information, such as RGB images or hyperspectral data, are not able to achieve great detection performance. In this study, a multi-modal feature fusion network, combining a RGB image network and hyperspectral band extraction network, was proposed to recognize HLB from four categories (HLB, suspected HLB, Zn-deficient, and healthy). Three contributions including a dimension-reduction scheme for hyperspectral data based on a soft attention mechanism, a feature fusion proposal based on a bilinear fusion method, and auxiliary classifiers to extract more useful information are introduced in this manuscript. The multi-modal feature fusion network can effectively classify the above four types of citrus leaves and is better than single-modal classifiers. In experiments, the highest accuracy of multi-modal network recognition was 97.89% when the amount of data was not very abundant (1,325 images of the four aforementioned types and 1,325 pieces of hyperspectral data), while the single-modal network with RGB images only achieved 87.98% recognition and the single-modal network using hyperspectral information only 89%. Results show that the proposed multi-modal network implementing the concept of multi-source information fusion provides a better way to detect citrus HLB and citrus deficiency. Frontiers Media S.A. 2021-12-23 /pmc/articles/PMC8751206/ /pubmed/35027917 http://dx.doi.org/10.3389/fpls.2021.809506 Text en Copyright © 2021 Yang, Wang, Hu, Lan and Deng. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Plant Science
Yang, Dongzi
Wang, Fengcheng
Hu, Yuqi
Lan, Yubin
Deng, Xiaoling
Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning
title Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning
title_full Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning
title_fullStr Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning
title_full_unstemmed Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning
title_short Citrus Huanglongbing Detection Based on Multi-Modal Feature Fusion Learning
title_sort citrus huanglongbing detection based on multi-modal feature fusion learning
topic Plant Science
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8751206/
https://www.ncbi.nlm.nih.gov/pubmed/35027917
http://dx.doi.org/10.3389/fpls.2021.809506
work_keys_str_mv AT yangdongzi citrushuanglongbingdetectionbasedonmultimodalfeaturefusionlearning
AT wangfengcheng citrushuanglongbingdetectionbasedonmultimodalfeaturefusionlearning
AT huyuqi citrushuanglongbingdetectionbasedonmultimodalfeaturefusionlearning
AT lanyubin citrushuanglongbingdetectionbasedonmultimodalfeaturefusionlearning
AT dengxiaoling citrushuanglongbingdetectionbasedonmultimodalfeaturefusionlearning