Cargando…

A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images

Anomaly detection aims to separate anomalous pixels from the background, and has become an important application of remotely sensed hyperspectral image processing. Anomaly detection methods based on low-rank and sparse representation (LRASR) can accurately detect anomalous pixels. However, with the...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhang, Yi, Wu, Zebin, Sun, Jin, Zhang, Yan, Zhu, Yaoqin, Liu, Jun, Zang, Qitao, Plaza, Antonio
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2018
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6263513/
https://www.ncbi.nlm.nih.gov/pubmed/30366454
http://dx.doi.org/10.3390/s18113627
_version_ 1783375310584020992
author Zhang, Yi
Wu, Zebin
Sun, Jin
Zhang, Yan
Zhu, Yaoqin
Liu, Jun
Zang, Qitao
Plaza, Antonio
author_facet Zhang, Yi
Wu, Zebin
Sun, Jin
Zhang, Yan
Zhu, Yaoqin
Liu, Jun
Zang, Qitao
Plaza, Antonio
author_sort Zhang, Yi
collection PubMed
description Anomaly detection aims to separate anomalous pixels from the background, and has become an important application of remotely sensed hyperspectral image processing. Anomaly detection methods based on low-rank and sparse representation (LRASR) can accurately detect anomalous pixels. However, with the significant volume increase of hyperspectral image repositories, such techniques consume a significant amount of time (mainly due to the massive amount of matrix computations involved). In this paper, we propose a novel distributed parallel algorithm (DPA) by redesigning key operators of LRASR in terms of MapReduce model to accelerate LRASR on cloud computing architectures. Independent computation operators are explored and executed in parallel on Spark. Specifically, we reconstitute the hyperspectral images in an appropriate format for efficient DPA processing, design the optimized storage strategy, and develop a pre-merge mechanism to reduce data transmission. Besides, a repartitioning policy is also proposed to improve DPA’s efficiency. Our experimental results demonstrate that the newly developed DPA achieves very high speedups when accelerating LRASR, in addition to maintaining similar accuracies. Moreover, our proposed DPA is shown to be scalable with the number of computing nodes and capable of processing big hyperspectral images involving massive amounts of data.
format Online
Article
Text
id pubmed-6263513
institution National Center for Biotechnology Information
language English
publishDate 2018
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-62635132018-12-12 A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images Zhang, Yi Wu, Zebin Sun, Jin Zhang, Yan Zhu, Yaoqin Liu, Jun Zang, Qitao Plaza, Antonio Sensors (Basel) Article Anomaly detection aims to separate anomalous pixels from the background, and has become an important application of remotely sensed hyperspectral image processing. Anomaly detection methods based on low-rank and sparse representation (LRASR) can accurately detect anomalous pixels. However, with the significant volume increase of hyperspectral image repositories, such techniques consume a significant amount of time (mainly due to the massive amount of matrix computations involved). In this paper, we propose a novel distributed parallel algorithm (DPA) by redesigning key operators of LRASR in terms of MapReduce model to accelerate LRASR on cloud computing architectures. Independent computation operators are explored and executed in parallel on Spark. Specifically, we reconstitute the hyperspectral images in an appropriate format for efficient DPA processing, design the optimized storage strategy, and develop a pre-merge mechanism to reduce data transmission. Besides, a repartitioning policy is also proposed to improve DPA’s efficiency. Our experimental results demonstrate that the newly developed DPA achieves very high speedups when accelerating LRASR, in addition to maintaining similar accuracies. Moreover, our proposed DPA is shown to be scalable with the number of computing nodes and capable of processing big hyperspectral images involving massive amounts of data. MDPI 2018-10-25 /pmc/articles/PMC6263513/ /pubmed/30366454 http://dx.doi.org/10.3390/s18113627 Text en © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Zhang, Yi
Wu, Zebin
Sun, Jin
Zhang, Yan
Zhu, Yaoqin
Liu, Jun
Zang, Qitao
Plaza, Antonio
A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images
title A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images
title_full A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images
title_fullStr A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images
title_full_unstemmed A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images
title_short A Distributed Parallel Algorithm Based on Low-Rank and Sparse Representation for Anomaly Detection in Hyperspectral Images
title_sort distributed parallel algorithm based on low-rank and sparse representation for anomaly detection in hyperspectral images
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6263513/
https://www.ncbi.nlm.nih.gov/pubmed/30366454
http://dx.doi.org/10.3390/s18113627
work_keys_str_mv AT zhangyi adistributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT wuzebin adistributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT sunjin adistributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT zhangyan adistributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT zhuyaoqin adistributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT liujun adistributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT zangqitao adistributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT plazaantonio adistributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT zhangyi distributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT wuzebin distributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT sunjin distributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT zhangyan distributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT zhuyaoqin distributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT liujun distributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT zangqitao distributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages
AT plazaantonio distributedparallelalgorithmbasedonlowrankandsparserepresentationforanomalydetectioninhyperspectralimages