Cargando…

SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction

Accurate hyperspectral remote sensing information is essential for feature identification and detection. Nevertheless, the hyperspectral imaging mechanism poses challenges in balancing the trade-off between spatial and spectral resolution. Hardware improvements are cost-intensive and depend on stric...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhu, Lingxuan, Wu, Jiaji, Biao, Wang, Liao, Yi, Gu, Dandan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10099040/
https://www.ncbi.nlm.nih.gov/pubmed/37050788
http://dx.doi.org/10.3390/s23073728
_version_ 1785024962361294848
author Zhu, Lingxuan
Wu, Jiaji
Biao, Wang
Liao, Yi
Gu, Dandan
author_facet Zhu, Lingxuan
Wu, Jiaji
Biao, Wang
Liao, Yi
Gu, Dandan
author_sort Zhu, Lingxuan
collection PubMed
description Accurate hyperspectral remote sensing information is essential for feature identification and detection. Nevertheless, the hyperspectral imaging mechanism poses challenges in balancing the trade-off between spatial and spectral resolution. Hardware improvements are cost-intensive and depend on strict environmental conditions and extra equipment. Recent spectral imaging methods have attempted to directly reconstruct hyperspectral information from widely available multispectral images. However, fixed mapping approaches used in previous spectral reconstruction models limit their reconstruction quality and generalizability, especially dealing with missing or contaminated bands. Moreover, data-hungry issues plague increasingly complex data-driven spectral reconstruction methods. This paper proposes SpectralMAE, a novel spectral reconstruction model that can take arbitrary combinations of bands as input and improve the utilization of data sources. In contrast to previous spectral reconstruction techniques, SpectralMAE explores the application of a self-supervised learning paradigm and proposes a masked autoencoder architecture for spectral dimensions. To further enhance the performance for specific sensor inputs, we propose a training strategy by combining random masking pre-training and fixed masking fine-tuning. Empirical evaluations on five remote sensing datasets demonstrate that SpectralMAE outperforms state-of-the-art methods in both qualitative and quantitative metrics.
format Online
Article
Text
id pubmed-10099040
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-100990402023-04-14 SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction Zhu, Lingxuan Wu, Jiaji Biao, Wang Liao, Yi Gu, Dandan Sensors (Basel) Article Accurate hyperspectral remote sensing information is essential for feature identification and detection. Nevertheless, the hyperspectral imaging mechanism poses challenges in balancing the trade-off between spatial and spectral resolution. Hardware improvements are cost-intensive and depend on strict environmental conditions and extra equipment. Recent spectral imaging methods have attempted to directly reconstruct hyperspectral information from widely available multispectral images. However, fixed mapping approaches used in previous spectral reconstruction models limit their reconstruction quality and generalizability, especially dealing with missing or contaminated bands. Moreover, data-hungry issues plague increasingly complex data-driven spectral reconstruction methods. This paper proposes SpectralMAE, a novel spectral reconstruction model that can take arbitrary combinations of bands as input and improve the utilization of data sources. In contrast to previous spectral reconstruction techniques, SpectralMAE explores the application of a self-supervised learning paradigm and proposes a masked autoencoder architecture for spectral dimensions. To further enhance the performance for specific sensor inputs, we propose a training strategy by combining random masking pre-training and fixed masking fine-tuning. Empirical evaluations on five remote sensing datasets demonstrate that SpectralMAE outperforms state-of-the-art methods in both qualitative and quantitative metrics. MDPI 2023-04-04 /pmc/articles/PMC10099040/ /pubmed/37050788 http://dx.doi.org/10.3390/s23073728 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Zhu, Lingxuan
Wu, Jiaji
Biao, Wang
Liao, Yi
Gu, Dandan
SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction
title SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction
title_full SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction
title_fullStr SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction
title_full_unstemmed SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction
title_short SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction
title_sort spectralmae: spectral masked autoencoder for hyperspectral remote sensing image reconstruction
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10099040/
https://www.ncbi.nlm.nih.gov/pubmed/37050788
http://dx.doi.org/10.3390/s23073728
work_keys_str_mv AT zhulingxuan spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction
AT wujiaji spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction
AT biaowang spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction
AT liaoyi spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction
AT gudandan spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction