Cargando…
High-resolution single-photon imaging with physics-informed deep learning
High-resolution single-photon imaging remains a big challenge due to the complex hardware manufacturing craft and noise disturbances. Here, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging with enhancement of bit depth and imaging quality. We first studied the co...
Autores principales: | , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10516985/ https://www.ncbi.nlm.nih.gov/pubmed/37737270 http://dx.doi.org/10.1038/s41467-023-41597-9 |
_version_ | 1785109239232987136 |
---|---|
author | Bian, Liheng Song, Haoze Peng, Lintao Chang, Xuyang Yang, Xi Horstmeyer, Roarke Ye, Lin Zhu, Chunli Qin, Tong Zheng, Dezhi Zhang, Jun |
author_facet | Bian, Liheng Song, Haoze Peng, Lintao Chang, Xuyang Yang, Xi Horstmeyer, Roarke Ye, Lin Zhu, Chunli Qin, Tong Zheng, Dezhi Zhang, Jun |
author_sort | Bian, Liheng |
collection | PubMed |
description | High-resolution single-photon imaging remains a big challenge due to the complex hardware manufacturing craft and noise disturbances. Here, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging with enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 × 32 pixels, 90 scenes, 10 different bit depths, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this physical noise model, we synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depths, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique in a series of experiments including microfluidic inspection, Fourier ptychography, and high-speed imaging. The experiments validate the technique’s state-of-the-art super-resolution SPAD imaging performance. |
format | Online Article Text |
id | pubmed-10516985 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-105169852023-09-24 High-resolution single-photon imaging with physics-informed deep learning Bian, Liheng Song, Haoze Peng, Lintao Chang, Xuyang Yang, Xi Horstmeyer, Roarke Ye, Lin Zhu, Chunli Qin, Tong Zheng, Dezhi Zhang, Jun Nat Commun Article High-resolution single-photon imaging remains a big challenge due to the complex hardware manufacturing craft and noise disturbances. Here, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging with enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 × 32 pixels, 90 scenes, 10 different bit depths, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this physical noise model, we synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depths, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique in a series of experiments including microfluidic inspection, Fourier ptychography, and high-speed imaging. The experiments validate the technique’s state-of-the-art super-resolution SPAD imaging performance. Nature Publishing Group UK 2023-09-22 /pmc/articles/PMC10516985/ /pubmed/37737270 http://dx.doi.org/10.1038/s41467-023-41597-9 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Bian, Liheng Song, Haoze Peng, Lintao Chang, Xuyang Yang, Xi Horstmeyer, Roarke Ye, Lin Zhu, Chunli Qin, Tong Zheng, Dezhi Zhang, Jun High-resolution single-photon imaging with physics-informed deep learning |
title | High-resolution single-photon imaging with physics-informed deep learning |
title_full | High-resolution single-photon imaging with physics-informed deep learning |
title_fullStr | High-resolution single-photon imaging with physics-informed deep learning |
title_full_unstemmed | High-resolution single-photon imaging with physics-informed deep learning |
title_short | High-resolution single-photon imaging with physics-informed deep learning |
title_sort | high-resolution single-photon imaging with physics-informed deep learning |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10516985/ https://www.ncbi.nlm.nih.gov/pubmed/37737270 http://dx.doi.org/10.1038/s41467-023-41597-9 |
work_keys_str_mv | AT bianliheng highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT songhaoze highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT penglintao highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT changxuyang highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT yangxi highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT horstmeyerroarke highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT yelin highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT zhuchunli highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT qintong highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT zhengdezhi highresolutionsinglephotonimagingwithphysicsinformeddeeplearning AT zhangjun highresolutionsinglephotonimagingwithphysicsinformeddeeplearning |