Cargando…
Incorporating the image formation process into deep learning improves network performance
We present Richardson–Lucy network (RLN), a fast and lightweight deep learning method for three-dimensional fluorescence microscopy deconvolution. RLN combines the traditional Richardson–Lucy iteration with a fully convolutional network structure, establishing a connection to the image formation pro...
Autores principales: | , , , , , , , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group US
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9636023/ https://www.ncbi.nlm.nih.gov/pubmed/36316563 http://dx.doi.org/10.1038/s41592-022-01652-7 |
_version_ | 1784824843923881984 |
---|---|
author | Li, Yue Su, Yijun Guo, Min Han, Xiaofei Liu, Jiamin Vishwasrao, Harshad D. Li, Xuesong Christensen, Ryan Sengupta, Titas Moyle, Mark W. Rey-Suarez, Ivan Chen, Jiji Upadhyaya, Arpita Usdin, Ted B. Colón-Ramos, Daniel Alfonso Liu, Huafeng Wu, Yicong Shroff, Hari |
author_facet | Li, Yue Su, Yijun Guo, Min Han, Xiaofei Liu, Jiamin Vishwasrao, Harshad D. Li, Xuesong Christensen, Ryan Sengupta, Titas Moyle, Mark W. Rey-Suarez, Ivan Chen, Jiji Upadhyaya, Arpita Usdin, Ted B. Colón-Ramos, Daniel Alfonso Liu, Huafeng Wu, Yicong Shroff, Hari |
author_sort | Li, Yue |
collection | PubMed |
description | We present Richardson–Lucy network (RLN), a fast and lightweight deep learning method for three-dimensional fluorescence microscopy deconvolution. RLN combines the traditional Richardson–Lucy iteration with a fully convolutional network structure, establishing a connection to the image formation process and thereby improving network performance. Containing only roughly 16,000 parameters, RLN enables four- to 50-fold faster processing than purely data-driven networks with many more parameters. By visual and quantitative analysis, we show that RLN provides better deconvolution, better generalizability and fewer artifacts than other networks, especially along the axial dimension. RLN outperforms classic Richardson–Lucy deconvolution on volumes contaminated with severe out of focus fluorescence or noise and provides four- to sixfold faster reconstructions of large, cleared-tissue datasets than classic multi-view pipelines. We demonstrate RLN’s performance on cells, tissues and embryos imaged with widefield-, light-sheet-, confocal- and super-resolution microscopy. |
format | Online Article Text |
id | pubmed-9636023 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Nature Publishing Group US |
record_format | MEDLINE/PubMed |
spelling | pubmed-96360232022-11-06 Incorporating the image formation process into deep learning improves network performance Li, Yue Su, Yijun Guo, Min Han, Xiaofei Liu, Jiamin Vishwasrao, Harshad D. Li, Xuesong Christensen, Ryan Sengupta, Titas Moyle, Mark W. Rey-Suarez, Ivan Chen, Jiji Upadhyaya, Arpita Usdin, Ted B. Colón-Ramos, Daniel Alfonso Liu, Huafeng Wu, Yicong Shroff, Hari Nat Methods Article We present Richardson–Lucy network (RLN), a fast and lightweight deep learning method for three-dimensional fluorescence microscopy deconvolution. RLN combines the traditional Richardson–Lucy iteration with a fully convolutional network structure, establishing a connection to the image formation process and thereby improving network performance. Containing only roughly 16,000 parameters, RLN enables four- to 50-fold faster processing than purely data-driven networks with many more parameters. By visual and quantitative analysis, we show that RLN provides better deconvolution, better generalizability and fewer artifacts than other networks, especially along the axial dimension. RLN outperforms classic Richardson–Lucy deconvolution on volumes contaminated with severe out of focus fluorescence or noise and provides four- to sixfold faster reconstructions of large, cleared-tissue datasets than classic multi-view pipelines. We demonstrate RLN’s performance on cells, tissues and embryos imaged with widefield-, light-sheet-, confocal- and super-resolution microscopy. Nature Publishing Group US 2022-10-31 2022 /pmc/articles/PMC9636023/ /pubmed/36316563 http://dx.doi.org/10.1038/s41592-022-01652-7 Text en © This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply 2022 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Li, Yue Su, Yijun Guo, Min Han, Xiaofei Liu, Jiamin Vishwasrao, Harshad D. Li, Xuesong Christensen, Ryan Sengupta, Titas Moyle, Mark W. Rey-Suarez, Ivan Chen, Jiji Upadhyaya, Arpita Usdin, Ted B. Colón-Ramos, Daniel Alfonso Liu, Huafeng Wu, Yicong Shroff, Hari Incorporating the image formation process into deep learning improves network performance |
title | Incorporating the image formation process into deep learning improves network performance |
title_full | Incorporating the image formation process into deep learning improves network performance |
title_fullStr | Incorporating the image formation process into deep learning improves network performance |
title_full_unstemmed | Incorporating the image formation process into deep learning improves network performance |
title_short | Incorporating the image formation process into deep learning improves network performance |
title_sort | incorporating the image formation process into deep learning improves network performance |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9636023/ https://www.ncbi.nlm.nih.gov/pubmed/36316563 http://dx.doi.org/10.1038/s41592-022-01652-7 |
work_keys_str_mv | AT liyue incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT suyijun incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT guomin incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT hanxiaofei incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT liujiamin incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT vishwasraoharshadd incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT lixuesong incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT christensenryan incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT senguptatitas incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT moylemarkw incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT reysuarezivan incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT chenjiji incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT upadhyayaarpita incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT usdintedb incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT colonramosdanielalfonso incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT liuhuafeng incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT wuyicong incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance AT shroffhari incorporatingtheimageformationprocessintodeeplearningimprovesnetworkperformance |