Cargando…

Universal adaptive optics for microscopy through embedded neural network control

The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutio...

Descripción completa

Detalles Bibliográficos
Autores principales: Hu, Qi, Hailstone, Martin, Wang, Jingyu, Wincott, Matthew, Stoychev, Danail, Atilgan, Huriye, Gala, Dalia, Chaiamarit, Tai, Parton, Richard M., Antonello, Jacopo, Packer, Adam M., Davis, Ilan, Booth, Martin J.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10641083/
https://www.ncbi.nlm.nih.gov/pubmed/37953294
http://dx.doi.org/10.1038/s41377-023-01297-x
_version_ 1785146696395653120
author Hu, Qi
Hailstone, Martin
Wang, Jingyu
Wincott, Matthew
Stoychev, Danail
Atilgan, Huriye
Gala, Dalia
Chaiamarit, Tai
Parton, Richard M.
Antonello, Jacopo
Packer, Adam M.
Davis, Ilan
Booth, Martin J.
author_facet Hu, Qi
Hailstone, Martin
Wang, Jingyu
Wincott, Matthew
Stoychev, Danail
Atilgan, Huriye
Gala, Dalia
Chaiamarit, Tai
Parton, Richard M.
Antonello, Jacopo
Packer, Adam M.
Davis, Ilan
Booth, Martin J.
author_sort Hu, Qi
collection PubMed
description The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution – one that can be readily transferred between microscope modalities – has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a “black box”, but provided physical insights on internal workings, which could influence future designs.
format Online
Article
Text
id pubmed-10641083
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-106410832023-11-14 Universal adaptive optics for microscopy through embedded neural network control Hu, Qi Hailstone, Martin Wang, Jingyu Wincott, Matthew Stoychev, Danail Atilgan, Huriye Gala, Dalia Chaiamarit, Tai Parton, Richard M. Antonello, Jacopo Packer, Adam M. Davis, Ilan Booth, Martin J. Light Sci Appl Article The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution – one that can be readily transferred between microscope modalities – has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a “black box”, but provided physical insights on internal workings, which could influence future designs. Nature Publishing Group UK 2023-11-13 /pmc/articles/PMC10641083/ /pubmed/37953294 http://dx.doi.org/10.1038/s41377-023-01297-x Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Hu, Qi
Hailstone, Martin
Wang, Jingyu
Wincott, Matthew
Stoychev, Danail
Atilgan, Huriye
Gala, Dalia
Chaiamarit, Tai
Parton, Richard M.
Antonello, Jacopo
Packer, Adam M.
Davis, Ilan
Booth, Martin J.
Universal adaptive optics for microscopy through embedded neural network control
title Universal adaptive optics for microscopy through embedded neural network control
title_full Universal adaptive optics for microscopy through embedded neural network control
title_fullStr Universal adaptive optics for microscopy through embedded neural network control
title_full_unstemmed Universal adaptive optics for microscopy through embedded neural network control
title_short Universal adaptive optics for microscopy through embedded neural network control
title_sort universal adaptive optics for microscopy through embedded neural network control
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10641083/
https://www.ncbi.nlm.nih.gov/pubmed/37953294
http://dx.doi.org/10.1038/s41377-023-01297-x
work_keys_str_mv AT huqi universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT hailstonemartin universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT wangjingyu universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT wincottmatthew universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT stoychevdanail universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT atilganhuriye universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT galadalia universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT chaiamarittai universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT partonrichardm universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT antonellojacopo universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT packeradamm universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT davisilan universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol
AT boothmartinj universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol