Full Text:   <1770>

Summary:  <956>

CLC number: O439

On-line Access: 2021-10-08

Received: 2020-08-21

Revision Accepted: 2020-12-17

Crosschecked: 2021-01-25

Cited: 0

Clicked: 3227

Citations:  Bibtex RefMan EndNote GB/T7714


Wei-gong Zhang


Ke Si


-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2021 Vol.22 No.10 P.1277-1288


Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes

Author(s):  Shuwen Hu, Lejia Hu, Wei Gong, Zhenghan Li, Ke Si

Affiliation(s):  Department of Neurology of the First Affiliated Hospital, State Key Laboratory of Modern Optical Instrumentation, Zhejiang University School of Medicine, Hangzhou 310009, China; more

Corresponding email(s):   weigong@zju.edu.cn, kesi@zju.edu.cn

Key Words:  Adaptive optics, Wavefront detection, Deep learning, Zernike coefficients, Microscopy

Share this article to: More |Next Article >>>

Shuwen Hu, Lejia Hu, Wei Gong, Zhenghan Li, Ke Si. Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(10): 1277-1288.

@article{title="Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes",
author="Shuwen Hu, Lejia Hu, Wei Gong, Zhenghan Li, Ke Si",
journal="Frontiers of Information Technology & Electronic Engineering",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes
%A Shuwen Hu
%A Lejia Hu
%A Wei Gong
%A Zhenghan Li
%A Ke Si
%J Frontiers of Information Technology & Electronic Engineering
%V 22
%N 10
%P 1277-1288
%@ 2095-9184
%D 2021
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2000422

T1 - Deep learning based wavefront sensor for complex wavefront detection in adaptive optical microscopes
A1 - Shuwen Hu
A1 - Lejia Hu
A1 - Wei Gong
A1 - Zhenghan Li
A1 - Ke Si
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 22
IS - 10
SP - 1277
EP - 1288
%@ 2095-9184
Y1 - 2021
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2000422

The Shack-Hartmann wavefront sensor (SHWS) is an essential tool for wavefront sensing in adaptive optical microscopes. However, the distorted spots induced by the complex wavefront challenge its detection performance. Here, we propose a deep learning based wavefront detection method which combines point spread function image based Zernike coefficient estimation and wavefront stitching. Rather than using the centroid displacements of each micro-lens, this method first estimates the zernike coefficients of local wavefront distribution over each micro-lens and then stitches the local wavefronts for reconstruction. The proposed method can offer low root mean square wavefront errors and high accuracy for complex wavefront detection, and has potential to be applied in adaptive optical microscopes.




Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]Booth MJ, 2014. Adaptive optical microscopy: the ongoing quest for a perfect image. Light Sci Appl, 3(4):e165.

[2]Booth MJ, Neil MAA, Juškaitis R, et al., 2002. Adaptive aberration correction in a confocal microscope. Proc Nat Acad Sci, 99(9):5788-5792.

[3]Cheng SF, Li HH, Luo YQ, et al., 2019. Artificial intelligence-assisted light control and computational imaging through scattering media. J Innov Opt Health Sci, 12(4):193006.

[4]Cornea A, Conn PM, 2014. Fluorescence Microscopy: Super Resolution and Other Novel Techniques. Elsevier, London, UK, p.249.

[5]Cui M, 2011. Parallel wavefront optimization method for focusing light through random scattering media. Opt Lett, 36(6):870-872.

[6]Cumming BP, Gu M, 2020. Direct determination of aberration functions in microscopy by an artificial neural network. Opt Expr, 28(10):14511-14521.

[7]Dai GM, 2008. Wavefront Optics for Vision Correction. SPIE Press, Bellingham, USA.

[8]Drozdzal M, Vorontsov E, Chartrand G, et al., 2016. The importance of skip connections in biomedical image segmentation. Int Workshop on Deep Learning in Medical Image Analysis and Int Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, p.179-187.

[9]Dubose TB, Gardner DF, Watnik AT, 2020. Intensity-enhanced deep network wavefront reconstruction in Shack–Hartmann sensors. Opt Lett, 45(7):1699-1702.

[10]Gómez SLS, González-Gutiérrez C, Alonso ED, et al., 2018. Improving adaptive optics reconstructions with a deep learning approach. Int Conf on Hybrid Artificial Intelligence Systems, p.74-83.

[11]Hu LJ, Hu SW, Gong W, et al., 2019. Learning-based Shack-Hartmann wavefront sensor for high-order aberration detection. Opt Expr, 27(23):33504-33517.

[12]Hu LJ, Hu SW, Li YN, et al., 2020. Reliability of wavefront shaping based on coherent optical adaptive technique in deep tissue focusing. J Biophoton, 13(1):e201900245.

[13]Hu SW, Hu LJ, Zhang BW, et al., 2020. Simplifying the detection of optical distortions by machine learning. J Innov Opt Health Sci, 13(3):2040001.

[14]Ji N, 2017. Adaptive optical fluorescence microscopy. Nat Methods, 14(4):374-280.

[15]Jin YC, Zhang YY, Hu LJ, et al., 2018. Machine learning guided rapid focusing with sensor-less aberration corrections. Opt Expr, 26(23):30162-30171.

[16]Li ZH, Yu ZP, Hui H, et al., 2020. Edge enhancement through scattering media enabled by optical wavefront shaping. Photon Res, 8(6):954-962.

[17]Liu R, Li ZY, Marvin JS, et al., 2019. Direct wavefront sensing enables functional imaging of infragranular axons and spines. Nat Methods, 16(7):615-618.

[18]Liu TL, Upadhyayula S, Milkie DE, et al., 2018. Observing the cell in its native state: imaging subcellular dynamics in multicellular organisms. Science, 360(6386):eaaq1392.

[19]Mahajan VN, Dai GM, 2007. Orthonormal polynomials in wavefront analysis: analytical solution. J Opt Soc Am A, 24(9):2994-3016.

[20]Nishizaki Y, Valdivia M, Horisaki R, et al., 2019. Deep learning wavefront sensing. Opt Expr, 27(1):240-251.

[21]Paine SW, Fienup JR, 2018. Machine learning for improved image-based wavefront sensing. Opt Lett, 43(6):1235-1238.

[22]Park JH, Kong LJ, Zhou YF, et al., 2017. Large-field-of-view imaging by multi-pupil adaptive optics. Nat Methods, 14(6):581-583.

[23]Rodríguez C, Ji N, 2018. Adaptive optical microscopy for neurobiology. Curr Opin Neurobiol, 50:83-91.

[24]Schott S, Bertolotti J, Léger JF, et al., 2015. Characterization of the angular memory effect of scattered light in biological tissues. Opt Expr, 23(10):13505-13516.

[25]Swanson R, Lamb M, Correia C, et al., 2018. Wavefront reconstruction and prediction with convolutional neural networks. Adaptive Optics Systems VI, Article 10703F.

[26]Tang JY, Germain RN, Cui M, 2012. Superpenetration optical microscopy by iterative multiphoton adaptive compensation technique. Proc Nat Acad Sci, 109(22):8434-8439.

[27]Vanberg PO, de Xivry GO, Absil O, et al., 2019. Machine learning for image-based wavefront sensing. 33rd Conf on Neural Information Processing Systems, p.1-6.

[28]Wang BK, Barbiero M, Zhang QM, et al., 2019. Super-resolution optical microscope: principle, instrumentation, and application. Front Inform Technol Electron Eng, 20(5):608-630.

[29]Wang K, Milkie DE, Saxena A, et al., 2014. Rapid adaptive optical recovery of optimal resolution over large volumes. Nat Methods, 11(6):625-628.

[30]Wang K, Sun WZ, Richie CT, et al., 2015. Direct wavefront sensing for high-resolution in vivo imaging in scattering tissue. Nat Commun, 6:7276.

[31]Yoon J, Lee M, Lee K, et al., 2015. Optogenetic control of cell signaling pathway through scattering skull using wavefront shaping. Sci Rep, 5:13289.

[32]Yu ZP, Xia MY, Li HH, et al., 2019. Implementation of digital optical phase conjugation with embedded calibration and phase rectification. Sci Rep, 9(1):1537.

[33]Zeng ZP, Xie H, Chen L, et al., 2017. Computational methods in super-resolution microscopy. Front Inform Technol Electron Eng, 18(9):1222-1235.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2022 Journal of Zhejiang University-SCIENCE