E-mail Alert
RSS
| Citation: |
Li ZS, Sun JS, Fan Y, Jin YB, Shen Q et al. Deep learning assisted variational Hilbert quantitative phase imaging. Opto-Electron Sci 2, 220023 (2023). doi: 10.29026/oes.2023.220023
|
Article Open Access
Deep learning assisted variational Hilbert quantitative phase imaging
-
Abstract
We propose a high-accuracy artifacts-free single-frame digital holographic phase demodulation scheme for relatively low-carrier frequency holograms—deep learning assisted variational Hilbert quantitative phase imaging (DL-VHQPI). The method, incorporating a conventional deep neural network into a complete physical model utilizing the idea of residual compensation, reliably and robustly recovers the quantitative phase information of the test objects. It can significantly alleviate spectrum-overlapping-caused phase artifacts under the slightly off-axis digital holographic system. Compared to the conventional end-to-end networks (without a physical model), the proposed method can reduce the dataset size dramatically while maintaining the imaging quality and model generalization. The DL-VHQPI is quantitatively studied by numerical simulation. The live-cell experiment is designed to demonstrate the method's practicality in biological research. The proposed idea of the deep learning-assisted physical model might be extended to diverse computational imaging techniques. -
-
References
[1] Fan Y, Li JJ, Lu LP, Sun JS, Hu Y et al. Smart computational light microscopes (SCLMs) of smart computational imaging laboratory (SCILab). PhotoniX 2, 19 (2021). doi: 10.1186/s43074-021-00040-2 [2] Lee K, Kim K, Jung J, Heo J, Cho S et al. Quantitative phase imaging techniques for the study of cell pathophysiology: from principles to applications. Sensors 13, 4170–4191 (2013). doi: 10.3390/s130404170 [3] Park Y, Depeursinge C, Popescu G. Quantitative phase imaging in biomedicine. Nat Photonics 12, 578–589 (2018). doi: 10.1038/s41566-018-0253-x [4] Vicar T, Balvan J, Jaros J, Jug F, Kolar R et al. Cell segmentation methods for label-free contrast microscopy: review and comprehensive comparison. BMC Bioinformatics 20, 360 (2019). doi: 10.1186/s12859-019-2880-8 [5] Gao P, Wirth R, Lackner J, Sunbul M, Jaeschke A et al. Superresolution imaging of live cells with genetically encoded silicon rhodamine-binding RNA aptamers. Biophys J 118, 145A (2020). doi: 10.1016/j.bpj.2019.11.916 [6] Li ZS, Fan Y, Sun JS, Zuo C, Chen Q. A commercialized digital holographic microscope with complete software supporting. Proc SPIE 11571, 115711C (2020). doi: 10.1117/12.2581220 [7] Kim MK. Principles and techniques of digital holographic microscopy. SPIE Rev 1, 018005 (2010). doi: 10.1117/6.0000006 [8] Kemper B, von Bally G. Digital holographic microscopy for live cell applications and technical inspection. Appl Opt 47, A52–A61 (2008). doi: 10.1364/AO.47.000A52 [9] Gao P, Yuan CJ. Resolution enhancement of digital holographic microscopy via synthetic aperture: a review. Light Adv Manuf 3, 105–120 (2022). doi: 10.37188/lam.2022.006 [10] Bettenworth D, Lenz P, Krausewitz P, Brückner M, Ketelhut S et al. Quantitative stain-free and continuous multimodal monitoring of wound healing in vitro with digital holographic microscopy. PLoS One 9, e107317 (2014). doi: 10.1371/journal.pone.0107317 [11] Coppola G, Ferraro P, Iodice M, De Nicola S, Finizio A et al. A digital holographic microscope for complete characterization of microelectromechanical systems. Meas Sci Technol 15, 529–539 (2004). doi: 10.1088/0957-0233/15/3/005 [12] Anand V, Han ML, Maksimovic J, Ng SH, Katkus T et al. Single-shot mid-infrared incoherent holography using Lucy-Richardson-Rosen algorithm. Opto-Electron Sci 1, 210006 (2022). doi: 10.29026/oes.2022.210006 [13] Xu K, Wang X E, Fan X H et al. Meta-holography: from concept to realization. Opto-Electron Eng 49, 220183 (2022). doi: 10.12086/oee.2022.220183 [14] Gao H, Fan XH, Xiong W, Hong MH. Recent advances in optical dynamic meta-holography. Opto-Electron Adv 4, 210030 (2021). doi: 10.29026/oea.2021.210030 [15] Gabai H, Baranes-Zeevi M, Zilberman M, Shaked NT. Continuous wide-field characterization of drug release from skin substitute using off-axis interferometry. Opt Lett 38, 3017–3020 (2013). doi: 10.1364/OL.38.003017 [16] Huang ZZ, Memmolo P, Ferraro P, Cao LC. Dual-plane coupled phase retrieval for non-prior holographic imaging. PhotoniX 3, 3 (2022). doi: 10.1186/s43074-021-00046-w [17] Wu XJ, Sun JS, Zhang JL, Lu LP, Chen R et al. Wavelength-scanning lensfree on-chip microscopy for wide-field pixel-super-resolved quantitative phase imaging. Opt Lett 46, 2023–2026 (2021). doi: 10.1364/OL.421869 [18] Wang HD, Göröcs Z, Luo W, Zhang YB, Rivenson Y et al. Computational out-of-focus imaging increases the space–bandwidth product in lens-based coherent microscopy. Optica 3, 1422–1429 (2016). doi: 10.1364/OPTICA.3.001422 [19] Micó V, García J, Zalevsky Z, Javidi B. Phase-shifting Gabor holography. Opt Lett 34, 1492–1494 (2009). doi: 10.1364/OL.34.001492 [20] Poon TC. Digital Holography and Three-Dimensional Display: Principles and Applications (Springer, New York, 2006). [21] Claus D, Iliescu D, Bryanston-Cross P. Quantitative space-bandwidth product analysis in digital holography. Appl Opt 50, H116–H127 (2011). doi: 10.1364/AO.50.00H116 [22] Zhong Z, Bai HY, Shan MG, Zhang YB, Guo LL. Fast phase retrieval in slightly off-axis digital holography. Opt Lasers Eng 97, 9–18 (2017). doi: 10.1016/j.optlaseng.2017.05.004 [23] Xue L, Lai JC, Wang SY, Li ZH. Single-shot slightly-off-axis interferometry based Hilbert phase microscopy of red blood cells. Biomed Opt Express 2, 987–995 (2011). doi: 10.1364/BOE.2.000987 [24] Shaked NT, Zhu YZ, Rinehart MT, Wax A. Two-step-only phase-shifting interferometry with optimized detector bandwidth for microscopy of live cells. Opt Express 17, 15585–15591 (2009). doi: 10.1364/OE.17.015585 [25] Pavillon N, Arfire C, Bergoënd I, Depeursinge C. Iterative method for zero-order suppression in off-axis digital holography. Opt Express 18, 15318–15331 (2010). doi: 10.1364/OE.18.015318 [26] Trusiak M, Picazo-Bueno JA, Patorski K, Zdankowski P, Mico V. Single-shot two-frame π-shifted spatially multiplexed interference phase microscopy. J Biomed Opt 24, 096004 (2019). doi: 10.1117/1.JBO.24.9.096004 [27] León-Rodríguez M, Rayas JA, Cordero RR, Martínez-García A, Martínez-Gonzalez A et al. Dual-plane slightly off-axis digital holography based on a single cube beam splitter. Appl Opt 57, 2727–2735 (2018). doi: 10.1364/AO.57.002727 [28] Han JH, Gao P, Yao BL, Gu YZ, Huang MJ. Slightly off-axis interferometry for microscopy with second wavelength assistance. Appl Opt 50, 2793–2798 (2011). doi: 10.1364/AO.50.002793 [29] Ikeda T, Popescu G, Dasari RR, Feld MS. Hilbert phase microscopy for investigating fast dynamics in transparent systems. Opt Lett 30, 1165–1167 (2005). doi: 10.1364/OL.30.001165 [30] Guo CS, Wang BY, Sha B, Lu YJ, Xu MY. Phase derivative method for reconstruction of slightly off-axis digital holograms. Opt Express 22, 30553–30558 (2014). doi: 10.1364/OE.22.030553 [31] Pavillon N, Seelamantula CS, Kühn J, Unser M, Depeursinge C. Suppression of the zero-order term in off-axis digital holography through nonlinear filtering. Appl Opt 48, H186–H195 (2009). doi: 10.1364/AO.48.00H186 [32] Baek Y, Lee K, Shin S, Park Y. Kramers–Kronig holographic imaging for high-space-bandwidth product. Optica 6, 45–51 (2019). doi: 10.1364/OPTICA.6.000045 [33] Baek Y, Park Y. Intensity-based holographic imaging via space-domain Kramers–Kronig relations. Nat Photonics 15, 354–360 (2021). doi: 10.1038/s41566-021-00760-8 [34] Trusiak M, Cywińska M, Micó V, Picazo-Bueno JÁ, Zuo C et al. Variational Hilbert quantitative phase imaging. Sci Rep 10, 13955 (2020). doi: 10.1038/s41598-020-69717-1 [35] Cywińska M, Trusiak M, Patorski K. Automatized fringe pattern preprocessing using unsupervised variational image decomposition. Opt Express 27, 22542–22562 (2019). doi: 10.1364/OE.27.022542 [36] Larkin KG, Bone DJ, Oldfield MA. Natural demodulation of two-dimensional fringe patterns. I. General background of the spiral phase quadrature transform. J Opt Soc Am A 18, 1862–1870 (2001). doi: 10.1364/JOSAA.18.001862 [37] Zuo C, Qian JM, Feng SJ, Yin W, Li YX et al. Deep learning in optical metrology: a review. Light Sci Appl 11, 39 (2022). doi: 10.1038/s41377-022-00714-x [38] Feng SJ, Chen Q, Gu GH, Tao TY, Zhang L et al. Fringe pattern analysis using deep learning. Adv Photonics 1, 025001 (2019). doi: 10.1117/1.AP.1.2.025001 [39] Feng SJ, Zuo C, Hu Y, Li YX, Chen Q. Deep-learning-based fringe-pattern analysis with uncertainty estimation. Optica 8, 1507–1510 (2021). doi: 10.1364/OPTICA.434311 [40] Cywińska M, Brzeski F, Krajnik W, Patorski K, Zuo C et al. DeepDensity: convolutional neural network based estimation of local fringe pattern density. Opt Lasers Eng 145, 106675 (2021). doi: 10.1016/j.optlaseng.2021.106675 [41] Pan B. Optical metrology embraces deep learning: keeping an open mind. Light Sci Appl 11, 139 (2022). doi: 10.1038/s41377-022-00829-1 [42] Zuo C, Qian JM, Feng SJ, Yin W, Li YX et al. Correction: deep learning in optical metrology: a review. Light Sci Appl 11, 74 (2022). doi: 10.1038/s41377-022-00757-0 [43] Feng SJ, Zuo C, Zhang L, Yin W, Chen Q. Generalized framework for non-sinusoidal fringe analysis using deep learning. Photonics Res 9, 1084–1098 (2021). doi: 10.1364/PRJ.420944 [44] Li YX, Qian JM, Feng SJ, Chen Q, Zuo C. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5, 210021 (2022). doi: 10.29026/oea.2022.210021 [45] Zheng CH, Wang TS, Liu ZQ et al. Deep transfer learning method to identify orbital angular momentum beams. Opto-Electron Eng 49, 210409 (2022). doi: 10.12086/oee.2022.210409 [46] Zheng ZH, Zhu SK, Chen Y, Chen HY, Chen JH. Towards integrated mode-division demultiplexing spectrometer by deep learning. Opto-Electron Sci 1, 220012 (2022). doi: 10.29026/oes.2022.220012 [47] Rivenson Y, Zhang YB, Günaydın H, Teng D, Ozcan A. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci Appl 7, 17141 (2018). doi: 10.1038/lsa.2017.141 [48] Rivenson Y, Wu YC, Ozcan A. Deep learning in holography and coherent imaging. Light Sci Appl 8, 85 (2019). doi: 10.1038/s41377-019-0196-0 [49] Chen HL, Huang LZ, Liu TR, Ozcan A. Fourier Imager Network (FIN): a deep neural network for hologram reconstruction with superior external generalization. Light Sci Appl 11, 254 (2022). doi: 10.1038/s41377-022-00949-8 [50] Lempitsky V, Vedaldi A, Ulyanov D. Deep image prior. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 9446–9454 (IEEE, 2018);http://doi.org/10.1109/CVPR.2018.00984. [51] Wang F, Bian YM, Wang HC, Lyu M, Pedrini G et al. Phase imaging with an untrained neural network. Light Sci Appl 9, 77 (2020). doi: 10.1038/s41377-020-0302-3 [52] Duran J, Coll B, Sbert C. Chambolle’s projection algorithm for total variation denoising. Image Process Line 3, 311–331 (2013). doi: 10.5201/ipol.2013.61 [53] Zhu XJ, Chen ZQ, Tang C. Variational image decomposition for automatic background and noise removal of fringe patterns. Opt Lett 38, 275–277 (2013). doi: 10.1364/OL.38.000275 [54] Bianco V, Memmolo P, Paturzo M, Finizio A, Javidi B et al. Quasi noise-free digital holography. Light Sci Appl 5, e16142 (2016). doi: 10.1038/lsa.2016.142 [55] Kluver JW. Elimination of slip and instability effects in certain M-type electron beams. Proc IEEE 51, 868–868 (1963). doi: 10.1109/proc.1963.2309 [56] Yang X, Yu QF, Fu SH. A combined method for obtaining fringe orientations of ESPI. Opt Commun 273, 60–66 (2007). doi: 10.1016/j.optcom.2006.12.026 [57] Deng M, Li S, Zhang ZY, Kang I, Fang NX et al. On the interplay between physical and content priors in deep learning for computational imaging. Opt Express 28, 24152–24170 (2020). doi: 10.1364/OE.395204 [58] Shannon CE. A mathematical theory of communication. ACM SIGMOBILE Mob Comput Commun Rev 5, 3–55 (2001). doi: 10.1145/584091.584093 [59] Cover TM. Elements of Information Theory. John Wiley & Sons, 1999). [60] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning (JMLR. org, 2015). [61] Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning 807–814 (Omnipress, 2010). [62] Kingma DP, Ba J. Adam: a method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations. https://arxiv.org/abs/1412.6980 (2015). [63] Choi W, Fang-Yen C, Oh S, Lue N, Dasari RR et al. Tomographic phase microscopy: quantitative 3D-mapping of refractive index in live cells. Imaging Microsc 10, 48–50 (2008). [64] Sung Y, Choi W, Fang-Yen C, Badizadegan K, Dasari RR et al. Optical diffraction tomography for high resolution live cell imaging. Opt Express 17, 266–277 (2009). doi: 10.1364/OE.17.000266 [65] Li JJ, Matlock AC, Li YZ, Chen Q, Zuo C et al. High-speed in vitro intensity diffraction tomography. Adv Photonics 1, 066004 (2019). doi: 10.1117/1.AP.1.6.066004 [66] Mico V, Zalevsky Z, García J. Superresolution optical system by common-path interferometry. Opt Express 14, 5168–5177 (2006). doi: 10.1364/OE.14.005168 [67] Zhang JW, Dai SQ, Ma CJ, Xi TL, Di JL et al. A review of common-path off-axis digital holography: towards high stable optical instrument manufacturing. Light Adv Manuf 2, 333–349 (2021). doi: 10.37188/lam.2021.023 -
Supplementary Information
Supplementary information for Deep learning assisted variational Hilbert quantitative phase imaging
-
Access History
Figures(4)
Tables(1)
Article Metrics
Export File
Citation
Li ZS, Sun JS, Fan Y, Jin YB, Shen Q et al. Deep learning assisted variational Hilbert quantitative phase imaging. Opto-Electron Sci 2, 220023 (2023). doi: 10.29026/oes.2023.220023
Format
Content
DownLoad:
-
Figure 1.
Flow chart of slightly off-axis interferometric fringe demodulation based on VHQPI.
-
Figure 2.
Deep learning-assisted VHQPI. (a) Total network structure, combining uVID and HST with CNN respectively for phase reconstruction. (b) CNN1 takes a hologram as input and consists of three convolutional layers and a group of residual blocks to achieve compensation of background residuals by learning. (c) The CNN2 network structure is the same as CNN1, except that CNN2 combines the original hologram and the result of the first process into a two-channel input for advanced background compensation.
-
Figure 3.
The experiment results under the numerical simulation. (a) The FT method phase recovery result. (b) The phase recovery result of VHQPI. (c) The phase result reconstructed by DL-VHQPI. (d) The ground truth. (e–g) The difference between the phase results of the three methods (i.e. FT, VHQPI, DL-VHQPI) and the ground truth. (h) Quantitative error analysis of three methods. (i) The cross-section of the phase results of FT, VHQPI, DL-VHQPI, and ground truth, and (j1–j4) are the DIC views of the partially enlarged views of their corresponding phase maps respectively.
-
Figure 4.
Results of holographic experiments on HeLa cells. (a) Low-carrier-frequency high-contrast hologram collected by slightly off-axis interferometry system. (b) Corresponding spatial frequency spectrum. (c) The result of phase recovery by slightly off-axis holography using FT method under ×20 lens. (d) The result of phase recovery using DL-VHQPI. (e1–e4) and (f1–f4) correspond to the local amplification results of “Area1” and “Area2” for the two samples under different phase recovery methods. Where (e2, e4, f2, f4) are the corresponding DIC views, respectively. (g) and (h) The DIC views after partial magnification of the phase map in the corresponding red box. (i) The numerical distribution of the cross-section and detail-preservation feature of the DL-VHQPI.
