Li ZS, Sun JS, Fan Y, Jin YB, Shen Q et al. Deep learning assisted variational Hilbert quantitative phase imaging. Opto-Electron Sci 2, 220023 (2023). doi: 10.29026/oes.2023.220023
Citation: Li ZS, Sun JS, Fan Y, Jin YB, Shen Q et al. Deep learning assisted variational Hilbert quantitative phase imaging. Opto-Electron Sci 2, 220023 (2023). doi: 10.29026/oes.2023.220023

Article Open Access

Deep learning assisted variational Hilbert quantitative phase imaging

More Information
  • We propose a high-accuracy artifacts-free single-frame digital holographic phase demodulation scheme for relatively low-carrier frequency holograms—deep learning assisted variational Hilbert quantitative phase imaging (DL-VHQPI). The method, incorporating a conventional deep neural network into a complete physical model utilizing the idea of residual compensation, reliably and robustly recovers the quantitative phase information of the test objects. It can significantly alleviate spectrum-overlapping-caused phase artifacts under the slightly off-axis digital holographic system. Compared to the conventional end-to-end networks (without a physical model), the proposed method can reduce the dataset size dramatically while maintaining the imaging quality and model generalization. The DL-VHQPI is quantitatively studied by numerical simulation. The live-cell experiment is designed to demonstrate the method's practicality in biological research. The proposed idea of the deep learning-assisted physical model might be extended to diverse computational imaging techniques.
  • 加载中
  • [1] Fan Y, Li JJ, Lu LP, Sun JS, Hu Y et al. Smart computational light microscopes (SCLMs) of smart computational imaging laboratory (SCILab). PhotoniX 2, 19 (2021). doi: 10.1186/s43074-021-00040-2

    CrossRef Google Scholar

    [2] Lee K, Kim K, Jung J, Heo J, Cho S et al. Quantitative phase imaging techniques for the study of cell pathophysiology: from principles to applications. Sensors 13, 4170–4191 (2013). doi: 10.3390/s130404170

    CrossRef Google Scholar

    [3] Park Y, Depeursinge C, Popescu G. Quantitative phase imaging in biomedicine. Nat Photonics 12, 578–589 (2018). doi: 10.1038/s41566-018-0253-x

    CrossRef Google Scholar

    [4] Vicar T, Balvan J, Jaros J, Jug F, Kolar R et al. Cell segmentation methods for label-free contrast microscopy: review and comprehensive comparison. BMC Bioinformatics 20, 360 (2019). doi: 10.1186/s12859-019-2880-8

    CrossRef Google Scholar

    [5] Gao P, Wirth R, Lackner J, Sunbul M, Jaeschke A et al. Superresolution imaging of live cells with genetically encoded silicon rhodamine-binding RNA aptamers. Biophys J 118, 145A (2020). doi: 10.1016/j.bpj.2019.11.916

    CrossRef Google Scholar

    [6] Li ZS, Fan Y, Sun JS, Zuo C, Chen Q. A commercialized digital holographic microscope with complete software supporting. Proc SPIE 11571, 115711C (2020). doi: 10.1117/12.2581220

    CrossRef Google Scholar

    [7] Kim MK. Principles and techniques of digital holographic microscopy. SPIE Rev 1, 018005 (2010). doi: 10.1117/6.0000006

    CrossRef Google Scholar

    [8] Kemper B, von Bally G. Digital holographic microscopy for live cell applications and technical inspection. Appl Opt 47, A52–A61 (2008). doi: 10.1364/AO.47.000A52

    CrossRef Google Scholar

    [9] Gao P, Yuan CJ. Resolution enhancement of digital holographic microscopy via synthetic aperture: a review. Light Adv Manuf 3, 105–120 (2022). doi: 10.37188/lam.2022.006

    CrossRef Google Scholar

    [10] Bettenworth D, Lenz P, Krausewitz P, Brückner M, Ketelhut S et al. Quantitative stain-free and continuous multimodal monitoring of wound healing in vitro with digital holographic microscopy. PLoS One 9, e107317 (2014). doi: 10.1371/journal.pone.0107317

    CrossRef Google Scholar

    [11] Coppola G, Ferraro P, Iodice M, De Nicola S, Finizio A et al. A digital holographic microscope for complete characterization of microelectromechanical systems. Meas Sci Technol 15, 529–539 (2004). doi: 10.1088/0957-0233/15/3/005

    CrossRef Google Scholar

    [12] Anand V, Han ML, Maksimovic J, Ng SH, Katkus T et al. Single-shot mid-infrared incoherent holography using Lucy-Richardson-Rosen algorithm. Opto-Electron Sci 1, 210006 (2022). doi: 10.29026/oes.2022.210006

    CrossRef Google Scholar

    [13] Xu K, Wang X E, Fan X H et al. Meta-holography: from concept to realization. Opto-Electron Eng 49, 220183 (2022). doi: 10.12086/oee.2022.220183

    CrossRef Google Scholar

    [14] Gao H, Fan XH, Xiong W, Hong MH. Recent advances in optical dynamic meta-holography. Opto-Electron Adv 4, 210030 (2021). doi: 10.29026/oea.2021.210030

    CrossRef Google Scholar

    [15] Gabai H, Baranes-Zeevi M, Zilberman M, Shaked NT. Continuous wide-field characterization of drug release from skin substitute using off-axis interferometry. Opt Lett 38, 3017–3020 (2013). doi: 10.1364/OL.38.003017

    CrossRef Google Scholar

    [16] Huang ZZ, Memmolo P, Ferraro P, Cao LC. Dual-plane coupled phase retrieval for non-prior holographic imaging. PhotoniX 3, 3 (2022). doi: 10.1186/s43074-021-00046-w

    CrossRef Google Scholar

    [17] Wu XJ, Sun JS, Zhang JL, Lu LP, Chen R et al. Wavelength-scanning lensfree on-chip microscopy for wide-field pixel-super-resolved quantitative phase imaging. Opt Lett 46, 2023–2026 (2021). doi: 10.1364/OL.421869

    CrossRef Google Scholar

    [18] Wang HD, Göröcs Z, Luo W, Zhang YB, Rivenson Y et al. Computational out-of-focus imaging increases the space–bandwidth product in lens-based coherent microscopy. Optica 3, 1422–1429 (2016). doi: 10.1364/OPTICA.3.001422

    CrossRef Google Scholar

    [19] Micó V, García J, Zalevsky Z, Javidi B. Phase-shifting Gabor holography. Opt Lett 34, 1492–1494 (2009). doi: 10.1364/OL.34.001492

    CrossRef Google Scholar

    [20] Poon TC. Digital Holography and Three-Dimensional Display: Principles and Applications (Springer, New York, 2006).

    Google Scholar

    [21] Claus D, Iliescu D, Bryanston-Cross P. Quantitative space-bandwidth product analysis in digital holography. Appl Opt 50, H116–H127 (2011). doi: 10.1364/AO.50.00H116

    CrossRef Google Scholar

    [22] Zhong Z, Bai HY, Shan MG, Zhang YB, Guo LL. Fast phase retrieval in slightly off-axis digital holography. Opt Lasers Eng 97, 9–18 (2017). doi: 10.1016/j.optlaseng.2017.05.004

    CrossRef Google Scholar

    [23] Xue L, Lai JC, Wang SY, Li ZH. Single-shot slightly-off-axis interferometry based Hilbert phase microscopy of red blood cells. Biomed Opt Express 2, 987–995 (2011). doi: 10.1364/BOE.2.000987

    CrossRef Google Scholar

    [24] Shaked NT, Zhu YZ, Rinehart MT, Wax A. Two-step-only phase-shifting interferometry with optimized detector bandwidth for microscopy of live cells. Opt Express 17, 15585–15591 (2009). doi: 10.1364/OE.17.015585

    CrossRef Google Scholar

    [25] Pavillon N, Arfire C, Bergoënd I, Depeursinge C. Iterative method for zero-order suppression in off-axis digital holography. Opt Express 18, 15318–15331 (2010). doi: 10.1364/OE.18.015318

    CrossRef Google Scholar

    [26] Trusiak M, Picazo-Bueno JA, Patorski K, Zdankowski P, Mico V. Single-shot two-frame π-shifted spatially multiplexed interference phase microscopy. J Biomed Opt 24, 096004 (2019). doi: 10.1117/1.JBO.24.9.096004

    CrossRef Google Scholar

    [27] León-Rodríguez M, Rayas JA, Cordero RR, Martínez-García A, Martínez-Gonzalez A et al. Dual-plane slightly off-axis digital holography based on a single cube beam splitter. Appl Opt 57, 2727–2735 (2018). doi: 10.1364/AO.57.002727

    CrossRef Google Scholar

    [28] Han JH, Gao P, Yao BL, Gu YZ, Huang MJ. Slightly off-axis interferometry for microscopy with second wavelength assistance. Appl Opt 50, 2793–2798 (2011). doi: 10.1364/AO.50.002793

    CrossRef Google Scholar

    [29] Ikeda T, Popescu G, Dasari RR, Feld MS. Hilbert phase microscopy for investigating fast dynamics in transparent systems. Opt Lett 30, 1165–1167 (2005). doi: 10.1364/OL.30.001165

    CrossRef Google Scholar

    [30] Guo CS, Wang BY, Sha B, Lu YJ, Xu MY. Phase derivative method for reconstruction of slightly off-axis digital holograms. Opt Express 22, 30553–30558 (2014). doi: 10.1364/OE.22.030553

    CrossRef Google Scholar

    [31] Pavillon N, Seelamantula CS, Kühn J, Unser M, Depeursinge C. Suppression of the zero-order term in off-axis digital holography through nonlinear filtering. Appl Opt 48, H186–H195 (2009). doi: 10.1364/AO.48.00H186

    CrossRef Google Scholar

    [32] Baek Y, Lee K, Shin S, Park Y. Kramers–Kronig holographic imaging for high-space-bandwidth product. Optica 6, 45–51 (2019). doi: 10.1364/OPTICA.6.000045

    CrossRef Google Scholar

    [33] Baek Y, Park Y. Intensity-based holographic imaging via space-domain Kramers–Kronig relations. Nat Photonics 15, 354–360 (2021). doi: 10.1038/s41566-021-00760-8

    CrossRef Google Scholar

    [34] Trusiak M, Cywińska M, Micó V, Picazo-Bueno JÁ, Zuo C et al. Variational Hilbert quantitative phase imaging. Sci Rep 10, 13955 (2020). doi: 10.1038/s41598-020-69717-1

    CrossRef Google Scholar

    [35] Cywińska M, Trusiak M, Patorski K. Automatized fringe pattern preprocessing using unsupervised variational image decomposition. Opt Express 27, 22542–22562 (2019). doi: 10.1364/OE.27.022542

    CrossRef Google Scholar

    [36] Larkin KG, Bone DJ, Oldfield MA. Natural demodulation of two-dimensional fringe patterns. I. General background of the spiral phase quadrature transform. J Opt Soc Am A 18, 1862–1870 (2001). doi: 10.1364/JOSAA.18.001862

    CrossRef Google Scholar

    [37] Zuo C, Qian JM, Feng SJ, Yin W, Li YX et al. Deep learning in optical metrology: a review. Light Sci Appl 11, 39 (2022). doi: 10.1038/s41377-022-00714-x

    CrossRef Google Scholar

    [38] Feng SJ, Chen Q, Gu GH, Tao TY, Zhang L et al. Fringe pattern analysis using deep learning. Adv Photonics 1, 025001 (2019). doi: 10.1117/1.AP.1.2.025001

    CrossRef Google Scholar

    [39] Feng SJ, Zuo C, Hu Y, Li YX, Chen Q. Deep-learning-based fringe-pattern analysis with uncertainty estimation. Optica 8, 1507–1510 (2021). doi: 10.1364/OPTICA.434311

    CrossRef Google Scholar

    [40] Cywińska M, Brzeski F, Krajnik W, Patorski K, Zuo C et al. DeepDensity: convolutional neural network based estimation of local fringe pattern density. Opt Lasers Eng 145, 106675 (2021). doi: 10.1016/j.optlaseng.2021.106675

    CrossRef Google Scholar

    [41] Pan B. Optical metrology embraces deep learning: keeping an open mind. Light Sci Appl 11, 139 (2022). doi: 10.1038/s41377-022-00829-1

    CrossRef Google Scholar

    [42] Zuo C, Qian JM, Feng SJ, Yin W, Li YX et al. Correction: deep learning in optical metrology: a review. Light Sci Appl 11, 74 (2022). doi: 10.1038/s41377-022-00757-0

    CrossRef Google Scholar

    [43] Feng SJ, Zuo C, Zhang L, Yin W, Chen Q. Generalized framework for non-sinusoidal fringe analysis using deep learning. Photonics Res 9, 1084–1098 (2021). doi: 10.1364/PRJ.420944

    CrossRef Google Scholar

    [44] Li YX, Qian JM, Feng SJ, Chen Q, Zuo C. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5, 210021 (2022). doi: 10.29026/oea.2022.210021

    CrossRef Google Scholar

    [45] Zheng CH, Wang TS, Liu ZQ et al. Deep transfer learning method to identify orbital angular momentum beams. Opto-Electron Eng 49, 210409 (2022). doi: 10.12086/oee.2022.210409

    CrossRef Google Scholar

    [46] Zheng ZH, Zhu SK, Chen Y, Chen HY, Chen JH. Towards integrated mode-division demultiplexing spectrometer by deep learning. Opto-Electron Sci 1, 220012 (2022). doi: 10.29026/oes.2022.220012

    CrossRef Google Scholar

    [47] Rivenson Y, Zhang YB, Günaydın H, Teng D, Ozcan A. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci Appl 7, 17141 (2018). doi: 10.1038/lsa.2017.141

    CrossRef Google Scholar

    [48] Rivenson Y, Wu YC, Ozcan A. Deep learning in holography and coherent imaging. Light Sci Appl 8, 85 (2019). doi: 10.1038/s41377-019-0196-0

    CrossRef Google Scholar

    [49] Chen HL, Huang LZ, Liu TR, Ozcan A. Fourier Imager Network (FIN): a deep neural network for hologram reconstruction with superior external generalization. Light Sci Appl 11, 254 (2022). doi: 10.1038/s41377-022-00949-8

    CrossRef Google Scholar

    [50] Lempitsky V, Vedaldi A, Ulyanov D. Deep image prior. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 9446–9454 (IEEE, 2018);http://doi.org/10.1109/CVPR.2018.00984.

    Google Scholar

    [51] Wang F, Bian YM, Wang HC, Lyu M, Pedrini G et al. Phase imaging with an untrained neural network. Light Sci Appl 9, 77 (2020). doi: 10.1038/s41377-020-0302-3

    CrossRef Google Scholar

    [52] Duran J, Coll B, Sbert C. Chambolle’s projection algorithm for total variation denoising. Image Process Line 3, 311–331 (2013). doi: 10.5201/ipol.2013.61

    CrossRef Google Scholar

    [53] Zhu XJ, Chen ZQ, Tang C. Variational image decomposition for automatic background and noise removal of fringe patterns. Opt Lett 38, 275–277 (2013). doi: 10.1364/OL.38.000275

    CrossRef Google Scholar

    [54] Bianco V, Memmolo P, Paturzo M, Finizio A, Javidi B et al. Quasi noise-free digital holography. Light Sci Appl 5, e16142 (2016). doi: 10.1038/lsa.2016.142

    CrossRef Google Scholar

    [55] Kluver JW. Elimination of slip and instability effects in certain M-type electron beams. Proc IEEE 51, 868–868 (1963). doi: 10.1109/proc.1963.2309

    CrossRef Google Scholar

    [56] Yang X, Yu QF, Fu SH. A combined method for obtaining fringe orientations of ESPI. Opt Commun 273, 60–66 (2007). doi: 10.1016/j.optcom.2006.12.026

    CrossRef Google Scholar

    [57] Deng M, Li S, Zhang ZY, Kang I, Fang NX et al. On the interplay between physical and content priors in deep learning for computational imaging. Opt Express 28, 24152–24170 (2020). doi: 10.1364/OE.395204

    CrossRef Google Scholar

    [58] Shannon CE. A mathematical theory of communication. ACM SIGMOBILE Mob Comput Commun Rev 5, 3–55 (2001). doi: 10.1145/584091.584093

    CrossRef Google Scholar

    [59] Cover TM. Elements of Information Theory. John Wiley & Sons, 1999).

    Google Scholar

    [60] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning (JMLR. org, 2015).

    Google Scholar

    [61] Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning 807–814 (Omnipress, 2010).

    Google Scholar

    [62] Kingma DP, Ba J. Adam: a method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations. https://arxiv.org/abs/1412.6980 (2015).

    Google Scholar

    [63] Choi W, Fang-Yen C, Oh S, Lue N, Dasari RR et al. Tomographic phase microscopy: quantitative 3D-mapping of refractive index in live cells. Imaging Microsc 10, 48–50 (2008).

    Google Scholar

    [64] Sung Y, Choi W, Fang-Yen C, Badizadegan K, Dasari RR et al. Optical diffraction tomography for high resolution live cell imaging. Opt Express 17, 266–277 (2009). doi: 10.1364/OE.17.000266

    CrossRef Google Scholar

    [65] Li JJ, Matlock AC, Li YZ, Chen Q, Zuo C et al. High-speed in vitro intensity diffraction tomography. Adv Photonics 1, 066004 (2019). doi: 10.1117/1.AP.1.6.066004

    CrossRef Google Scholar

    [66] Mico V, Zalevsky Z, García J. Superresolution optical system by common-path interferometry. Opt Express 14, 5168–5177 (2006). doi: 10.1364/OE.14.005168

    CrossRef Google Scholar

    [67] Zhang JW, Dai SQ, Ma CJ, Xi TL, Di JL et al. A review of common-path off-axis digital holography: towards high stable optical instrument manufacturing. Light Adv Manuf 2, 333–349 (2021). doi: 10.37188/lam.2021.023

    CrossRef Google Scholar

  • Supplementary information for Deep learning assisted variational Hilbert quantitative phase imaging
  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(4)

Tables(1)

Article Metrics

Article views(6783) PDF downloads(586) Cited by(0)

Access History
Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint