Yin W, Che YX, Li XS et al. Physics-informed deep learning for fringe pattern analysis. Opto-Electron Adv 7, 230034 (2024). doi: 10.29026/oea.2024.230034
Citation: Yin W, Che YX, Li XS et al. Physics-informed deep learning for fringe pattern analysis. Opto-Electron Adv 7, 230034 (2024). doi: 10.29026/oea.2024.230034

Article Open Access

Physics-informed deep learning for fringe pattern analysis

More Information
  • Recently, deep learning has yielded transformative success across optics and photonics, especially in optical metrology. Deep neural networks (DNNs) with a fully convolutional architecture (e.g., U-Net and its derivatives) have been widely implemented in an end-to-end manner to accomplish various optical metrology tasks, such as fringe denoising, phase unwrapping, and fringe analysis. However, the task of training a DNN to accurately identify an image-to-image transform from massive input and output data pairs seems at best naïve, as the physical laws governing the image formation or other domain expertise pertaining to the measurement have not yet been fully exploited in current deep learning practice. To this end, we introduce a physics-informed deep learning method for fringe pattern analysis (PI-FPA) to overcome this limit by integrating a lightweight DNN with a learning-enhanced Fourier transform profilometry (LeFTP) module. By parameterizing conventional phase retrieval methods, the LeFTP module embeds the prior knowledge in the network structure and the loss function to directly provide reliable phase results for new types of samples, while circumventing the requirement of collecting a large amount of high-quality data in supervised learning methods. Guided by the initial phase from LeFTP, the phase recovery ability of the lightweight DNN is enhanced to further improve the phase accuracy at a low computational cost compared with existing end-to-end networks. Experimental results demonstrate that PI-FPA enables more accurate and computationally efficient single-shot phase retrieval, exhibiting its excellent generalization to various unseen objects during training. The proposed PI-FPA presents that challenging issues in optical metrology can be potentially overcome through the synergy of physics-priors-based traditional tools and data-driven learning approaches, opening new avenues to achieve fast and accurate single-shot 3D imaging.
  • 加载中
  • [1] Gåsvik KJ. Optical Metrology 3rd ed (John Wiley & Sons, West Sussex, 2002).

    Google Scholar

    [2] Wyant JC, Creath K. Recent advances in interferometric optical testing. Laser Focus 21, 118–132 (1985). doi: 10.3103/S875669902004007X

    CrossRef Google Scholar

    [3] Kulkarni R, Rastogi P. Optical measurement techniques-A push for digitization. Opt Lasers Eng 87, 1–17 (2016). doi: 10.1016/j.optlaseng.2016.05.002

    CrossRef Google Scholar

    [4] Hariharan P. Basics of Interferometry 2nd ed (Elsevier, Amsterdam, 2010).

    Google Scholar

    [5] Schnars U, Falldorf C, Watson J et al. Digital Holography and Wavefront Sensing 2nd ed (Springer, Berlin Heidelberg, 2015).

    Google Scholar

    [6] Gorthi SS, Rastogi P. Fringe projection techniques: whither we are? Opt Lasers Eng 48, 133–140 (2010). doi: 10.1016/j.optlaseng.2009.09.001

    CrossRef Google Scholar

    [7] Geng J. Structured-light 3D surface imaging: a tutorial. Adv Opt Photon 3, 128–160 (2011). doi: 10.1364/AOP.3.000128

    CrossRef Google Scholar

    [8] Servin M, Quiroga JA, Padilla JM. Fringe Pattern Analysis for Optical Metrology: Theory, Algorithms, and Applications (John Wiley & Sons, Weinheim, 2014).

    Google Scholar

    [9] Su XY, Chen WJ. Fourier transform profilometry: a review. Opt Lasers Eng 35, 263–284 (2001). doi: 10.1016/S0143-8166(01)00023-9

    CrossRef Google Scholar

    [10] Kemao Q. Windowed fourier transform for fringe pattern analysis. Appl Opt 43, 2695–2702 (2004). doi: 10.1364/AO.43.002695

    CrossRef Google Scholar

    [11] Zuo C, Feng SJ, Hiang L et al. Phase shifting algorithms for fringe projection profilometry: a review. Opt Lasers Eng 109, 23–59 (2018). doi: 10.1016/j.optlaseng.2018.04.019

    CrossRef Google Scholar

    [12] Barbastathis G, Ozcan A, Situ G. On the use of deep learning for computational imaging. Optica 6, 921–943 (2019). doi: 10.1364/OPTICA.6.000921

    CrossRef Google Scholar

    [13] Zuo C, Qian JM, Feng SJ et al. Deep learning in optical metrology: a review. Light Sci Appl 11, 39 (2022). doi: 10.1038/s41377-022-00714-x

    CrossRef Google Scholar

    [14] Yan KT, Yi YJ, Huang CT et al. Fringe pattern denoising based on deep learning. Opt Commun 437, 148–152 (2019). doi: 10.1016/j.optcom.2018.12.058

    CrossRef Google Scholar

    [15] Kulkarni R, Rastogi P. Fringe denoising algorithms: a review. Opt Lasers Eng 135, 106190 (2020). doi: 10.1016/j.optlaseng.2020.106190

    CrossRef Google Scholar

    [16] Feng SJ, Chen Q, Gu GH et al. Fringe pattern analysis using deep learning. Adv Photon 1, 025001 (2019). doi: 10.1117/1.AP.1.2.025001

    CrossRef Google Scholar

    [17] Ren ZB, Xu ZM, Lam EYM. End-to-end deep learning framework for digital holographic reconstruction. Adv Photon 1, 016004 (2019). doi: 10.1117/1.AP.1.1.016004

    CrossRef Google Scholar

    [18] Rivenson Y, Zhang YB, Günaydın H et al. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci Appl 7, 17141 (2018).

    Google Scholar

    [19] Liu KX, Wu JC, He ZH et al. 4K-DMDNet: diffraction model-driven network for 4K computer-generated holography. Opto-Electron Adv 6, 220135 (2023). doi: 10.29026/oea.2023.220135

    CrossRef Google Scholar

    [20] Feng SJ, Zuo C, Yin W et al. Micro deep learning profilometry for high-speed 3D surface imaging. Opt Lasers Eng 121, 416–427 (2019). doi: 10.1016/j.optlaseng.2019.04.020

    CrossRef Google Scholar

    [21] Qiao G, Huang YY, Song YP et al. A single-shot phase retrieval method for phase measuring deflectometry based on deep learning. Opt Commun 476, 126303 (2020). doi: 10.1016/j.optcom.2020.126303

    CrossRef Google Scholar

    [22] Li YX, Qian JM, Feng SJ et al. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5, 210021 (2022). doi: 10.29026/oea.2022.210021

    CrossRef Google Scholar

    [23] Yang T, Zhang ZZ, Li HH et al. Single-shot phase extraction for fringe projection profilometry using deep convolutional generative adversarial network. Meas Sci Technol 32, 015007 (2020). doi: 10.1088/1361-6501/aba5c5

    CrossRef Google Scholar

    [24] Yin W, Zhong JX, Feng SJ et al. Composite deep learning framework for absolute 3D shape measurement based on single fringe phase retrieval and speckle correlation. J Phys Photon 2, 045009 (2020). doi: 10.1088/2515-7647/abbcd9

    CrossRef Google Scholar

    [25] Feng SJ, Xiao YL, Yin W et al. Fringe-pattern analysis with ensemble deep learning. Adv Photon Nexus 2, 036010 (2023). doi: 10.1117/1.APN.2.3.036010

    CrossRef Google Scholar

    [26] Osten W. What optical metrology can do for experimental me chanics. Appl Mech Mater 70, 1–20 (2011). doi: 10.4028/www.scientific.net/AMM.70.1

    CrossRef Google Scholar

    [27] Goy A, Arthur K, Li S et al. Low photon count phase retrieval using deep learning. Phys Rev Lett 121, 243902 (2018). doi: 10.1103/PhysRevLett.121.243902

    CrossRef Google Scholar

    [28] Wang F, Bian YM, Wang HC et al. Phase imaging with an untrained neural network. Light Sci Appl 9, 77 (2020). doi: 10.1038/s41377-020-0302-3

    CrossRef Google Scholar

    [29] Saba A, Gigli C, Ayoub AB et al. Physics-informed neural networks for diffraction tomography. Adv Photon 4, 066001 (2022).

    Google Scholar

    [30] Reid GT. Automatic fringe pattern analysis: a review. Opt Lasers Eng 7, 37–68 (1986–1987).

    Google Scholar

    [31] Rajshekhar G, Rastogi P. Fringe analysis: premise and perspectives. Opt Lasers Eng 50, iii–x (2012). doi: 10.1016/j.optlaseng.2012.04.006

    CrossRef Google Scholar

    [32] Weise T, Leibe B, Van Gool L. Fast 3D scanning with automatic motion compensation. In Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition 1–8 (IEEE, 2007); http://doi.org/10.1109/CVPR.2007.383291.

    Google Scholar

    [33] Feng SJ, Zuo C, Tao TY et al. Robust dynamic 3-D measurements with motion-compensated phase-shifting profilometry. Opt Lasers Eng 103, 127–138 (2018). doi: 10.1016/j.optlaseng.2017.12.001

    CrossRef Google Scholar

    [34] Zuo C, Tao TY, Feng SJ et al. Micro Fourier Transform Profilometry (μFTP): 3D shape measurement at 10, 000 frames per second. Opt Lasers Eng 102, 70–91 (2018). doi: 10.1016/j.optlaseng.2017.10.013

    CrossRef Google Scholar

    [35] Yu CQ, Wang JB, Peng C et al. BiSeNet: bilateral segmentation network for real-time semantic segmentation. In Proceedings of the 15th European Conference on Computer Vision 325–341 (Springer, 2018); http://doi.org/10.1007/978-3-030-01261-8_20.

    Google Scholar

    [36] Fan MY, Lai SQ, Huang JS et al. Rethinking BiSeNet for real-time semantic segmentation. In Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition 9716–9725 (IEEE, 2021); http://doi.org/10.1109/CVPR46437.2021.00959.

    Google Scholar

    [37] Tao TY, Chen Q, Feng SJ et al. High-precision real-time 3D shape measurement based on a quad-camera system. J Opt 20, 014009 (2018). doi: 10.1088/2040-8986/aa9e0f

    CrossRef Google Scholar

    [38] Feng SJ, Chen Q, Zuo C. Graphics processing unit-assisted real-time three-dimensional measurement using speckle-embedded fringe. Appl Opt 54, 6865–6873 (2015). doi: 10.1364/AO.54.006865

    CrossRef Google Scholar

    [39] Liu K, Wang YC, Lau DL et al. Dual-frequency pattern scheme for high-speed 3-D shape measurement. Opt Express 18, 5229–5244 (2010). doi: 10.1364/OE.18.005229

    CrossRef Google Scholar

  • Supplementary information for Physics-informed deep learning for fringe pattern analysis
    supplymentary movie
  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(6)

Tables(1)

Article Metrics

Article views(6473) PDF downloads(1144) Cited by(0)

Access History
Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint