Li YX, Qian JM, Feng SJ, Chen Q, Zuo C. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5, 210021 (2022). doi: 10.29026/oea.2022.210021
Citation: Li YX, Qian JM, Feng SJ, Chen Q, Zuo C. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5, 210021 (2022). doi: 10.29026/oea.2022.210021

Original Article Open Access

Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement

More Information
  • Single-shot high-speed 3D imaging is important for reconstructions of dynamic objects. For fringe projection profilometry (FPP), however, it is still challenging to recover accurate 3D shapes of isolated objects by a single fringe image. In this paper, we demonstrate that the deep neural networks can be trained to directly recover the absolute phase from a unique fringe image that involves spatially multiplexed fringe patterns of different frequencies. The extracted phase is free from spectrum-aliasing problem which is hard to avoid for traditional spatial-multiplexing methods. Experiments on both static and dynamic scenes show that the proposed approach is robust to object motion and can obtain high-quality 3D reconstructions of isolated objects within a single fringe image.

  • 加载中
  • [1] Gorthi SS, Rastogi P. Fringe projection techniques: whither we are. Opt Lasers Eng 48, 133–140 (2010). doi: 10.1016/j.optlaseng.2009.09.001

    CrossRef Google Scholar

    [2] Zhang ZH, Towers CE, Towers DP. Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection. Opt Express 14, 6444–6455 (2006). doi: 10.1364/OE.14.006444

    CrossRef Google Scholar

    [3] Su XY, Zhang QC. Dynamic 3-D shape measurement method: a review. Opt Lasers Eng 48, 191–204 (2010). doi: 10.1016/j.optlaseng.2009.03.012

    CrossRef Google Scholar

    [4] Tao TY, Chen Q, Da J, Feng SJ, Hu Y et al. Real-time 3-D shape measurement with composite phase-shifting fringes and multi-view system. Opt Express 24, 20253–20269 (2016). doi: 10.1364/OE.24.020253

    CrossRef Google Scholar

    [5] Feng SJ, Zhang L, Zuo C, Tao TY, Chen Q et al. High dynamic range 3D measurements with fringe projection profilometry: a review. Meas Sci Technol 29, 122001 (2018). doi: 10.1088/1361-6501/aae4fb

    CrossRef Google Scholar

    [6] Feng SJ, Zuo C, Tao TY, Hu Y, Zhang ML et al. Robust dynamic 3-D measurements with motion-compensated phase-shifting profilometry. Opt Lasers Eng 103, 127–138 (2018). doi: 10.1016/j.optlaseng.2017.12.001

    CrossRef Google Scholar

    [7] Pan B, Xie HM, Wang ZY, Qian KM, Wang ZY. Study on subset size selection in digital image correlation for speckle patterns. Opt Express 16, 7037–7048 (2008). doi: 10.1364/OE.16.007037

    CrossRef Google Scholar

    [8] Hu Y, Chen Q, Feng SJ, Zuo C. Microscopic fringe projection profilometry: a review. Opt Lasers Eng 135 (2020). doi: 10.1016/j.optlaseng.2020.106192

    CrossRef Google Scholar

    [9] Tao TY, Chen Q, Feng SJ, Qian JM, Hu Y et al. High-speed real-time 3D shape measurement based on adaptive depth constraint. Opt Express 26, 22440–22456 (2018). doi: 10.1364/OE.26.022440

    CrossRef Google Scholar

    [10] Qian JM, Feng SJ, Tao TY, Hu Y, Liu K et al. High-resolution real-time 360° 3D model reconstruction of a handheld object with fringe projection profilometry. Opt Lett 44, 5751–5754 (2019). doi: 10.1364/OL.44.005751

    CrossRef Google Scholar

    [11] Qian JM, Feng SJ, Xu MZ, Tao TY, Shang YH et al. High-resolution real-time 360° 3D surface defect inspection with fringe projection profilometry. Opt Lasers Eng 137, 106382 (2021). doi: 10.1016/j.optlaseng.2020.106382

    CrossRef Google Scholar

    [12] Zuo C, Chen Q, Gu GH, Feng SJ, Feng FXY et al. High-speed three-dimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection. Opt Lasers Eng 51, 953–960 (2013). doi: 10.1016/j.optlaseng.2013.02.012

    CrossRef Google Scholar

    [13] Heist S, Lutzke P, Schmidt I, Dietrich P, Kühmstedt P et al. High-speed three-dimensional shape measurement using GOBO projection. Opt Lasers Eng 87, 90–96 (2016). doi: 10.1016/j.optlaseng.2016.02.017

    CrossRef Google Scholar

    [14] Heist S, Kühmstedt P, Tünnermann A, Notni G. Theoretical considerations on aperiodic sinusoidal fringes in comparison to phase-shifted sinusoidal fringes for high-speed three-dimensional shape measurement. Appl Opt 54, 10541–10551 (2015). doi: 10.1364/AO.54.010541

    CrossRef Google Scholar

    [15] Takeda M, Mutoh K. Fourier transform profilometry for the automatic measurement of 3-D object shapes. Appl Opt 22, 3977–3982 (1983). doi: 10.1364/AO.22.003977

    CrossRef Google Scholar

    [16] Su XY, Chen WJ. Fourier transform profilometry: : a review. Opt Lasers Eng 35, 263–284 (2001). doi: 10.1016/S0143-8166(01)00023-9

    CrossRef Google Scholar

    [17] Kemao Q. Two-dimensional windowed Fourier transform for fringe pattern analysis: principles, applications and implementations. Opt Lasers Eng 45, 304–317 (2007). doi: 10.1016/j.optlaseng.2005.10.012

    CrossRef Google Scholar

    [18] Huang L, Kemao Q, Pan B, Asundi AK. Comparison of Fourier transform, windowed Fourier transform, and wavelet transform methods for phase extraction from a single fringe pattern in fringe projection profilometry. Opt Lasers Eng 48, 141–148 (2010). doi: 10.1016/j.optlaseng.2009.04.003

    CrossRef Google Scholar

    [19] Zhang ZH, Jing Z, Wang ZH, Kuang DF. Comparison of Fourier transform, windowed Fourier transform, and wavelet transform methods for phase calculation at discontinuities in fringe projection profilometry. Opt Lasers Eng 50, 1152–1160 (2012). doi: 10.1016/j.optlaseng.2012.03.004

    CrossRef Google Scholar

    [20] Zuo C, Feng SJ, Huang L, Tao TY, Yin W et al. Phase shifting algorithms for fringe projection profilometry: a review. Opt Lasers Eng 109, 23–59 (2018). doi: 10.1016/j.optlaseng.2018.04.019

    CrossRef Google Scholar

    [21] Pan B, Kemao Q, Huang L, Asundi A. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Opt Lett 34, 416–418 (2009). doi: 10.1364/OL.34.000416

    CrossRef Google Scholar

    [22] Zuo C, Huang L, Zhang ML, Chen Q, Asundi A. Temporal phase unwrapping algorithms for fringe projection profilometry: a comparative review. Opt Lasers Eng 85, 84–103 (2016). doi: 10.1016/j.optlaseng.2016.04.022

    CrossRef Google Scholar

    [23] Liu K, Wang YC, Lau DL, Hao Q, Hassebrook LG. Dual-frequency pattern scheme for high-speed 3-D shape measurement. Opt Express 18, 5229–5244 (2010). doi: 10.1364/OE.18.005229

    CrossRef Google Scholar

    [24] Zuo C, Tao TY, Feng SJ, Huang L, Asundi A et al. Micro Fourier transform profilometry (µftp): 3D shape measurement at 10, 000 frames per second. Opt Lasers Eng 102, 70–91 (2018). doi: 10.1016/j.optlaseng.2017.10.013

    CrossRef Google Scholar

    [25] Takeda M, Gu Q, Kinoshita M, Takai H, Takahashi Y. Frequency-multiplex Fourier-transform profilometry: a single-shot three- dimensional shape measurement of objects with large height discontinuities and/or surface isolations. Appl Opt 36, 5347–5354 (1997). doi: 10.1364/AO.36.005347

    CrossRef Google Scholar

    [26] Zhong JG, Zhang YL. Absolute phase-measurement technique based on number theory in multifrequency grating projection profilometry. Appl Opt 40, 492–500 (2001). doi: 10.1364/AO.40.000492

    CrossRef Google Scholar

    [27] Guan C, Hassebrook LG, Lau DL. Composite structured light pattern for three-dimensional video. Opt Express 11, 406–417 (2003). doi: 10.1364/OE.11.000406

    CrossRef Google Scholar

    [28] Sansoni G, Redaelli E. A 3D vision system based on one-shot projection and phase demodulation for fast profilometry. Meas Sci Technol 16, 1109–1118 (2005). doi: 10.1088/0957-0233/16/5/009

    CrossRef Google Scholar

    [29] Yue HM, Su XY, Liu YZ. Fourier transform profilometry based on composite structured light pattern. Opt Laser Technol 39, 1170–1175 (2007). doi: 10.1016/j.optlastec.2006.08.014

    CrossRef Google Scholar

    [30] Chen WJ, Su XY, Cao Y, Xiang LQ, Zhang QC. Fourier transform profilometry based on a fringe pattern with two frequency components. Optik-Int J Light Electron Opt 119, 57–62 (2008). doi: 10.1016/j.ijleo.2006.05.024

    CrossRef Google Scholar

    [31] Zhang ZH. Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques. Opt Lasers Eng 50, 1097–1106 (2012). doi: 10.1016/j.optlaseng.2012.01.007

    CrossRef Google Scholar

    [32] García-Isáis C, Ochoa NA. One shot profilometry using a composite fringe pattern. Opt Lasers Eng 53, 25–30 (2014). doi: 10.1016/j.optlaseng.2013.08.006

    CrossRef Google Scholar

    [33] Feng SJ, Chen Q, Gu GH, Tao TY, Zhang L et al. Fringe pattern analysis using deep learning. Adv Photonics 1, 025001 (2019).

    Google Scholar

    [34] Yin W, Chen Q, Feng SJ, Tao TY, Huang L et al. Temporal phase unwrapping using deep learning. Sci Rep 9, 20175 (2019). doi: 10.1038/s41598-019-56222-3

    CrossRef Google Scholar

    [35] van der Jeught S, Dirckx JJJ. Deep neural networks for single shot structured light profilometry. Opt Express 27, 17091–17101 (2019). doi: 10.1364/OE.27.017091

    CrossRef Google Scholar

    [36] Qian JM, Feng SJ, Tao TY, Hu Y, Li YX et al. Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement. APL Photonics 5, 046105 (2020). doi: 10.1063/5.0003217

    CrossRef Google Scholar

    [37] Feng SJ, Zuo C, Yin W, Gu GH, Chen Q. Micro deep learning profilometry for high-speed 3D surface imaging. Opt Lasers Eng 121, 416–427 (2019). doi: 10.1016/j.optlaseng.2019.04.020

    CrossRef Google Scholar

    [38] Qian JM, Feng SJ, Li YX, Tao TY, Han J et al. Single-shot absolute 3D shape measurement with deep-learning-based color fringe projection profilometry. Opt Lett 45, 1842–1845 (2020). doi: 10.1364/OL.388994

    CrossRef Google Scholar

    [39] Shi JS, Zhu XJ, Wang HY, Song LM, Guo QH. Label enhanced and patch based deep learning for phase retrieval from single frame fringe pattern in fringe projection 3D measurement. Opt Express 27, 28929–28943 (2019). doi: 10.1364/OE.27.028929

    CrossRef Google Scholar

    [40] Nguyen H, Wang YZ, Wang ZY. Single-shot 3D shape reconstruction using structured light and deep convolutional neural networks. Sensors 20, 3718 (2020).

    Google Scholar

    [41] Zheng Y, Wang SD, Li Q, Li BW. Fringe projection profilometry by conducting deep learning from its digital twin. Opt Express 28, 36568–36583 (2020). doi: 10.1364/OE.410428

    CrossRef Google Scholar

    [42] Zhang S. Absolute phase retrieval methods for digital fringe projection profilometry: a review. Opt Lasers Eng 107, 28–37 (2018). doi: 10.1016/j.optlaseng.2018.03.003

    CrossRef Google Scholar

    [43] Ghiglia DC, Pritt MD. Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (Wiley-Interscience, New York, 1998).

    Google Scholar

    [44] Chollet F. Deep Learning with Python (Manning Publications, Shelter Island, 2018).

    Google Scholar

    [45] Qian JM, Tao TX, Feng SJ, Chen Q, Zuo C. Motion-artifact-free dynamic 3D shape measurement with hybrid Fourier-transform phase- shifting profilometry. Opt Express 27, 2713–2731 (2019). doi: 10.1364/OE.27.002713

    CrossRef Google Scholar

    [46] Lilienblum E, Michaelis B. Optical 3D surface reconstruction by a multi-period phase shift method. J Comput 2, 73–83 (2007).

    Google Scholar

    [47] Pribanić T, Mrvoš S, Salvi J. Efficient multiple phase shift patterns for dense 3D acquisition in structured light scanning. Image Vis Comput 28, 1255–1266 (2010). doi: 10.1016/j.imavis.2010.01.003

    CrossRef Google Scholar

    [48] Ding Y, Xi JT, Yu YG, Chicharo J. Recovering the absolute phase maps of two fringe patterns with selected frequencies. Opt Lett 36, 2518–2520 (2011). doi: 10.1364/OL.36.002518

    CrossRef Google Scholar

    [49] Ding Y, Xi JT, Yu YG, Cheng WQ, Wang S et al. Frequency selection in absolute phase maps recovery with two frequency projection fringes. Opt Express 20, 13238–13251 (2012). doi: 10.1364/OE.20.013238

    CrossRef Google Scholar

    [50] Yin W, Zuo C, Feng SJ, Tao TY, Hu Y et al. High-speed three-dimensional shape measurement using geometry- constraint-based number-theoretical phase unwrapping. Opt Lasers Eng 115, 21–31 (2019). doi: 10.1016/j.optlaseng.2018.11.006

    CrossRef Google Scholar

    [51] Zhang Z. A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22, 1330–1334 (2000). doi: 10.1109/34.888718

    CrossRef Google Scholar

    [52] Zhang S, Huang PS. Novel method for structured light system calibration. Opt Eng 45, 083601 (2006). doi: 10.1117/1.2336196

    CrossRef Google Scholar

    [53] Huang L, Zhang QC, Asundi A. Camera calibration with active phase target: improvement on feature detection and optimization. Opt Lett 38, 1446–1448 (2013). doi: 10.1364/OL.38.001446

    CrossRef Google Scholar

  • Supplementary Information visualization 1
  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(11)

Tables(1)

Article Metrics

Article views(3828) PDF downloads(561) Cited by(0)

Access History

Other Articles By Authors

Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint