Priori knowledge assisted dynamic 3D shape measurement with fringe projection

Chu Dongya; Zhang Guanghui; Song Renjie; Zhang Xiaosong; Ying Xiaolin; Li Yong

doi:10.12086/oee.2022.210449

Article navigation > Opto-Electronic Engineering > 2022 Vol. 49 > No. 8 > 210449

Next Article Previous Article

Chu D Y, Zhang G H, Song R J, et al. Priori knowledge assisted dynamic 3D shape measurement with fringe projection[J]. Opto-Electron Eng, 2022, 49(8): 210449. doi: 10.12086/oee.2022.210449

Citation:

Chu D Y, Zhang G H, Song R J, et al. Priori knowledge assisted dynamic 3D shape measurement with fringe projection[J]. Opto-Electron Eng, 2022, 49(8): 210449. doi: 10.12086/oee.2022.210449

Priori knowledge assisted dynamic 3D shape measurement with fringe projection

1.
Institute of Information Optics, Zhejiang Normal University, Jinhua, Zhejiang 321004, China
2.
Key Laboratory of Optical Information Detecting and Display Technology of Zhejiang Province, Jinhua, Zhejiang 321004, China

Fund Project: The Natural Science Foundation of Zhejiang Province (LY17F050002)

More Information

Corresponding author: liyong@zjnu.cn

Received Date 20 January 2022

Revised Date 01 April 2022

Available Online 04 July 2022

Published Date 25 August 2022

Abstract

Abstract

A two-step scheme is proposed to improve the performance of dynamic 3D shape measurement according to its characteristics. 1) The initial 3D shape of the object and the corresponding 3D coordinates of feature points in the 2D image are obtained by measuring the static object or its CAD model. 2) Carry out the 3D measurement in the process of object motion and change. By detecting the feature points in the dynamic image, the motion parameters of the object at different times are calculated according to the corresponding relationship between two-dimensional and three-dimensional coordinates. Then the approximate shape of the object is estimated from the initial shape. The approximate phase of the fringe pattern at this time is calculated. Then, combined with the approximate phase and the wrapped phase of the actual fringe, the unwrapped phase is calculated, and the 3D shape of the object at that time is obtained. Compared with the temporal phase unwrapping method, the proposed scheme improves the measurement speed under the same measurement reliability. Compared with the spatial phase unwrapping method, this scheme improves the measurement reliability at the same measurement speed and is not affected by fringe discontinuity. A static and dynamic dual-mode 3D measurement system was built by using a DLP projector and high-speed camera. The 3D shape measurement of 1280×1024 points at 70 f/s is realized. The experimental results show that the scheme can measure not only the rigid moving object but also the non-rigid moving object, as long as the fringe change caused by its deformation does not exceed half a period. The proposed method has a large tolerance for the change of object pose at adjacent times as well.
- machine vision /
- optical three-dimensional shape measurement /
- phase measurement profilometry /
- phase unwrapping

FullText(HTML)

References

[1]	Geng J. Structured-light 3D surface imaging: a tutorial[J]. Adv Opt Photon, 2011, 3(2): 128−160. doi: 10.1364/AOP.3.000128 CrossRef Google Scholar
[2]	李勇, 张广汇, 马利红, 等. 条纹投影动态三维表面成像技术综述[J]. 红外与激光工程, 2020, 49(3): 0303005. doi: 10.3788/IRLA202049.0303005 CrossRef Google Scholar Li Y, Zhang G H, Ma L H, et al. Review of dynamic three-dimensional surface imaging based on fringe projection[J]. Infrared Laser Eng, 2020, 49(3): 0303005. doi: 10.3788/IRLA202049.0303005 CrossRef Google Scholar
[3]	应晓霖, 姚建云, 张晓松, 等. 采用LD的光源步进条纹投影三维测量系统[J]. 光电工程, 2021, 48(11): 210298. doi: 10.12086/oee.2021.210298 CrossRef Google Scholar Ying X L, Yao J Y, Zhang X S, et al. Fringe projection based three-dimensional measurement system by the light-source-stepping method using LD[J]. Opto-Electron Eng, 2021, 48(11): 210298. doi: 10.12086/oee.2021.210298 CrossRef Google Scholar
[4]	Srinivasan V, Liu H C, Halioua M. Automated phase-measuring profilometry of 3-D diffuse objects[J]. Appl Opt, 1984, 23(18): 3105−3108. doi: 10.1364/AO.23.003105 CrossRef Google Scholar
[5]	Takeda M, Mutoh K. Fourier transform profilometry for the automatic measurement of 3-D object shapes[J]. Appl Opt, 1983, 22(24): 3977−3982. doi: 10.1364/AO.22.003977 CrossRef Google Scholar
[6]	Zhong J G, Weng J W. Spatial carrier-fringe pattern analysis by means of wavelet transform: wavelet transform profilometry[J]. Appl Opt, 2004, 43(26): 4993−4998. doi: 10.1364/AO.43.004993 CrossRef Google Scholar
[7]	Su X Y, Chen W J. Reliability-guided phase unwrapping algorithm: a review[J]. Opt Lasers Eng, 2004, 42(3): 245−261. doi: 10.1016/j.optlaseng.2003.11.002 CrossRef Google Scholar
[8]	Zhao M, Huang L, Zhang Q C, et al. Quality-guided phase unwrapping technique: comparison of quality maps and guiding strategies[J]. Appl Opt, 2011, 50(33): 6214−6224. doi: 10.1364/AO.50.006214 CrossRef Google Scholar
[9]	Huntley J M, Saldner H. Temporal phase-unwrapping algorithm for automated interferogram analysis[J]. Appl Opt, 1993, 32(17): 3047−3052. doi: 10.1364/AO.32.003047 CrossRef Google Scholar
[10]	Zuo C, Huang L, Zhang M L, et al. Temporal phase unwrapping algorithms for fringe projection profilometry: a comparative review[J]. Opt Lasers Eng, 2016, 85: 84−103. doi: 10.1016/j.optlaseng.2016.04.022 CrossRef Google Scholar
[11]	Cheng Y Y, Wyant J C. Two-wavelength phase shifting interferometry[J]. Appl Opt, 1984, 23(24): 4539−4543. doi: 10.1364/AO.23.004539 CrossRef Google Scholar
[12]	Cheng Y Y, Wyant J C. Multiple-wavelength phase-shifting interferometry[J]. Appl Opt, 1985, 24(6): 804−807. doi: 10.1364/AO.24.000804 CrossRef Google Scholar
[13]	卢世江, 李勇, 王辉, 等. 含孤立物体场景的高速、高密度三维面形采集[J]. 光电工程, 2012, 39(1): 34−39. doi: 10.3969/j.issn.1003-501X.2012.01.007 CrossRef Google Scholar Lu S J, Li Y, Wang H, et al. High-speed and dense 3D shape capture of scene consisting of spatially isolated objects[J]. Opto-Electron Eng, 2012, 39(1): 34−39. doi: 10.3969/j.issn.1003-501X.2012.01.007 CrossRef Google Scholar
[14]	Zheng D L, Kemao Q, Da F P, et al. Ternary Gray code-based phase unwrapping for 3D measurement using binary patterns with projector defocusing[J]. Appl Opt, 2017, 56(13): 3660−3665. doi: 10.1364/AO.56.003660 CrossRef Google Scholar
[15]	An Y T, Zhang S. Three-dimensional absolute shape measurement by combining binary statistical pattern matching with phase-shifting methods[J]. Appl Opt, 2017, 56(19): 5418−5426. doi: 10.1364/AO.56.005418 CrossRef Google Scholar
[16]	Li Z W, Zhong K, Li Y F, et al. Multiview phase shifting: a full-resolution and high-speed 3D measurement framework for arbitrary shape dynamic objects[J]. Opt Lett, 2013, 38(9): 1389−1391. doi: 10.1364/OL.38.001389 CrossRef Google Scholar
[17]	Guo X P, Zhao H, Jia P P, et al. Multiview fringe matching profilometry in a projector–camera system[J]. Opt Lett, 2018, 43(15): 3618−3621. doi: 10.1364/OL.43.003618 CrossRef Google Scholar
[18]	An Y T, Hyun J S, Zhang S. Pixel-wise absolute phase unwrapping using geometric constraints of structured light system[J]. Opt Express, 2016, 24(16): 18445−18459. doi: 10.1364/OE.24.018445 CrossRef Google Scholar
[19]	Li B W, Bell T, Zhang S. Computer-aided-design-model-assisted absolute three-dimensional shape measurement[J]. Appl Opt, 2017, 56(24): 6770−6776. doi: 10.1364/AO.56.006770 CrossRef Google Scholar
[20]	Dai J F, An Y T, Zhang S. Absolute three-dimensional shape measurement with a known object[J]. Opt Express, 2017, 25(9): 10384−10396. doi: 10.1364/OE.25.010384 CrossRef Google Scholar
[21]	平乙杉, 刘元坤. 基于单应性矩阵的线结构光系统简易标定方法[J]. 光电工程, 2019, 46(12): 180677. doi: 10.12086/oee.2019.180677 CrossRef Google Scholar Ping Y S, Liu Y K. An easy line-structured light system calibration method based on homography matrix[J]. Opto-Electron Eng, 2019, 46(12): 180677. doi: 10.12086/oee.2019.180677 CrossRef Google Scholar
[22]	Li Y, Su X Y, Wu Q Y. Accurate phase–height mapping algorithm for PMP[J]. J Mod Opt, 2006, 53(14): 1955−1964. doi: 10.1080/09500340600720789 CrossRef Google Scholar
[23]	Ren S Q, Cao X D, Wei Y C, et al. Face alignment at 3000 FPS via regressing local binary features[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 2014: 1685–1692. Google Scholar

Overview

Overview

In the deformation measurement of moving objects with fringe projection, such as the deformation of objects in high-speed flight and the measurement of the unconstrained facial expression changes, we pursue to project as few patterns as possible and obtain as high measurement accuracy as possible. Obtaining the unwrapped phase is one of the key steps in fringe projection 3D measurement, which generally needs the assistance of other information. The common methods are spatial phase unwrapping algorithm, temporal phase unwrapping algorithm, and multi-view geometric constraint. These methods solve the phase unwrapping problem well to some extent, but they have their limitations. The spatial phase unwrapping method is difficult to deal with spatially discontinuous or isolated regions. The temporal phase unwrapping method takes a long time and requires higher hardware at the same measurement speed. The multi-view geometric constraint method reduces the measurement area and increases the complexity and cost of the whole system. In most circumstances, the initial shape of the moving objects can be obtained. According to this fact, a two-step scheme is proposed to improve the performance of dynamic 3D shape measurement. 1) The initial 3D shape of the object and the corresponding 3D coordinates of feature points in the 2D image are obtained by measuring the static object or its CAD model. 2) Carry out the 3D measurement of object motion and change. By detecting the feature points in the dynamic image, the motion parameters of the object at different times are calculated according to the corresponding relationship between 2D and 3D coordinates. Then the approximate shape of the object is estimated from the initial shape. The approximate phase of the fringe pattern at this time is calculated. Then, combined with the approximate phase and the wrapped phase of the actual fringe, the unwrapped phase is calculated, and the 3D shape of the object at that time is obtained. Compared with the temporal phase unwrapping method, the proposed scheme improves the measurement speed under the same measurement reliability. Compared with the spatial phase unwrapping method, this scheme improves the measurement reliability at the same measurement speed and is not affected by fringe discontinuity. A static and dynamic dual-mode 3D measurement system was built by using a DLP projector and high-speed camera. The 3D shape measurement of 1280×1024 points at 70 f/s is realized. The experimental results show that the scheme is feasible and has a large tolerance for the change of the object pose at adjacent times.