Advanced
3-D Reconstruction of Human Face Using the Derivative Moiré Topography
3-D Reconstruction of Human Face Using the Derivative Moiré Topography
Journal of the Optical Society of Korea. 2014. Oct, 18(5): 500-506
Copyright © 2014, Journal of the Optical Society of Korea
  • Received : April 04, 2014
  • Accepted : September 09, 2014
  • Published : October 25, 2014
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Yoon Jae Bae
R&D Center, Fibertec, Seoul 135-010, Korea
Byeong Wan Ha
School of Electronics, Telecommunication and Computer Engineering, Korea Aerospace University, Goyang 412-791, Korea
Ji An Park
School of Electronics, Telecommunication and Computer Engineering, Korea Aerospace University, Goyang 412-791, Korea
Choon Sik Cho
R&D Center, Fibertec, Seoul 135-010, Korea
cscho@kau.ac.kr
Abstract
A new 3-D reconstruction algorithm for the human face is proposed using the derivative Moiré topography which ensures fast and robust reconstruction even for rough surfaces. The Moiré interference fringe pattern is initially obtained through the projection Moiré topography based on phase shifting, and then differentiated to provide a full unwrapped phase map for a human face. 2π ambiguity, which has been a chronically unsolved problem with Moiré topography, is successfully surmounted by differentiating the Moiré fringe patterns both in x- and y-directions when the object is located in the x-y plane. A real human face is used for verifying the proposed derivative Moiré topography. A human face of 4 different phase-shifted images taken in the fixed plane is almost fully reconstructed in 3-D format in 0.1 mm lateral resolution.
Keywords
I. INTRODUCTION
Noncontact 3-D surface reconstruction techniques have been extending their applications to various industrial fields such as medical, manufacturing and biometric areas. A large number of noncontact 3-D reconstruction techniques developed so far show their advantages and drawbacks as listed in [1] . Among them Moiré topography has been of prime interest because it provides fast operation for smooth surfaces since it was introduced [2 - 4] . For Moiré topography, when the projected grating of the projection system falls on the object, the deformed grating is generated on its surface, Moiré interference fringe patterns are then generated on the imaging system, yielding contours of the same heights expressed in wrapped phases. Due to diffraction, a virtual grating is often employed for replacing reference and/or projected gratings. Projection Moiré topography was thereafter developed to increase the resolution significantly using the virtual grating for the projection system. In the meantime, the phase shifting method has been devised to ensure repeatability of the projection Moiré topography where multiple phase shifts are carried out to easily calculate the phase map for objects [5 , 6] .
Although Moiré topography usually ensures fast and efficient 3-D reconstruction in smooth surfaces [7 , 8] , two big obstacles of noise and imperfect phase unwrapping make one hesitate to capitalize on this. To eliminate noise occurring in a wrapped phase map, cosine transform or wavelet transform has been tried [9 - 12] . The projection Moiré topography with phase shifting method provides simple and reliable wrapped phase images for objects with –π/2 to π/2 range, called 2π ambiguity, however it is very difficult to convert this wrapped phase to a noise free unwrapped phase map. Abrupt phase change more than π cannot be detected easily, so that very few techniques have been tried for achieving more exact phase unwrapping. The weighted least squares method with discrete cosine transform (DCT) was introduced and utilized where the least squares method was used for removing discontinuity (or noise) in the phase image and DCT was employed for phase unwrapping in a strict sense, however this method shows difficulty in tracking a sudden change of phase while unwrapping the wrapped phase image, and it requires huge computing power because of a large amount of computation [13] .
In this work, the derivative Moiré topography by differentiating the wrapped phase map separately in x- and y-directions is proposed to resolve the 2π ambiguity problem. Compared with the least squares method with DCT, the proposed technique tracks sudden phase change very efficiently using the simple differentiation in x- and y-directions. This also does not introduce a large amount of computational burden, leading to very fast attainment of phase unwrapping.
II. THE PROPOSED MOIRÉ TOPOGRAPHY
The classical measurement setup to obtain the Moiré fringe pattern using the phase shifting method is shown as illustrated in Figure 1 where Id (·) is the light intensity of the deformed grating at (x, y) under test, Ir (·) is the light intensity of the reference grating with phase-shifting, and Ik indicates the resulting Moiré intensity at (x, y) when Δ( t ) = ( k – 1)×90°. To avoid diffraction virtual projected and reference gratings for the phase shifting method can be established as shown in Figure 2(a) where the object is located in the x-y plane, projection and imaging systems are apart by d , and the object is away by L from the projection system. Figure 2(b) describes the real measurement setup used for this work where four different virtual gratings for the projection and the reference are devised by applying the phase difference of 90° between consecutive gratings. A light source with four virtual projected gratings is emitted in sequence from the projection system and creates the Moiré interference fringe patterns equivalently to Fig. 1 . The Moiré fringe pattern is therefore created when the deformed grating in (1) is superimposed on the reference grating in (2). In principle, the intensity of the Moiré pattern, IMoiré ( x, y ), is extracted from the fourth term of (3) which is the product of intensity of the deformed grating ( I1 ) and that of the transmitted imaging through the reference grating of the imaging system ( I2 ) where Is is the intensity of the light source, A is the modulation of gratings, R is the reflectance of the object surface, M is the magnifying rate of the projection system, G is the pitch of gratings, and h ( x, y ) is the height of the object as in (4) [2 4] . If the argument within cos (∙) is represented by ϕ ( x, y ) as described in (5), the height of the object can be calculated as shown in (6) since
PPT Slide
Lager Image
in Fig. 2(a) . To obtain ϕ ( x, y ) to a high precision, the phase shifting method as described in Fig. 2 is capitalized using the traditional equation as expressed in (3) where ϕo ( x, y ) means the principal value of the wrapped phase, -90° to 90°. For the phase shifting method, projected and reference gratings ( Ir ) can be generated virtually with a computer, the deformed grating ( Id ) is obtained on the surface object as seen in (7), and the resulting interference pattern in (8) can be extracted by multiplying Id and Ir with the computer, yielding no more diffraction and interference where ad is the magnitude of the deformed grating, ar is the magnitude of the reference grating, D(x,y) is the average intensity of the resulting interference pattern, γ(x,y) shows the visibility of the interference pattern, and ϕ(x,y) = ϕd(x,y) - ϕr(x,y) yields the phase of the interference pattern. As seen in (9), ϕ° ( x, y ) can only have the principal values from -90° to 90° which is called 2π ambiguity, therefore phase unwrapping should be carried out, in other words, the absolute Moiré order is required to be found accordingly.
PPT Slide
Lager Image
The conventional Moiré topography based on the phase shifting method.
PPT Slide
Lager Image
Measurement setup for the proposed Moiré topography. (a) Phase shifting method based on virtually generated 4 reference gratings. (b) Real measurement setup.
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
In this work, we perform the phase unwrapping by differentiating (9) twice in x- and y-directions, leading to (10) where a primed subscript means the differentiation with respect to the subscript. Since the derivative phases as expressed in (10) are continuous functions even around the big phase jump, phase can be tracked precisely without difficulty even if it is greater than 90º or less than -90º. The principal phase is at first calculated using (9) and then one differentiation is carried out in one direction ( i.e . x- or y-direction) and thereafter another differentiation is taken in the other direction.
PPT Slide
Lager Image
The absolute Moiré order at the ( i , j )-th pixel, ni,j , to obtain the exact phase can be determined using (11) at first in the x-direction. In Moiré topography, the unwrapped phase map is composed of many contour lines where the absolute order difference between a contour and its neighboring contour is always 1 for the regular case in which no discontinuous contours exist. Even in the case for which the height of the object changes abruptly, contours are distributed densely not discontinuously. The irregular case occurs when two different heights are overlapped or black spots such as the pupil of eye, nostril or eyebrow are created, providing discontinuous contours, in which no Moiré patterns are generated and then no continuous phase contours are obtained. Therefore, the absolute order can be calculated using the derivative of phase change based on current phase contours. A sudden phase change more than nπ yields addition of n from the current absolute order, where the order is updated in x- and y-directions while scanning all of the pixels of the object. But since the irregular case provides incorrect phase derivative, this error is accumulated through scanning up to the final pixel point.
PPT Slide
Lager Image
where fix (•) function removes the fractional part and returns the integer part, adding the corresponding integer (n) value when the phase becomes greater than nπ. Figure 3 illustrates an example showing the case for n equal to 1 at the i -th pixel. With this ni,j , the phase is unwrapped in the x-direction as expressed in (12),
PPT Slide
Lager Image
Principle of phase unwrapping.
PPT Slide
Lager Image
where the calculated phase from (12) may accumulate errors due to discontinuous contours. To resolve this error accumulation, phase compensation is carried out by calculating a different derivative with respect to wider step in the x-direction. Accumulated phase error is detected and incorrect phase becomes recovered by computation of intermediate order ( nxc ) as represented in (13) where a needs to be predefined as a big number (usually ≥ 50). The left term for j a in (13) indicates accumulation of errors by summing phase variations from [( k -1) a +1] th pixel to [ ka +1] th pixel, and the right term expresses accumulation of errors by summing phase variation from [ ka +1] th pixel to jth pixel as shown in Fig. 4 . For j a , phase variation from 1st pixel to j th pixel is calculated for nxc . However, it should be noted that although phase error is not accumulated using (13) and (14), the absolute order cannot be obtained correctly in the vicinity of discontinuous contours due to overlapping heights or black spots.
PPT Slide
Lager Image
Detection of phase error accumulation.
PPT Slide
Lager Image
PPT Slide
Lager Image
The absolute Moiré order is now calculated once more in the y-direction using (15),
PPT Slide
Lager Image
Eventually the phase is unwrapped using (16),
PPT Slide
Lager Image
III. MEASUREMENT RESULTS
To validate the proposed method, a human face as shown in Fig. 5 is taken for an object which is located 60 cm away from the projection and imaging systems. The projection system used here has 1200×800 pixels and provides 28 cm×35 cm image to the plane 60 cm ( L ) away from the projection and imaging systems which are 11.5 cm ( d ) apart from each other. Projected and reference gratings are generated virtually with pitch ( G ) of 1 mm. The imaging system consisting of a CCD camera and a lens has 2592×1944 pixels, therefore the minimum resolution is calculated to be 0.144 mm in the x-direction and 0.135 mm in the y-direction. Four deformed gratings obtained on the surface of the object using four different projected gratings are exhibited in Fig. 6 where the original images are converted to the corresponding gray values to provide the light intensity ( Ik ). Using deformed and reference gratings, Moiré fringe patterns are calculated initially using (9) where the phase is not unwrapped. The initial phase for Moiré fringe patterns is calculated by subtracting the phase of the reference grating from that of the deformed grating as deduced in (8). Due to abrupt phase variation, (10) to (16) are used for producing the exact phase unwrapping specifically in the vicinity of sudden phase change.
PPT Slide
Lager Image
A human face photo used for this work. FIG.
PPT Slide
Lager Image
4 different deformed grating images obtained with virtual reference gratings.
Figure 7 exhibits initial phase map obtained with (9) which shows the principal phase image from -90° to 90°. Figure 8(a) displays the unwrapped phase map applying the phase derivative in the x-direction, Fig. 8(b) shows the compensated phase map employing the additional derivative as introduced in (13), and Fig. 8(c) shows the finally unwrapped phase map which applied the phase derivatives in x- and y-directions as introduced in (10) to (16). Discontinuous contours due to overlapping two different heights and black spots lead to failure of phase unwrapping as shown in Fig. 8(c) where black spots provide errors in obtaining exactly unwrapped phase, as expected in Sec. II. A detailed procedure from (11) to (16) is offered for verification of the proposed method as displayed in Table 1 where phases before unwrapping are obtained from (9), and from these data the final unwrapped phases are successfully calculated using (11)-(16). The negative phases illustrate relatively lowered phases with respect to the highest altitude of cheek in Fig. 4 .
PPT Slide
Lager Image
Initial phase wrapped image.
PPT Slide
Lager Image
Phase unwrapped images based on (a) x-directed differentiation, (b) phase compensation, (c) y-directed differentiation.
Results for phases and orders at 10 pixels between brows inFig. 4
PPT Slide
Lager Image
Results for phases and orders at 10 pixels between brows in Fig. 4
The final 3-D picture of a human face is reconstructed as illustrated in Fig. 9 . For a clear view, the background image is removed and textured for vivid visualization. Finally a moving average with 50×50 window size is performed for removing ripples specifically around black spots such as eyebrow and nostril, providing smooth 3-D surface and adding a bit more CPU time. Vague stripes still appear on the cheek as seen in Fig. 9 since the light intensity of projected gratings are not perfectly sinusoidal, leading to some discontinuity for phase unwrapping.
PPT Slide
Lager Image
The final 3-D reconstructed image.
Compared with the least squares method with DCT, the proposed derivative Moiré technique shows fast computation and precise reconstruction for even a rough surface such as a human face. DCT has been applied to unwrap the phase image of the human face as taken in Fig. 5 , however it does not produce the complete 3-D reconstruction. Therefore a very simple and periodic principal phase image was used to compare DCT and the proposed method as seen in Fig. 10 . CPU times have been computed for three cases as summarized in Table 2 , where the derivative Moiré topography shows much faster time taken for phase unwrapping.
PPT Slide
Lager Image
A principal phase image used for comparison.
Comparison of two phase unwrapping methods in terms of CPU time
PPT Slide
Lager Image
Comparison of two phase unwrapping methods in terms of CPU time
IV. CONCLUSION
3-D reconstruction has been successfully implemented using the derivative Moiré topography based on phase unwrapping using differentiation of phase in x- and y-directions. Abrupt phase change is identified successfully by this method, leading to the precise and fast height computation of objects containing rough surfaces. Differentiation is performed both in x- and y-directions for obtaining noise free phase unwrapping. From this, errors occurring from inexact phase unwrapping are eliminated as the differentiation iterates. The classical phase shifting method is replaced by building four phase-shifted virtual reference gratings. A human face is exploited to validate the proposed method. This proposed method will provide a potential direction for systems necessitating robust and fast 3-D reconstruction.
Acknowledgements
This research was supported by a grant (A131306) from Gyeonggi Technology Development Program funded by Gyeonggi Province, Korea.
References
Sansoni G. , Trebeschi M. , Docchio F. 2009 “State-of-the-art and applications of 3D imaging sensors in industry, cultural heritage, medicine, and criminal investigation,” Sensors 9 568 - 601
Takasaki H. 1970 “Moiré topography,” Appl. Opt. 9 1467 - 1472
Kafri O. , Glatt I. 1990 The Physics of Moiré Metrology John Wiley & Sons
Amidror I. 2000 The Theory of the Moiré Phenomenon Kluwer Academic Publishers Dordrecht, Netherlands
Li N. 2010 “Simulation of a small feature gauging system using phase-shift projection Moiré,” The 2nd IEEE International Conference on Information Management and Engineering (ICIME) 366 - 369
Degrieck J. , Van Paepegem W. , Boone P. 2001 “Application of digital phase shift shadow Moiré to micro deformation measurement of curved surface,” Optics and Lasers in Engineering 36 29 - 40
Spagnolo G. S. , Ambrosini D. , Paoletti D. 2004 “Low-cost optoelectronic system for three-dimensional artwork texture measurement,” IEEE Transactions on Image Processing 13 390 - 396
Wenzel K. , Antal A. , Molnar K. , Toth B. , Tamas P. 2009 “New optical equipment in 3D surface measuring,” Journal of Automation, Mobile Robotics & Intelligent 3 29 - 32
Avilaq A. J. , Rezaie A. H. 2013 “3D face reconstruction using modified shadow moiré,” 21st Iranian Conference on Electrical Engineering 1 - 5
Mohammadi F. , Madanipour K. , Rezaie A. H. 2010 “Application of digital phase shift Moiré to reconstruction of human face,” IEEE European Modelling Symposium on Computer Modelling and Simulation 306 - 309
Liu H. , Cartwright A. , Basaran C. 2004 “Experimental verification of improvement of phase shifting Moiré interferometry using wavelet-based image processing,” Opt. Eng. 43 1206 - 1214
Lu Y. , Wang X. , Zhang X. 2007 “Weighted least-squares phase unwrapping algorithm based on derivative variance correlation map,” Optik 118 62 - 66
Ghiglia D. C. , Romero L. A. 1994 “Robust two-dimensional weighted and unweighted phase unwrapping that uses fast transforms and iterative methods,” J. Opt. Soc. Am. 11 107 - 117