A new 3D reconstruction algorithm for the human face is proposed using the derivative Moiré topography which ensures fast and robust reconstruction even for rough surfaces. The Moiré interference fringe pattern is initially obtained through the projection Moiré topography based on phase shifting, and then differentiated to provide a full unwrapped phase map for a human face. 2π ambiguity, which has been a chronically unsolved problem with Moiré topography, is successfully surmounted by differentiating the Moiré fringe patterns both in x and ydirections when the object is located in the xy plane. A real human face is used for verifying the proposed derivative Moiré topography. A human face of 4 different phaseshifted images taken in the fixed plane is almost fully reconstructed in 3D format in 0.1 mm lateral resolution.
I. INTRODUCTION
Noncontact 3D surface reconstruction techniques have been extending their applications to various industrial fields such as medical, manufacturing and biometric areas. A large number of noncontact 3D reconstruction techniques developed so far show their advantages and drawbacks as listed in
[1]
. Among them Moiré topography has been of prime interest because it provides fast operation for smooth surfaces since it was introduced
[2

4]
. For Moiré topography, when the projected grating of the projection system falls on the object, the deformed grating is generated on its surface, Moiré interference fringe patterns are then generated on the imaging system, yielding contours of the same heights expressed in wrapped phases. Due to diffraction, a virtual grating is often employed for replacing reference and/or projected gratings. Projection Moiré topography was thereafter developed to increase the resolution significantly using the virtual grating for the projection system. In the meantime, the phase shifting method has been devised to ensure repeatability of the projection Moiré topography where multiple phase shifts are carried out to easily calculate the phase map for objects
[5
,
6]
.
Although Moiré topography usually ensures fast and efficient 3D reconstruction in smooth surfaces
[7
,
8]
, two big obstacles of noise and imperfect phase unwrapping make one hesitate to capitalize on this. To eliminate noise occurring in a wrapped phase map, cosine transform or wavelet transform has been tried
[9

12]
. The projection Moiré topography with phase shifting method provides simple and reliable wrapped phase images for objects with –π/2 to π/2 range, called 2π ambiguity, however it is very difficult to convert this wrapped phase to a noise free unwrapped phase map. Abrupt phase change more than π cannot be detected easily, so that very few techniques have been tried for achieving more exact phase unwrapping. The weighted least squares method with discrete cosine transform (DCT) was introduced and utilized where the least squares method was used for removing discontinuity (or noise) in the phase image and DCT was employed for phase unwrapping in a strict sense, however this method shows difficulty in tracking a sudden change of phase while unwrapping the wrapped phase image, and it requires huge computing power because of a large amount of computation
[13]
.
In this work, the derivative Moiré topography by differentiating the wrapped phase map separately in x and ydirections is proposed to resolve the 2π ambiguity problem. Compared with the least squares method with DCT, the proposed technique tracks sudden phase change very efficiently using the simple differentiation in x and ydirections. This also does not introduce a large amount of computational burden, leading to very fast attainment of phase unwrapping.
II. THE PROPOSED MOIRÉ TOPOGRAPHY
The classical measurement setup to obtain the Moiré fringe pattern using the phase shifting method is shown as illustrated in
Figure 1
where
I_{d}
(·) is the light intensity of the deformed grating at (x, y) under test,
I_{r}
(·) is the light intensity of the reference grating with phaseshifting, and Ik indicates the resulting Moiré intensity at (x, y) when Δ(
t
) = (
k
– 1)×90°. To avoid diffraction virtual projected and reference gratings for the phase shifting method can be established as shown in
Figure 2(a)
where the object is located in the xy plane, projection and imaging systems are apart by
d
, and the object is away by
L
from the projection system.
Figure 2(b)
describes the real measurement setup used for this work where four different virtual gratings for the projection and the reference are devised by applying the phase difference of 90° between consecutive gratings. A light source with four virtual projected gratings is emitted in sequence from the projection system and creates the Moiré interference fringe patterns equivalently to
Fig. 1
. The Moiré fringe pattern is therefore created when the deformed grating in (1) is superimposed on the reference grating in (2). In principle, the intensity of the Moiré pattern,
I_{Moiré}
(
x, y
), is extracted from the fourth term of (3) which is the product of intensity of the deformed grating (
I_{1}
) and that of the transmitted imaging through the reference grating of the imaging system (
I_{2}
) where Is is the intensity of the light source,
A
is the modulation of gratings,
R
is the reflectance of the object surface,
M
is the magnifying rate of the projection system,
G
is the pitch of gratings, and
h
(
x, y
) is the height of the object as in (4)
[2
–
4]
. If the argument within cos (∙) is represented by
ϕ
(
x, y
) as described in (5), the height of the object can be calculated as shown in (6) since
in
Fig. 2(a)
. To obtain
ϕ
(
x, y
) to a high precision, the phase shifting method as described in
Fig. 2
is capitalized using the traditional equation as expressed in (3) where
ϕo
(
x, y
) means the principal value of the wrapped phase, 90° to 90°. For the phase shifting method, projected and reference gratings (
I_{r}
) can be generated virtually with a computer, the deformed grating (
I_{d}
) is obtained on the surface object as seen in (7), and the resulting interference pattern in (8) can be extracted by multiplying
I_{d}
and
I_{r}
with the computer, yielding no more diffraction and interference where
a_{d}
is the magnitude of the deformed grating,
a_{r}
is the magnitude of the reference grating, D(x,y) is the average intensity of the resulting interference pattern, γ(x,y) shows the visibility of the interference pattern, and ϕ(x,y) = ϕd(x,y)  ϕr(x,y) yields the phase of the interference pattern. As seen in (9),
ϕ°
(
x, y
) can only have the principal values from 90° to 90° which is called 2π ambiguity, therefore phase unwrapping should be carried out, in other words, the absolute Moiré order is required to be found accordingly.
The conventional Moiré topography based on the phase shifting method.
Measurement setup for the proposed Moiré topography. (a) Phase shifting method based on virtually generated 4 reference gratings. (b) Real measurement setup.
In this work, we perform the phase unwrapping by differentiating (9) twice in x and ydirections, leading to (10) where a primed subscript means the differentiation with respect to the subscript. Since the derivative phases as expressed in (10) are continuous functions even around the big phase jump, phase can be tracked precisely without difficulty even if it is greater than 90º or less than 90º. The principal phase is at first calculated using (9) and then one differentiation is carried out in one direction (
i.e
. x or ydirection) and thereafter another differentiation is taken in the other direction.
The absolute Moiré order at the (
i
,
j
)th pixel,
n_{i,j}
, to obtain the exact phase can be determined using (11) at first in the xdirection. In Moiré topography, the unwrapped phase map is composed of many contour lines where the absolute order difference between a contour and its neighboring contour is always 1 for the regular case in which no discontinuous contours exist. Even in the case for which the height of the object changes abruptly, contours are distributed densely not discontinuously. The irregular case occurs when two different heights are overlapped or black spots such as the pupil of eye, nostril or eyebrow are created, providing discontinuous contours, in which no Moiré patterns are generated and then no continuous phase contours are obtained. Therefore, the absolute order can be calculated using the derivative of phase change based on current phase contours. A sudden phase change more than nπ yields addition of n from the current absolute order, where the order is updated in x and ydirections while scanning all of the pixels of the object. But since the irregular case provides incorrect phase derivative, this error is accumulated through scanning up to the final pixel point.
where
fix
(•) function removes the fractional part and returns the integer part, adding the corresponding integer (n) value when the phase becomes greater than nπ.
Figure 3
illustrates an example showing the case for n equal to 1 at the
i
th pixel. With this
n_{i,j}
, the phase is unwrapped in the xdirection as expressed in (12),
Principle of phase unwrapping.
where the calculated phase from (12) may accumulate errors due to discontinuous contours. To resolve this error accumulation, phase compensation is carried out by calculating a different derivative with respect to wider step in the xdirection. Accumulated phase error is detected and incorrect phase becomes recovered by computation of intermediate order (
n^{xc}
) as represented in (13) where
a
needs to be predefined as a big number (usually ≥ 50). The left term for
j
≥
a
in (13) indicates accumulation of errors by summing phase variations from [(
k
1)
a
+1]
^{th}
pixel to [
ka
+1]
^{th}
pixel, and the right term expresses accumulation of errors by summing phase variation from [
ka
+1]
^{th}
pixel to jth pixel as shown in
Fig. 4
. For
j
＜
a
, phase variation from 1st pixel to
j
^{th}
pixel is calculated for
n^{xc}
. However, it should be noted that although phase error is not accumulated using (13) and (14), the absolute order cannot be obtained correctly in the vicinity of discontinuous contours due to overlapping heights or black spots.
Detection of phase error accumulation.
The absolute Moiré order is now calculated once more in the ydirection using (15),
Eventually the phase is unwrapped using (16),
III. MEASUREMENT RESULTS
To validate the proposed method, a human face as shown in
Fig. 5
is taken for an object which is located 60 cm away from the projection and imaging systems. The projection system used here has 1200×800 pixels and provides 28 cm×35 cm image to the plane 60 cm (
L
) away from the projection and imaging systems which are 11.5 cm (
d
) apart from each other. Projected and reference gratings are generated virtually with pitch (
G
) of 1 mm. The imaging system consisting of a CCD camera and a lens has 2592×1944 pixels, therefore the minimum resolution is calculated to be 0.144 mm in the xdirection and 0.135 mm in the ydirection. Four deformed gratings obtained on the surface of the object using four different projected gratings are exhibited in
Fig. 6
where the original images are converted to the corresponding gray values to provide the light intensity (
I_{k}
). Using deformed and reference gratings, Moiré fringe patterns are calculated initially using (9) where the phase is not unwrapped. The initial phase for Moiré fringe patterns is calculated by subtracting the phase of the reference grating from that of the deformed grating as deduced in (8). Due to abrupt phase variation, (10) to (16) are used for producing the exact phase unwrapping specifically in the vicinity of sudden phase change.
A human face photo used for this work. FIG.
4 different deformed grating images obtained with virtual reference gratings.
Figure 7
exhibits initial phase map obtained with (9) which shows the principal phase image from 90° to 90°.
Figure 8(a)
displays the unwrapped phase map applying the phase derivative in the xdirection,
Fig. 8(b)
shows the compensated phase map employing the additional derivative as introduced in (13), and
Fig. 8(c)
shows the finally unwrapped phase map which applied the phase derivatives in x and ydirections as introduced in (10) to (16). Discontinuous contours due to overlapping two different heights and black spots lead to failure of phase unwrapping as shown in
Fig. 8(c)
where black spots provide errors in obtaining exactly unwrapped phase, as expected in Sec. II. A detailed procedure from (11) to (16) is offered for verification of the proposed method as displayed in
Table 1
where phases before unwrapping are obtained from (9), and from these data the final unwrapped phases are successfully calculated using (11)(16). The negative phases illustrate relatively lowered phases with respect to the highest altitude of cheek in
Fig. 4
.
Initial phase wrapped image.
Phase unwrapped images based on (a) xdirected differentiation, (b) phase compensation, (c) ydirected differentiation.
Results for phases and orders at 10 pixels between brows inFig. 4
Results for phases and orders at 10 pixels between brows in Fig. 4
The final 3D picture of a human face is reconstructed as illustrated in
Fig. 9
. For a clear view, the background image is removed and textured for vivid visualization. Finally a moving average with 50×50 window size is performed for removing ripples specifically around black spots such as eyebrow and nostril, providing smooth 3D surface and adding a bit more CPU time. Vague stripes still appear on the cheek as seen in
Fig. 9
since the light intensity of projected gratings are not perfectly sinusoidal, leading to some discontinuity for phase unwrapping.
The final 3D reconstructed image.
Compared with the least squares method with DCT, the proposed derivative Moiré technique shows fast computation and precise reconstruction for even a rough surface such as a human face. DCT has been applied to unwrap the phase image of the human face as taken in
Fig. 5
, however it does not produce the complete 3D reconstruction. Therefore a very simple and periodic principal phase image was used to compare DCT and the proposed method as seen in
Fig. 10
. CPU times have been computed for three cases as summarized in
Table 2
, where the derivative Moiré topography shows much faster time taken for phase unwrapping.
A principal phase image used for comparison.
Comparison of two phase unwrapping methods in terms of CPU time
Comparison of two phase unwrapping methods in terms of CPU time
IV. CONCLUSION
3D reconstruction has been successfully implemented using the derivative Moiré topography based on phase unwrapping using differentiation of phase in x and ydirections. Abrupt phase change is identified successfully by this method, leading to the precise and fast height computation of objects containing rough surfaces. Differentiation is performed both in x and ydirections for obtaining noise free phase unwrapping. From this, errors occurring from inexact phase unwrapping are eliminated as the differentiation iterates. The classical phase shifting method is replaced by building four phaseshifted virtual reference gratings. A human face is exploited to validate the proposed method. This proposed method will provide a potential direction for systems necessitating robust and fast 3D reconstruction.
Acknowledgements
This research was supported by a grant (A131306) from Gyeonggi Technology Development Program funded by Gyeonggi Province, Korea.
Sansoni G.
,
Trebeschi M.
,
Docchio F.
2009
“Stateoftheart and applications of 3D imaging sensors in industry, cultural heritage, medicine, and criminal investigation,”
Sensors
9
568 
601
Takasaki H.
1970
“Moiré topography,”
Appl. Opt.
9
1467 
1472
Kafri O.
,
Glatt I.
1990
The Physics of Moiré Metrology
John Wiley & Sons
Amidror I.
2000
The Theory of the Moiré Phenomenon
Kluwer Academic Publishers
Dordrecht, Netherlands
Li N.
2010
“Simulation of a small feature gauging system using phaseshift projection Moiré,”
The 2nd IEEE International Conference on Information Management and Engineering (ICIME)
366 
369
Degrieck J.
,
Van Paepegem W.
,
Boone P.
2001
“Application of digital phase shift shadow Moiré to micro deformation measurement of curved surface,”
Optics and Lasers in Engineering
36
29 
40
Spagnolo G. S.
,
Ambrosini D.
,
Paoletti D.
2004
“Lowcost optoelectronic system for threedimensional artwork texture measurement,”
IEEE Transactions on Image Processing
13
390 
396
Wenzel K.
,
Antal A.
,
Molnar K.
,
Toth B.
,
Tamas P.
2009
“New optical equipment in 3D surface measuring,”
Journal of Automation, Mobile Robotics ＆ Intelligent
3
29 
32
Avilaq A. J.
,
Rezaie A. H.
2013
“3D face reconstruction using modified shadow moiré,”
21st Iranian Conference on Electrical Engineering
1 
5
Mohammadi F.
,
Madanipour K.
,
Rezaie A. H.
2010
“Application of digital phase shift Moiré to reconstruction of human face,”
IEEE European Modelling Symposium on Computer Modelling and Simulation
306 
309
Liu H.
,
Cartwright A.
,
Basaran C.
2004
“Experimental verification of improvement of phase shifting Moiré interferometry using waveletbased image processing,”
Opt. Eng.
43
1206 
1214
Lu Y.
,
Wang X.
,
Zhang X.
2007
“Weighted leastsquares phase unwrapping algorithm based on derivative variance correlation map,”
Optik
118
62 
66
Ghiglia D. C.
,
Romero L. A.
1994
“Robust twodimensional weighted and unweighted phase unwrapping that uses fast transforms and iterative methods,”
J. Opt. Soc. Am.
11
107 
117