An offaxis integral floating system using a concave mirror is analyzed to resolve the image distortion incurred by the offaxis optical arrangement. The concave mirror can be adopted as the floating device to improve the optical efficiency. The image distortion due to the tilting axis of the concave mirror needs to be analyzed precisely to generate the predistortion image. In this paper, we calculate the image deformation in the offaxis structure of the concave mirror using the geometrical optics. Using the calculation results, the compensated elemental image can be generated for the predistortion integrated image, which can be projected to the floating 3D image without image distortion. The basic experiments of the offaxis integral floating are presented to prove and verify the proposal.
I. INTRODUCTION
Integral imaging is one of the emerging autostereoscopic threedimensional (3D) display techniques, which reconstructs a 3D image to integrate the elemental image through a lens array or pinhole array
[1

4]
. The elemental image is composed of the arrayed 2D images which have the directional information of the 3D object. The integral imaging system can express the full parallax and the various depth 3D images which, however, have the limitations of the expressible depth range and the viewing angle. Recently, many techniques have been researched and reported to enhance the 3D image quality of the integral imaging display
[5

11]
. Moreover, many integralimagingbased techniques have also been studied to inherit the merits of the integral imaging.
Integral floating is one of the techniques based on the integral imaging
[12

17]
. The integral floating employs the additional large aperture floating device in front of the integral imaging system, which can enhance the 3D image quality such as the improved depth range, the reduced seam noises from the lens array, and the increased viewing area. However, the general large aperture device is difficult to manufacture and often results in the image distortion.
The Fresnel lens, which is composed of the tiny concentric circular prisms that refract the parallel rays to a focal point, is generally used as the floating device in the integral floating system because it induces less aberration compared with the general spherical lens. However, the Fresnel lens cannot be perfectly free of the aberrations
[18

19]
and because of its circular prism patterns , circular seam noise and optical loss are inevitable, causing the degradation of the image quality
[5]
. The concave mirror which is optically the same with the convex lens can be used as the floating device to reduce the disadvantages of the Fresnel lens and to improve the floating image quality. However, using the concave mirror as the floating device incurs the problem of the onaxis system configuration, which means the object and the observer are located on the same side of the mirror, because the mirror switches the direction of the optical axis. Therefore, the object space and the image space of the concave mirror are overlapped in the onaxis floating system as shown in
Fig. 1
(a). In this case, it is impossible to compose the floating system without a beam splitter. Due to the characteristics of the beam splitter, the optical efficiency of the system should be decreased to less than
Integral floating system using concave mirror  (a) Conventional system and (b) proposed system.
50%. Furthermore, the system using the beam splitter should be complicated and bulky
[17]
.
The offaxis integral floating where the optical axis of system is tilted from that of the mirror can be a solution of the problems mentioned above as shown in
Fig. 1
(b). The offaxis integral floating system can accomplish the relatively simple system configuration with high optical efficiency without the seam noises from the Fresnel lens patterns. But the offaxis integral floating causes the image deformation which isresulted from the optical aberrations. Therefore, the predistortion image is needed to reconstruct a correct 3D floated image. In this paper, we analyze and simulate the image deformation due to the tilted angle between the optical axis of the system and the mirror. Using the simulation data, we can make the predistortion image which can be floated as the correct image. For the offaxis integral floating system, the compensated elemental image is generated for the predistortion integrated image. The experimental results of the image deformation analysis and the offaxis integral floating are presented to prove the feasibility of the proposal.
II. ANALYSIS OF IMAGE DISTORTION
The monochromatic optical aberrations can be classified into the five cases; the spherical aberration, the stigmatism,
Diagram of calculation for image distortion point Q.
(a) Experiment for image distortion of a 35mm cube, (b) experimental result and (c) simulation result.
the coma, the field curvature, and the distortion. The former three cases are the focusing error, which means the rays cannot converge on the focal point or the image point because of the surface curvature and the tilting axis, and can influence the image definition. The other two cases are the error about the image position due to the tilting axis and the chief rays. These are about the image deformation and must be considered in the offaxis image process. We start with the simulation of the image deformation due to the tiling angle using the simple geometrical optics.
Figure 2
shows the simulation for the image distortion. Firstly, we select two rays which start to travel from one source point P of the object and have the sufficiently small variations of the angle from the center ray from the source point and travels toward the center point of the spherical concave mirror. And then we find the intersection point of the two reflection rays which is the image point Q. Applying this process to the other points of the object, we can get the whole reflection image of the object.
Fig. 3
shows the comparison of the image of the real object and the simulation results. The pattern of simulation results is coincident with the real image distortion.
Based on the simulation of the image distortion, we can make the process to resolve the image distortion because the process of the image formation is reversible. In other words, we can find the predistortion image which forms the correct image through the concave mirror by performing the reverse process of the simulation for the image distortion, so that the predistortion image can compensate the image distortion.
III. PREDISTORTION PROCESS
In this chapter, we deal with the predistortion process to resolve the image distortion, which means the sequential operations to generate the compensated image. In order to find the predistortion image, we assume that an observer’s eye sees the point P
_{o}
which is one point of the correct 3D image as shown in
Fig. 4
. Then, we can find the predistortion point Q
_{o}
which converges to the point P
_{o}
after the reflection by the concave mirror. The point Q
_{o}
can be calculated through finding the chief ray that expresses the viewing direction and applying the reverse method of the simulation for the image distortion.
The chief ray is determined by the point P
_{o}
and the position of the observer’s eye. The reflection point C
_{o}
can be calculated by solving the equations of the chief ray and the concave mirror. To make the calculation simple, the points P
_{o}
and C
_{o}
should be projected onto the zx plane. The distance from the point O to the point P
_{o}
,
and the angle between
and the optical axis of the concave mirror, θ
_{P}
, are given as follows;
Diagram of obtaining predistortion image point Q_{o}.
When the point C
_{o}
is projected to the point C (0, 0, R), where R is radius of the concave mirror. The x, y, and z coordinates of the point P, p
_{1}
, p
_{2}
, and p
_{3}
, are given as;
And then, we should calculate the predistortion point Q which is the concave mirror image of the point P.
Fig. 5
shows the process for calculating the point Q. We select the two rays very near the chief ray, l
_{B}
and l
_{D}
, which start from the point P. When the reflection rays of l
_{B}
and l
_{D}
are defined as l
_{B}
’ and l
_{D}
’, the point Q can be obtained by solving the equations of l
_{B}
’ and l
_{D}
’. The point Q(q
_{1}
, q
_{2}
, q
_{3}
) is given by
Because the point Q is the predistortion point just on the zx plane, we should project the point Q onto the original image plane, and then the real predistortion point Q
_{o}
can be derived. This projection process can be performed by the three linear equations derived by the location relationships among the points on the original image plane and zx plane. Because the point Q
_{o}
is on the original image plane, the first equation of the simultaneous equations is derived by finding the plane equation of the original image plane
Diagram of calculation process in zx plane for predistortion image point Q.
where the vector
is a normal vector of the original image plane. The other two equations can be derived by two scalar production;
Finally, the predistortion point Q
_{o}
can be obtained by solving the three equations, (9), (10), and (11) as follows;
As mentioned above, the whole predistortion image for the correct 3D image can be found by adopting this method to the other points of the correct 3D image. The predistortion image can be used to generate the compensated elemental image which can be reconstructed to the predistortion integrated image.
The obtained predistortion image is for only one view of the observer’s eye. If the viewing direction of observer is moved, the chief ray should be changed. Therefore, the predistortion image should be generated according to the viewing direction of the observer.
IV. RECONSTRUCTION PROCESS
Using the predistortion process, we can reconstruct the correct 3D image without the image distortion using the offaxis integral floating system with the concave mirror. The sequential operations of the reconstruction process are as follows; (1) the 3D image to be displayed is assumed and defined as the certain coordinates. (2) The predistortion image coordinates are calculated by the predistortion process as described in chapter 3. (3) The compensated elemental image is generated using the predistortion image coordinates in consideration of the system specification of the integral imaging system to be used. (4) The predistortion integrated image is reconstructed by the defined integral imaging system. (4) Finally, the concave mirror is set up in front of the integral imaging system with the proper location and angle to project the correct 3D floating image.
Through the process, we can achieve the offaxis integral floating system using the concave mirror with the high efficiency and the simple structure and no distortion.
V. EXPERIMENT
In order to prove the compensation effect of the proposed method, the 15° offaxis integral floating system is set up. The 3D image to be displayed is composed of three characters, ‘K’, ‘H’, and ‘U’ whose heights are 35 mm. As shown in
Fig. 6
(a), the character ‘H’ is located on 300 mm along the zaxis and 15° tilted from the axis. The character ‘K’
(a) Experimental condition and (b) front scene of computergenerated 3D image.
and ‘U’ are located on 30 mm nearer and farther than the character ‘H’ along the optical axis, respectively.
Fig. 6
(b) shows the front scene of the computergenerated three characters used as a 3D object.
Table 1
describes the specifications of the experimental setup such as the lens array, the display device, and the concave mirror. The pickup process of the compensated elemental image is performed by the computational simulation tools.
Fig. 7
(a) shows the elemental images of the noncompensated image while
Fig. 7
(b) shows the compensated
Specifications of experimental setup
Specifications of experimental setup
Elemental images of (a) noncompensated image and its partially magnified image and (b) predistortion image and its partially magnified image when offaxis and viewing angle are 15°.
elemental imaging for the predistortion integrated image when both angles of the offaxis and the view are the same as 15°.
Figure 8
shows the implemented offaxis integral floating system. And
Fig. 9
shows the predistortion image reconstructed by the integral imaging system which is part of the offaxis integral floating system.
Fig. 10
and
11
represent the experimental results. The images of experimental results can be taken by camera located on the positions of the observer’s eyes indicated in
Fig. 6
(a). As shown in
Fig. 11
, the noncompensated 3D image and the compensated 3D image are compared with each other when the angles of the offaxis and of the view are the same as 15°. The character ‘K’ becomes larger than the other characters due to the lens effect of the concave mirror. Reversely, the character ‘U’ becomes smaller due to the same reason for
Experimental setup of offaxis integral floating system.
Predistortion 3D image reconstructed by integral imaging system with elemental image of Fig. 7 (b).
(a) Noncompensated 3D image and (b) compensated 3D image for 15° of offaxis angle and viewing angle.
Simulation images for angle of view in (a) 12.5°, (b) 15°, and (c) 17.5° and experimental results for angle of view in (d) 12.5°, (e) 15°, and (f) 17.5°.
the character ‘K’. The characters in the noncompensated 3D image are wider as the characters get farther from the zaxis. The characters in the compensated 3D image have mostly uniform sizes compared with the noncompensated 3D image.
Figure 11
shows the results of the compensated 3D images in three different views, 12.5°, 15°, and 17.5°.
Fig. 11
(a), (b), and (c) are the simulation results and
Fig. 11
(d), (e), and (f) are the experimental results in each view. The simulation results and the experimental results are coincident with each other.
VI. CONCLUSION
The image distortion of the offaxis integral floating system using the concave mirror is analyzed to resolve. In the integral floating system using the concave mirror, the offaxis structure is inevitable, which results in the image distortion. So, we analyze the image distortion and generate the predistortion image which compensates the image deformation caused by the tilted angle through the predistortion process. And then, we can realize the distortionfree offaxis integral floating system with the high efficiency and the simple structure. The experimental results for oneview are presented in only one each elemental image. However, the freeviewing direction system can be realized by using the headtracking method or the directionally selective pickup method for the integral imaging system.
Acknowledgements
This research was supported by Ministry of Knowledge Economy(MKE), Korea as a project, “Development of interactive user interface based 3D system”.
Lippmann G.
(1908)
“Épreuves réversibles: photographies intégrals”
C. R. Acad. Sci.
146
446 
451
Ives H. E.
(1931)
“Optical properties of a Lippmann lenticulated sheet”
J. Opt. Soc. Am.
21
171 
176
Okoshi T.
(1980)
“Threedimensional displays”
Proc. IEEE
68
548 
564
Okano F.
,
Arai J.
,
Hoshino H.
,
Yuyama I.
(1999)
“Threedimensional video system based on integral photography”
Opt. Eng.
38
1072 
1077
Min S.W.
,
Jung S.
,
Park J.H.
,
Lee B.
(2002)
“Study for wideviewing integral photography using an aspheric Fresnellens array”
Opt. Eng.
41
2572 
Jang J. S.
,
Jin F.
,
Javidi B.
(2003)
“Threedimensional integral imaging with large depth of focus using real and virtual image fields”
Opt. Lett.
28
1421 
1423
Min S.W.
,
Hong J.
,
Lee B.
(2004)
“Analysis of an optical depth converter used in a threedimensional integral imaging system”
Appl. Opt.
43
4539 
4549
Lee B.
,
Hong J.
,
Kim J.
,
Park J.H.
(2004)
“Analysis of the expressible depth range of threedimensional integral imaging system”
J. Opt. Soc. Korea
8
65 
71
Yamamoto K.
,
Mishina T.
,
Oi R.
,
Senoh T.
,
Okui M.
(2009)
“Crosstalk elimination using an aperture for recording elemental images of integral photography”
J. Opt. Soc. Am. A
26
680 
690
Park J.H.
,
Hong K.
,
Lee B.
(2009)
“Recent progress in threedimensional information processing based on integral imaging”
Appl. Opt.
48
H77 
H94
Wei T.C.
,
Shin D.H.
,
Lee B.G.
(2009)
“Resolutionenhanced reconstruction of 3D object using depthreversed elemental images for partially occluded object recognition”
J. Opt. Soc. Korea
13
139 
145
Min S.W.
,
Hahn M.
,
Kim J.
,
Lee B.
(2005)
“Threedimensional electrofloating display system using an integral imaging method”
Opt. Express
13
4358 
4369
Kakeya H.
(2007)
“MOEVision: simple multiview display with clear floating image”
Proc. SPIE
6490
64900J 
Kakeya H.
(2008)
“Formulation of coarse integral imaging and its applications”
Proc. SPIE
6803
680317 
Kim J.
,
Min S.W.
,
Kim Y.
,
Lee B.
(2008)
“Analysis on viewing characteristics of an integral floating system”
Appl. Opt.
47
D80 
D86
Kim J.
,
Min S.W.
,
Lee B.
(2009)
“Viewing window expansion of integral floating display”
Appl. Opt.
48
862 
867
Jung J.
,
Hong J.
,
Lee B.
,
Min S.W.
2011
“Augmented reality system based on integral floating method”
Optical Society of America
in Proc. Digital Holography and Threedimensional Imaging, OSA Techinal Digest (CD)
USA
paper DTuC22.
Moon J.W.
,
Lim G.T.
,
Jang S.J.
,
Koo J.S.
,
Park J.I.
,
Kim E.S.
(2005)
“Compensation of image distortion in Fresnel lensbased 3D projection display system using a curved screen”
Proc. SPIE
6016
601616 
Baasantseren G.
,
Park J.H.
,
Erdenebat M.
,
Seo S.W.
,
Kim N.
(2010)
“Integral floating image display using two lenses with reduced distortion and enhanced depth”
J. Soc. Inf. Display
18
519 
526