Advanced
Projection-Type Integral Imaging Using a Pico-projector
Projection-Type Integral Imaging Using a Pico-projector
Journal of the Optical Society of Korea. 2014. Dec, 18(6): 714-719
Copyright © 2014, Optical Society of Korea
  • Received : September 09, 2014
  • Accepted : October 10, 2014
  • Published : December 25, 2014
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Yucheol Yang
Sung-Wook Min
mins@khu.ac.kr
Abstract
A pico-projector is a compact and mobile projector that has an infinite focus. We apply the pico-projector to a projection-type integral imaging system, which can expand the image depth to form multiple central depth planes. In a projection-type integral imaging system, the image flipping problem arises because the expanded elemental images pass through a lens array. To solve this problem, we propose the ray tracing of a pico-projector at a central depth plane and compensate the elemental image using a pixel-mapping process. Experiments to verify the proposed method are performed, and the results are presented.
Keywords
I. INTRODUCTION
It has been demonstrated that integral imaging is an autostereoscopic technique that shows a full-color, full-parallax, real three-dimensional (3D) image [1] . Also, real-time 3D pictures can be provided with relatively low eye fatigue by an integral imaging display. However, integral imaging systems have many challenges, such as a narrow viewing angle, the spatial limitation of resolution, and the expressible depth range. It has been reported that these parameters exhibit a trade-off relationship [2] . Since the first proposal by Lippmann, integral imaging systems have been further developed and improved by many research groups [3 - 5] .
A projector can be used as the display device for the elemental images in an integral imaging system, which is called projection-type integral imaging. Employing a diffuser screen in the elemental image plane, the projection-type integral imaging system uses the same method as an ordinary integral imaging system [6 - 11] , but, due to the diffuser, the resolution and brightness of the elemental image are restricted in projection-type integral imaging. To overcome these weaknesses, previous studies have proposed some kinds of projection-type systems without the diffuser and reported a method for evaluating the visibility of the reconstructed images in terms of the fill factor [12] . This system has the strength that spatial and/or temporal multiplexing is feasible. Also, multiple elemental image layers can be used within the depth of field of the projector [13 , 14] .
In this paper, we apply a pico-projector to projection-type integral imaging, without a diffuser, in real-display mode. Due to the infinite focus of the pico-projector, the location of central depth plane (CDP) can be controlled in a projection-type integral imaging system. Using matrix analysis, ray tracing is performed at the CDP, and the compensation of elemental images for this system is proposed. Some experiments and their results are presented to prove the feasibility of the proposed method.
II. PROJECTION-TYPE INTEGRAL IMAGING SYSTEM USING A PICO-PROJECTOR
The pico-projector is a compact and mobile projector that has an infinite focus. As this technology is expected to be embedded in other devices such as smart phones and digital cameras, pico-projectors are drawing much attention. According to its operating method, a pico-projector is categorized according to three types: digital light processing (DLP), liquid crystal on silicon (LCoS), or laser beam steering (LBS). Figure 1 (a) shows the basic layout of a LBS-type picoprojector [15] . As shown, there is no projection lens set. The projected beam with low numerical aperture (NA) leaves the pico-projector directly and creates an image that is always in focus. The projected image grows as its distance from the pico-projector increases, as shown in Fig. 1(b) .
PPT Slide
Lager Image
(a) Basic layout of an LBS-type pico-projector. (b) Field of view of an LBS-type pico-projector.
In an integral imaging system, the reconstructed image can be formed near the CDP. The location of the CDP depends on the gap between the lens array and the elemental image, according to the Gauss lens law. In a projectiontype integral imaging system without a diffuser, multiple elemental images can be projected within the depth of field, which generates multiple central depth planes [16] . Because the depth of field of a pico-projector is almost infinite, the region of expressible CDP can be expanded by using multiple elemental image planes. As shown in Fig. 2 , elemental images with different gaps can be projected and integrated into reconstructed images with different CDPs. This means that the location of the CDP can be controlled solely by image processing, without changing the distance between projector and lens array. In this paper, we divide projection-type integral imaging systems into two cases, to analyze and propose the processing for the compensated elemental images.
PPT Slide
Lager Image
Expansion of CDP in an integral imaging system using a pico-projector.
First, as in a conventional integral imaging system, the elemental image is aligned with the gap g between the lens array and the displayed image. In this case, when the rate of image expansion is considered, the projected elemental image with gap is equivalent to the expanded image at the lens array. As shown in Fig. 3 , the size X of the elemental image at the lens array is given by
PPT Slide
Lager Image
Image flipping problem in a projection-type integral imaging system.
PPT Slide
Lager Image
where z is the distance between the pico-projector and the lens array, g is the gap between the elemental image and the lens array, and L is the total size of the lens array.
Because the size of the elemental image at the lens array is bigger than the lens array, some rays passing through neighboring lenses lead to image flipping. As the gap becomes bigger and the distance z becomes smaller, the part of the image passing through neighboring lenses grows.
To solve the image flipping problem in a projection-type integral imaging system, the size of the elemental image can be matched to that of the elemental lens at the location of the lens array, where the gap is zero in Eq. (1). In this case image flipping does not occur, but the size and location of the reconstructed image are deformed from their intended values, due to gap variation. Therefore, elemental image compensation seems necessary to render the integrated image without distortion or image flipping.
In this paper, we analyze the image transfer matrix for a projection-type integral imaging system, and propose an elemental image compensation method to resolve the deformation of the reconstructed image, to present the intended size and depth of the integrated image without the image flipping problem.
III. MATRIX ANALYSIS AND PROPOSED SYSTEM
The compensation of the elemental image for the pico-projector is implemented using pixel mapping. Because each pixel has an angle correspondeding to the pixel emitted from the pico-projector, ray tracing at the CDP becomes effective and simple with the matrix method. The elemental image projected at the lens array passes the lens array and travels to the CDP, which is expressed by
PPT Slide
Lager Image
where dCDP is the distance between the lens array and the CDP, ypico-projector is the distance between a ray from the pico-projector and the center of the corresponding lens at the lens array, θpico-projector is the angle of the ray at the lens array, yreal is the distance between a ray from the pico-projector and the center of the corresponding lens at the CDP, θreal is the angle of the ray at the CDP, and f is the focal length of the lens array.
The angle θpico-projector depends on the position ypico-projector and the distance z between the lens array and the pico-projector. Substituting Eqs. (3), Eqs. (2) may be approximated.
PPT Slide
Lager Image
PPT Slide
Lager Image
where n is the number of the corresponding elemental lens. In Eq. (4), yreal is the linear expression of ypico-projector . The magnification is related to the distance z , and the shifting distance takes on different values according to the number of the elemental lens in the lens array. In a projection-type integral imaging system using a pico-projector, the reconstructed image at the CDP is a function of the distance z between the lens array and the pico-projector.
A conventional integral imaging system is described by
PPT Slide
Lager Image
where yimaging is the distance between a ray from the integral imaging system and the center of the corresponding lens at gap g , and θimaging is the angle of the ray at the gap g . yreal in Eq. (5) represents the pinhole-array approximation; this expression is the same with inverse reaction of the pickup process, but the yreal in Eq. (4) is different than the inverse reaction of the pickup process.
The elemental image of a projection-type integral imaging system must be compensated to represent the integrated image at the intended location with the exact intended size. This compensation can be performed to match the corresponding terms of Eq. (4) with Eq. (5), which means ypico-projector of Eq. (4) should be calculated using yimaging of Eq. (5). Figure 4 is a schematic diagram of the pixel mapping process for the proposed compensation that adjusts each elemental image according to the change of the sub-image region. We implement the proposed algorithm using matrix analysis, and reproduce the elemental image with different z and g .
PPT Slide
Lager Image
Compensation of an elemental image using a pixel-mapping process.
IV. EXPERIMENTS AND RESULTS
Figure 5 shows a schematic diagram of the experimental setup. We use the SHOWX+TM laser pico-projector with WVGA resolution (848 × 480). In the experiment, we set the distance z between pico-projector and lens array so that the size of the elemental image is aligned at the lens array. The pixel resolution of the elemental image is dictated by the total pitch of the lens array and the distance z between pico-projector and lens array. The lens array has 13 × 13 square lenses, the size of each elemental lens being 10 mm × 10 mm. The focal length of the lens array is 15 mm.
PPT Slide
Lager Image
Experimental setup.
Like the system shown in Fig. 2 , two elemental images with different gaps are used to confirm the expansion of the expressible depth. Also, the sizes and locations of the reconstructed images must be adjusted to be like the designed reconstructed images. As shown in Fig. 6 , images 1 and 2 are intended to represent the reconstructed image. The size of each square in images 1 and 2 is the same. Two elemental images are generated from image 1 with a gap of 2.14 cm, and from image 2 with a gap of 1.8 cm. Comparing the experimental results for three cases, the reconstructed images are observed using a diffuser and measured at the distance dobserved between lens array and diffuser. In the first case, as shown in Fig. 7(a) , the elemental images are not compensated but simply magnified according to the gap g, the expanded elemental images being projected to the lens array. In the second case, as shown in Fig. 7(b) , the elemental images are resized and matched to the size of the elemental lens at the position of the lens array. In the last case, as shown in Fig. 7(c) , the elemental images are compensated using the proposed method.
PPT Slide
Lager Image
Intended images 1 and 2 for computer-generated elemental images.
PPT Slide
Lager Image
Projection-type integral imaging systems and projected elemental images for (a) case 1 (no compensation), (b) case 2 (size adjustment), and (c) case 3 (elemental image compensation).
Figure 8 shows the elemental images on the diffuser located at the different gaps, 2.14 cm and 1.8 cm, generated for case 1 of Fig. 7 . The elemental image without compensation is used to check the changes of the elemental image planes. In this experiment, two elemental image planes are implemented. Figure 8(a) shows the elemental image when the diffuser is set 2.14 cm from the lens array, while in Figure 8(b) it is located at 1.8 cm. Comparison of the period of the elemental image and the size of elemental lens show that the elemental image plane for the red image is located at gap 1, while the elemental image plane for the blue image is set at gap 2. The integrated images obtained from these elemental images prove this assumption, as shown in Fig. 9 , as they are located at the intended position calculated using the lens equation.
PPT Slide
Lager Image
Elemental image planes on a diffuser located at (a) g2 = 1.8 cm and (b) g1 = 2.14 cm.
PPT Slide
Lager Image
Experimental results for reconstructed images for the three cases in Fig. 7.
Figure 9 shows the experimental results. The first column in Fig. 9 shows the results for case 1, while the second and third columns show the results for cases 2 and 3. The reconstructed images for images 1 and 2 are located at each dobserved , as shown in the first and second rows of Fig. 9 respectively.
As expected, case 1 presents the image flipping problem, and in case 2 the reconstructed images of elemental images 1 and 2 are located at 5.5 cm and 11 cm respectively, which are not the same as the designed depth. Especially, the reconstructed image of elemental image 2 in the second row shows a problem in terms of its size. Meanwhile, case 3 shows the intended size and depth, without image flipping.
V. CONCLUSION
In this paper, we apply a pico-projector to a projection-type integral imaging system to expand the CDP. We propose an elemental image compensation method using matrix analysis and an image mapping algorithm. The results of basic experiments show that the proposed system has multiple CDPs without the image flipping problem, unlike a conventional system.
The image flipping problem occurs in a projection-type integral imaging system due to the divergence property of the projector. In this paper, the compensation is performed assuming low numerical aperture light from the pico-projector. However, the proposed method could also be applied directly to a system adopting a conventional projector, considering the working distance of the chosen projector. So to speak, the proposed compensation method can be used to represent the intended integrated image without a flipped image for a projection-type integral imaging system in general. Experiments are performed to verify the reconstructed image without the image flipping problem. In this paper, reconstructed images with two different depths are investigated using a diffuser. Plane images are used for convenience in the experiments, yet not only separated plane images but also a continuous-volume image can be represented using the proposed system. However, the visibility of the integrated image might be degraded, so as to be unobservable by the naked eye, because of the small aperture of the projection system. This low visibility is caused by the aperture limitation of the projection system and generally occurs in a projection-type integral imaging system without a diffuser, as reported in previous studies [12] . The main proposal and contribution of this paper concern elemental image compensation to resolve image flipping and deformation, not visibility improvement. The reconstructed image on a diffuser located at the CDP can verify the proposed method to modify the size and location of the integrated images, without image flipping. Visibility enhancement could be achieved by adopting special viewing aids like a 3D screen, which is an optical device to resolve the aperture limitation due to the projection lens set by keeping the parallaxes of a 3D image, and can be a proper solution to improve the visibility of projection-type integral imaging [17 - 19] . Further studies are needed to apply a 3D screen to the proposed system effectively, which could increase the practical utility of the projection-type integral imaging system.
Acknowledgements
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. 2013-067321).
References
Lippmann G. 1908 “La photographie integrale,” Comptes Rendus Acad. Sci. 146 446 - 451
Min S.-W. , Kim J. , Lee B. 2005 “New characteristic equation of three-dimensional integral imaging system and its applications,” Jpn. J. Appl. Phys. 44 L71 - L74
Javidi B. , Okano F. 2002 Three Dimensional Television, Video, and Display Technologies Springer
Lee B. , Park J.-H. , Min S.-W. , Poon T.-C. 2006 Digital Holography and Three-Dimensional Display Springer “Three-dimensional display and information processing based on integral imaging,” 333 - 378
Park J.-H. , Hong K. , Lee B. 2009 “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48 H77 - H94
Kim Y. , Choi H. , Kim J. , Cho S.-W. , Kim Y. , Park G. , Lee B. 2007 “Depth-enhanced integral imaging display system with electrically variable image planes using polymer-dispersed liquid-crystal layers,” Appl. Opt. 46 3766 - 3773
Liao H. , Iwahara M. , Hata N. , Dohi T. 2004 “High-quality integral videography using a multiprojector,” Opt. Express 12 1067 - 1076
Jang J. S. , Oh Y. S. , Javidi B. 2004 “Spatiotemporally multiplexed integral imaging projector for large-scale high-resolution three-dimensional display,” Opt. Express 12 557 - 563
Jang J.-S. , Javidi B. 2004 “Three-dimensional projection integral imaging using micro-convex-mirror arrays,” Opt. Express 12 1077 - 1083
Okui M. , Arai J. , Nojiri Y. , Okano F. 2006 “Optical screen for direct projection of integral imaging,” Appl. Opt. 45 9132 - 9139
Martínez-Cuenca R. , Navarro H. , Saavedra G. , Javidi B. , Martínez-Corral M. 2007 “Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system,” Opt. Express 15 16255 - 16260
Park S.-g. , Song B.-S. , Min S.-W. 2010 “Analysis of image visibility in projection-type integral imaging system without diffuser,” J. Opt. Soc. Korea 14 121 - 126
Kim Y. , Park S.-g. , Min S.-W. , Lee B. 2009 “Integral imaging system using a dual-mode technique,” Appl. Opt. 48 H71 - H76
Jang J.-S. , Jin F. , Javidi B. 2003 “Three-dimensional integral imaging with large depth of focus by use of real and virtual image fields,” Opt. Lett. 28 1421 - 1423
Davis W. O. , Sprague R. , Miller J. 2008 “MEMS-based pico projector display,” Proc. IEEE/LEOS Int. Conf. Opt. MEMs Nanophoton. 31 - 32
Kim Y. , Park S.-G. , Min S.-W. , Lee B. 2011 “Projection-type integral imaging system using multiple elemental image layers,” Appl. Opt. 50 B18 - B24
Min S.-W. , Hong J. , Lee B. 2004 “Analysis of an optical depth converter used in a three-dimensional integral imaging system,” Appl. Opt. 43 4539 - 4549
Song B.-S. , Choi S. , Sung H. , Min S.-W. 2014 “Reflection-type three-dimensional screen using retroreflector,” J. Opt. Soc. Korea 18 225 - 229
Min S.-W. 2014 “A novel 3D screen,” SPIE Newsroom http://spie.org/x108294.xml