Advanced
Integral-floating Display with 360 Degree Horizontal Viewing Angle
Integral-floating Display with 360 Degree Horizontal Viewing Angle
Journal of the Optical Society of Korea. 2012. Dec, 16(4): 365-371
Copyright ©2012, Optical Society of Korea
  • Received : September 09, 2012
  • Accepted : October 10, 2012
  • Published : December 25, 2012
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Munkh-Uchral Erdenebat
Department of Electrical and Computer Engineering, Chungbuk National University, Chungbuk 361-763, Korea
Ganbat Baasantseren
School of Information Technology, National University of Mongolia, Ulaanbaatar 210646, Mongolia
Nam Kim
Department of Electrical and Computer Engineering, Chungbuk National University, Chungbuk 361-763, Korea
Ki-Chul Kwon
Department of Electrical and Computer Engineering, Chungbuk National University, Chungbuk 361-763, Korea
Jina Byeon
Department of Electrical and Computer Engineering, Chungbuk National University, Chungbuk 361-763, Korea
Kwan-Hee Yoo
Department of Digital Informatics and Convergence, Chungbuk National University, Chungbuk 361-763, Korea
Jae-Hyeung Park
Department of Electrical and Computer Engineering, Chungbuk National University, Chungbuk 361-763, Korea
jh.park@cbnu.ac.kr
Abstract
A three-dimensional integral-floating display with 360 degree horizontal viewing angle is proposed. A lens array integrates two-dimensional elemental images projected by a digital micro-mirror device, reconstructing three-dimensional images. The three-dimensional images are then relayed to a mirror via double floating lenses. The mirror rotates in synchronization with the digital micro-mirror device to direct the relayed three-dimensional images to corresponding horizontal directions. By combining integral imaging and the rotating mirror scheme, the proposed method displays full-parallax three-dimensional images with 360 degree horizontal viewing angle.
Keywords
I. INTRODUCTION
Integral imaging is a distinguished technique among various three-dimensional (3D) display methods. It has two main parts: pick-up and reconstruction as shown in Fig. 1 . In the pick-up part, the light coming from 3D
Lager Image
Basic scheme of integral imaging technique.
objects passes through a lens array, being recorded at the sensor plane as a set of elemental images. The elemental images include different perspectives’ information of the captured 3D object. In the reconstruction part, a full-parallax 3D image can be displayed in real-time with different perspectives according to viewing direction from a recorded set of elemental images by using the same lens array. However, reconstructed 3D images have narrow viewing angles limited by the numerical aperture of each lens in the lens array [1 - 5] .
Some research has been conducted in order to solve the narrow viewing angle problem [6 - 8] . Shifting a dynamic barrier array which is allocated between display device and lens array [6] , or moving a lens array [7 - 8] in synchronization with the display device can improve the viewing angle of the integrated 3D images. However, those methods require two-dimensional (2D) scanning movement of the lens array or barrier array, and hence they are not easy to implement in real-time. Viewing angle enhancement methods without mechanical movement, such as using a double display device and time multiplexing [9] , embossed screen [10] , curved lens array and curved screen [11 - 12] , double collimated illumination [13] and negative index plano-concave lens array [14] have also been reported. Those methods, however, do not provide a viewing angle wide enough for a comfortable viewing experience.
Recently, a rotating-mirror-screen type volumetric light field display that projects 2D images to corresponding directions has been reported, demonstrating a 360 degree horizontal viewing angle which means displayed 3D image can be observed from any horizontal direction around the display system [15 - 17] . In an implementation, a high frame rate digital micro-mirror device (DMD) projects 288 different perspectives per round when the mirror screen rotates more than 900 rounds per minute (rpm) as shown in Fig. 2 . Main advantage of this system is that it can present 360 degree horizontal viewing angle 3D images, enabling walk-around viewing. However, the presented 3D images have only horizontal parallax and a viewer tracking system is required for vertical parallax.
Lager Image
Basic scheme of light field display using a rotating mirror screen.
Also the angular light ray sampling rate is given by the angular step of the rotating mirror which is limited by the frame rate of the projector. In order to present 3D images with angular light ray sampling rate high enough for correct accommodative response of the observer, the projector needs to operate at a very high speed, which can impose a problem in practical implementation.
Full-parallax omni-directional 3D display by combining integral imaging display with the rotating mirror screen configuration has been reported by D. Miyazaki et al. [18] and M.-U. Erdenebat et al. [19] In D. Miyazaki et al.’s work, however, experimental demonstration is not presented. M.-U. Erdenebat et al.’s work suffers from flipped images that degrade the final image quality.
In this paper, we report a new implementation of the full-parallax omni-directional 3D display by combining integral imaging display with the rotating mirror screen configuration. The proposed system can provide 3D images over 360 degree horizontal angle, overcoming the narrow viewing angle of the conventional integral imaging display. Full-parallax images are presented with a higher angular light ray sampling rate than conventional rotating screen light field display and without using any tracking system. Unlike previous works by D. Miyazaki et al. and M.-U. Erdenebat et al., the proposed system tailors the viewing angle of the relayed 3D images to match the angular step of the rotating mirror and uses a collimated elemental image projection to remove repeated 3D images in integral imaging display. In the following sections, we present the system design with an experimental result.
II. PROPOSED SYSTEM WITH 360 DEGREE HORIZONTAL VIEWING ANGLE
Figure 3 shows schematic configuration of the proposed method. For a given mirror angle, the corresponding elemental images are projected by a DMD projector to a lens array through a collimating lens. The lens array integrates the elemental images into a 3D image. The reconstructed 3D image is then relayed by a double floating lens system to the mirror space. Finally, the mirror directs the relayed 3D image to a specific direction. The DMD projector and the rotating mirror operate in synchronization such that proper 3D images can be projected to the corresponding directions.
In usual integral imaging display, the elemental images are displayed on a flat panel display or projected on a diffuser screen, which causes repeated 3D images in the imaging space [2] . These repeated images can be relayed to the rotating mirror screen and seen at the same time when the integral imaging is combined with the rotating mirror configuration [19] . In our system, a collimating lens is used instead of the diffuser screen, making only a correct single 3D image to be relayed to the rotating
Lager Image
Configuration of proposed system.
Lager Image
Flipped image effect: (a) when elemental images are projected on a diffusive screen and (b) when elemental images are collimated by a collimating lens.
mirror screen and observed [13] . Figure 4 illustrates the difference between the diffusive screen case and the collimating lens case.
In the proposed method, the double floating lens system which has a 4- f configuration as shown in Fig. 3 is used to relay the initial 3D image reconstructed by the lens array to the rotating mirror screen. One function of the double floating lens system is to eliminate longitudinal position dependency of the magnification ratio in the relay of the integrated 3D images [20] . Another function of the double floating lens system is to tailor the horizontal viewing angle of initial 3D image to match the angular step of the rotating mirror. The viewing angle of the relayed 3D image is given by:
Lager Image
where θI is the viewing angle of initially reconstructed 3D image, fFL1 and fFL2 are the focal lengths of the floating lenses, respectively. By controlling fFL1 and fFL2 , it is possible to match the viewing angle.
Note that the combination of the integral imaging display and the rotating mirror screen configuration with a viewing angle matching can enhance the angular light ray sampling rate over conventional rotating screen light field display at a given projector speed. In conventional rotating mirror screen light field display which projects a 2D image to the rotating mirror, the angular spacing between the reconstructed light rays is given by the angular step of the rotating mirror. In the proposed system, however, the angular spacing between light rays is given by θF/N where N is the one dimensional pixel count of each elemental image [21 - 22] which is N = 17 in our implementation. When the viewing angle θF of the relayed image is exactly matched to the angular step of the rotating mirror by use of the 4- f relay optics in the proposed method, the proposed method can exhibit N times higher angular light ray sampling rate over conventional 2D image projection rotating mirror screen light field display. Also note that θF does not need to be large comparable to the viewing angle of a conventional integral imaging display since it only needs to cover the angular step of the rotating mirror. Therefore it is possible to achieve a higher angular light ray sampling rate over N for each elemental image. Finally, it should also be noted that these advantages of the proposed method accompany reduction of the vertical viewing angle. In the proposed configuration, the vertical viewing angle is also given by θF which is smaller than the viewing angle of the conventional integral imaging display, limiting viewing region in the vertical direction.
In the proposed method, the elemental images are calculated considering the angle of the rotating mirror. As shown in Fig. 5 , let us define three coordinate systems ( x , y , z ), ( u , v , w ), and ( x″ , y″ , z″ ). ( x , y , z ) and ( x″ , y″ , z″ ) are fixed world coordinate systems with their origins at the center of the rotating mirror and the lens array, respectively. ( u , v , w ) is a local coordinate system with u and v on the mirror plane. For a desired 3D image point P ( x , y , z ), the corresponding ( u , v , w ) coordinates can be found as:
Lager Image
Lager Image
Coordinates for elemental image generation.
where ϕ is the azimuthal angle of the rotating mirror. The mirror reflected image P′ ( u′ , v′ , w′ ) = P′ ( u , v ,– w ) is then represented by ( x,y,z ) coordinates by:
Lager Image
Finally, P′ ( x′ , y′ , z′ ) point in the rotating mirror space
Lager Image
Experimental setup. Note that the system is laid down on the optical table.
corresponds to a point P″ ( x″ , y″ , z″ ) in the lens array space through a double floating lens system. The coordinates of the point P″ ( x″ , y″ , z″ ) is given by:
Lager Image
where g is the gap between image sensor and lens array, and fLA is the focal length of the elemental lens. Therefore, the elemental images for a point P″ ( x″ , y″ , z″ ) are generated following eqs. (1)-(4), in order to display a point P ( x , y , z ) in the rotating mirror space.
III. EXPERIMENTAL RESULTS
Figure 6 shows the experimental setup. We used a collimating lens with 85 mm focal length, a high-speed DMD projector with 1024×768 resolution, a lens array with 150×150 elemental lenses of 1×1 mm lens pitch and 3.3 mm focal length. The double floating lens system consists of two Fresnel lenses with 110 mm and 318 mm focal lengths, respectively. For a rotating mirror system, a flat mirror of 100×150 mm 2 size and a motor are used. The motor is synchronized to the DMD projector so that a different image can be projected in every 1.8 degrees on the rotating mirror. Note that in our experimental setup the whole system was laid on the optical table for easier implementation. Also note that the viewing angle of initial 3D images integrated by the lens array is about 12 degrees. After the double floating lens, it is reduced to 4 degrees along horizontal and vertical directions. We chose 4 degree viewing angle to reduce overlapping between the angular projections while keeping vertical parallax visible.
The object used in the experiment is a virtual 3D skeleton of a pyramid as shown in Fig. 7 (a), and Fig. 7 (b) shows the point cloud representation of the object.
Lager Image
Three examples of the generated elemental image sets. (a) 3D object “Pyramid”, (b) point cloud representation of the 3D object and (c) generated sets of elemental images.
Lager Image
Experimental result. Images were captured from (a) 4 degrees for vertical direction, (b) -40 degrees for horizontal direction, (c) center, (d) 45 degrees for horizontal direction and (e) -4 degrees for vertical direction.
For each object point, the elemental image points are calculated considering azimuthal angle of the rotating mirror and the double floating lens as explained in a previous section and these elemental image points for every object point are aggregated to generate final elemental images. Three examples of the generated elemental images are shown in Fig. 7 (c).
The experimental result is shown in Fig. 8 . Top and bottom images were taken at ±4 degrees and left and right images were taken at -40 and +45 degrees. From Fig. 7 , it can be confirmed that the full-parallax 3D images can be displayed with a wide horizontal viewing angle by the proposed method. Note that -40 and +45 degree horizontal viewing angle limitation comes only from the fact that our experimental setup is laid on the optical table. Figure 8 shows another experimental result
Lager Image
Experimental result when the object is rotated by 90 degrees. Images were captured from (a) bottom side, (b) center and (c) top side.
Lager Image
Experimental result when (a) the mirror is rotating and (b) stationary.
emphasizing the vertical parallax of the displayed 3D image. In this case, the object was rotated by 90 degrees to present the vertical parallax more clearly.
Although the experimental results shown in Figs. 8 and 9 successfully demonstrate full-parallax 3D image display in a wide horizontal viewing angle, they also show low image quality. One reason is magnification of each 3D pixel by the 4- f relay optics. When the initial 3D image is relayed by the 4- f optics, each 3D pixel is magnified by a factor of fFL2/fFL1 , degrading the final image quality. Low imaging performance of the Fresnel lenses used in 4- f relay optics of current implementation is another cause of non-satisfying image quality. Finally, imperfect matching between the relayed 3D images for different rotating mirror angles also accounts for blurred reconstruction. In current implementation, the viewing angle of the relayed 3D image is 4 degrees while the angular step of the rotating mirror is 1.8 degrees, creating angular overlapping of the relayed 3D images. If they are not matched exactly, the observed images can be blurred. Figure 10 shows captured images when the mirror is rotating and stationary. It can be observed that the captured image when the mirror is rotating is more blurred than when the mirror is stationary. This shows that the relayed 3D images are not matched perfectly in our current implementation, possibly due to system misalignment.
IV. CONCLUSION
In this paper, we proposed a 360 degree horizontal viewing angle full-parallax 3D display system. The proposed system combines a rotating mirror screen and integral imaging. The full-parallax but narrow viewing angle 3D images reconstructed by the integral imaging principle are projected to different directions by the rotating mirror screen, achieving 360 degree horizontal viewing angle. Unlike conventional rotating mirror based light field display systems which have only horizontal parallax, the proposed method presents full-parallax 3D images. The combination of the integral imaging and the rotating mirror configuration can also enhance the angular light ray sampling rate in the horizontal direction over a conventional rotating mirror based light field display system and integral imaging system. However, the vertical viewing angle of the proposed method is smaller than that of usual integral imaging display, constituting a drawback. Low 3D image quality of the current implementation is another point that needs to be enhanced by further research.
Acknowledgements
This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government (MEST) (No. 2012-0030815, No. 2012-0027306) and supported by the IT R&D program of MKE/KEIT. [KI001810039169, Development of Core Technologies of Holographic 3D Video System for Acquisition and Reconstruction of 3D Information]
References
Lippmann G. (1908) “La photographie integrale” C. R. Acad. Sci. 146 446 - 451
Park J.-H. , Min S.-W. , Jung S. , Lee B. (2001) “Analysis of viewing parameters for two display methods based on integral photography” Appl. Opt. 40 5217 - 52332
Park S.-G. , Song B.-S. , Min S.-W. (2010) “Analysis of image visibility in projection-type integral imaging system without diffuser” J. Opt. Soc. Korea 14 121 - 126
Saveljev V. , Kim S.-K. (2010) “Simulation of Moire effect in 3D displays” J. Opt. Soc. Korea 14 310 - 315
Kwon K.-C. , Park C. , Erdenebat M.-U. , Jeong J.-S. , Choi J.-H. , Kim N. , Park J.-H. , Lim Y.-T. , Yoo K.-H. (2012) “High speed image space parallel processing for computergenerated integral imaging system” Opt. Express 20 732 - 740
Choi H. , Min S.-W. , Jung S. , Park J.-H. , Lee B. (2003) “Multiple-viewing-zone integral imaging using a dynamic barrier array for three-dimensional displays” Opt. Express 11 927 - 932
Lee B. , Jung S. , Park J.-H. (2002) “Viewing-angle enhanced integral imaging by lens switching” Opt. Lett. 27 818 - 820
Jang J.-S. , Javidi B. (2003) “Improvement of viewing angle in integral imaging by use of moving lenslet arrays with low fill factor” Appl. Opt. 42 1996 - 2002
Jung S. , Park J.-H. , Choi H. , Lee B. (2003) “Viewingangle- enhanced integral three-dimensional imaging along all directions without mechanical movement” Opt. Express 11 1346 - 1356
Min S.-W. , Kim J. , Lee B. (2004) “Wide-viewing projectiontype integral imaging system with an embossed screen” Opt. Lett. 29 2420 - 2422
Kim Y. , Park J.-H. , Choi H. , Jung S. , Min S.-W. , Lee B. (2004) “Viewing-angle-enhanced integral imaging system using a curved lens array” Opt. Express 12 421 - 429
Kim Y. , Park J.-H. , Min S.-W. , Jung S. , Choi H. , Lee B. (2005) “A wide-viewing-angle integral 3D imaging system by curving a screen and a lens array” Appl. Opt. 44 546 - 552
Park J.-H. , Kim J. , Bae J.-P. , Kim Y. , Lee B. (2005) “Viewing angle enhancement of three-dimension/two-dimension convertible integral imaging display using double collimated or noncollimated illumination” Jpn. J. Appl. Phys. 44 L991 - L994
Kim H. , Hahn J. , Lee B. (2008) “The use of a negative index planoconcave lens array for wide-viewing angle integral imaging” Opt. Express 16 21865 - 21880
McDowall I. , Bolas M. (2005) “Fast light for display, sensing and control applications” IEEE VR 2005 – Workshop on Emerging Display Technologies 35 - 36
Jones A. , McDowall I. , Yamada H. , Bolas M. , Debevec P. (2007) “Rendering for an interactive 360° light field display” Proc. ACM SIGGRAPH 26 1 - 10
Jones A. , Lang M. , Yu X. , Busch J. , McDowall I. , Bolas M. , Debevec P. (2009) “Achieving eye contact in a one-to-many 3D video teleconferencing system” Proc. ACM SIGGRAPH 28 64 -
Miyazaki D. , Akasaka N. , Okoda K. , Maeda Y. , Mukai T. (2012) “Floating three-dimensional display viewable from 360 degrees” Proc. SPIE 8288 1H -
Erdenebat M.-U. , Baasantseren G. , Park J.-H. , Kim N. , Kwon K.-C. 2010 “Full-parallax 360 degree integral imaging display” in Proc. IMID 2010 Seoul, Korea
Baasantseren G. , Park J.-H. , Erdenebat M.-U. , Seo S.-W. , Kim N. (2010) “Integral floating-image display using two lenses with reduced distortion and enhanced depth” J. Soc. Inf. Disp. 18 519 - 526
Shin M. , Baasantseren G. , Kwon K.-C. , Kim N. , Park J.-H. (2010) “Three-dimensional display system based on integral imaging with viewing direction control” Jpn. J. Appl. Phys. 49 072501 -
Hong J. , Yeom J. , Lee B. (2012) “Enhancing angular sampling rate of integral floating display using dynamically variable apertures” Opt. Express 20 10242 - 10255