Advanced
Design and Implementation of Digital Hologram Content Using Modified Depth Information
Design and Implementation of Digital Hologram Content Using Modified Depth Information
Journal of Information and Communication Convergence Engineering. 2014. Jun, 12(2): 122-127
Copyright © 2014, The Korea Institute of Information and Commucation Engineering
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/li-censes/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • Received : February 28, 2014
  • Accepted : May 19, 2014
  • Published : June 30, 2014
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Scott Park
DDR IP Division, Uniquify Inc., San Jose, CA 95131, USA
Hyun-Jun Choi
Department of Electronic Engineering, Mokpo National Maritime University, Mokpo 530-729, Korea
Moon-Seok Kim
Department of Cultural Industry in Northeast Asia, Kwangwoon University, Seoul 139-701, Korea
Dong-Wook Kim
Department of Electronic Material Engineering, Kwangwoon University, Seoul 139-701, Korea
Young-Ho Seo
College of Liberal Arts, Kwangwoon University, Seoul 139-701, Korea
yhseo@kw.ac.kr

Abstract
This paper proposes a method to manipulate digital hologram contents by manipulating and/or synthesizing the depth information. To synthesize digital holograms themselves in order to create new digital hologram contents. This paper uses both the depth information obtained by converting the disparity information by using a stereo matching method and that obtained by taking pictures with a depth camera. In addition, assuming that digital holograms are created using the computer-generated holography method, we propose a technique for authoring and compositing hologram contents by using either the changes in the three-dimensional positions of objects in the hologram or by combining the objects with other contents by means of changes in the depth information. Further, more than one digital hologram was synthesized to form a hologram. The reconstructed result from the synthesized hologram also contained all the objects in each digital hologram before synthesis at the same positions and distances.
Keywords
I. INTRODUCTION
Recently, with the increasing interest in three-dimensional (3D) images, there has been a demand for solving the problems of discomfort, dizziness, etc., related to a stereo or multi-view 3D image. Therefore, research on holograms, which can realize perfect 3D imaging, has become more active. However, the high cost of generating holograms is a challenge [1] .
In order to make the contents of digital holograms continuous and widespread, rather than using them as entertainment for a one-off event, it is necessary to diversify these contents. Therefore, we need to author hologram contents efficiently in order to reduce the cost and effort of content development as well as the authoring time, and to create hologram convergence contents that are applicable to various industrial fields [2] . Accordingly, a study on how to author hologram contents, such as producing hologram videos using one sheet of holograms, has been conducted previously. Currently, the development of a 3D broadcasting technology using digital holography is underway at MIT in the USA, at NHK and ATR in Japan, and at HHI in Germany.
In this paper, we discuss a measure to secure digital hologram contents. In addition, assuming that digital holograms are created using the computer-generated hologram (CGH) method, we propose a technique for authoring and compositing hologram contents by using either the changes in the 3D positions of the objects in the hologram or by combining the objects with other contents by using the changes in the depth information.
The rest of this paper is organized as follows: Section II introduces the CGH method, and Section III explains the proposed algorithm. The experimental results of the proposed algorithm are described in Section IV, and Section V presents the conclusion.
II. COMPUTER-GENERATED HOLOGRAM
A system for digital holograms uses electronic equipment such as a charge-coupled device (CCD) camera instead of an optical one to record the interference pattern of holography and transmit it as a video signal. The image is reconstructed at the receiver side by illuminating a laser beam to the received interference pattern uploaded on a spatial light modulator.
This section describes the CGH calculation method [2 - 5] and the method using a recursive addition system. The CGH generating equation is defined as Eq. (1):
PPT Slide
Lager Image
where α or j indicates a particular point on the hologram or the 3D object, respectively, k denotes the wave number of the reference wave defined as 2π/λ, p represents the pixel pitch of the hologram, and ( xα , yα ) and ( xj , yj , zj ) represent the coordinates of the hologram and the 3D object, respectively.
Fig. 1 shows a sample coordinate array system for the 3D object and the digital hologram for the application of the CGH method, where the 3D object (light source) consists of 2 × 2 [pixel 2 ], and the digital hologram is captured as an 4 × 4 [pixel 2 ] image. To generate a digital hologram with this setup, the calculation of Eq. (1) must be carried out 256 (2 × 2 × 8 × 8) times.
PPT Slide
Lager Image
Computer-generated hologram operation of the conventional method.
III. PROPOSED ALGORITHM
In this section, a method for creating new digital hologram contents by a manipulation of the depth information acquired using a matching method and time-of-flight (TOF) cameras, is introduced.
- A. Manipulation of Depth Information Acquired Using a Stereo Matching Method
Fig. 2 shows a schematic view of the vertical and horizontal movements of objects while acquiring disparity information by using matching methods. First, for the vertical movement shown in Fig. 2(a) , d1 and d2 , which are the disparity values at positions A1B1 and A2B2 , can be expressed by Eq. (2):
PPT Slide
Lager Image
The two equations can be combined to form Eq. (3):
PPT Slide
Lager Image
This implies that when objects are moved from position A1B1 to position A2B2 or vice versa, a disparity value should be adjusted to satisfy Eq. (3).
In addition, the size of an object projected by two cameras can be written as S1 = d1 in the A1B1 position and S2 = d2 in the A2B2 position. Therefore, the size of an object according to depth shall follow Eq. (4), which is similar to Eq. (3):
PPT Slide
Lager Image
Thus, when objects move vertically, the sizes of the moved objects and their disparity values are inversely proportional to distance and can be computed using Eqs. (3) and (4). In the case of the horizontal movement, as shown in Fig. 2(b) , the disparity and the sizes before and after the movement remain the same. Thus, the sizes and the disparity of objects do not change when the objects move horizontally.
PPT Slide
Lager Image
Changes in disparity information and size by object movement for stereo matching method: (a) vertical movement and (b) horizontal movement.
- B. Manipulation of Depth Information Acquired with TOF Cameras
Fig. 3 shows the changes in the sizes of the depth information acquired with TOF cameras. It can be seen that the same object moves vertically. The thickness of the object does not change before and after the movement; therefore, this can be defined as Eqs. (5)–(7), as given below:
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
The depth information of the TOF method for the vertical movement of objects can be easily obtained using Eq. (7).
In addition, the sizes of the object on the screen, SA and SB , before and after the movement can be defined by Eq. (8).
PPT Slide
Lager Image
where h denotes the actual size of the object. Thus, the relationship between the sizes displayed on the screen before and after the movement can be expressed by Eq. (9).
PPT Slide
Lager Image
PPT Slide
Lager Image
Depth and object size obtained by time-of-flight depth camera.
In conventional 3D videos, a viewer’s attention is restricted to only a small area. In fact, it was reported that viewer can recognize 3D effect only in the limited area that they focus on directly, while the surrounding areas are actually seen as 2D effect. This is also true of holograms.
As mentioned above, considerable computation is required to create holograms, and with region of interest (ROI) processing, the computation time can be reduced.
Fig. 4 shows a hologram image in which the depth information of the areas of interest and non-interest are the same, and a hologram image in which the resolution of the depth information in the non-interest area is reduced by 1/4.
Fig. 4(a) and (e) show the original image and the depth image in which the resolution of the non-interest area is reduced by 1/4, while Fig. 4(b) and (f) show the results when CGHs of Fig. 4(a) and (e) are created. The images in Fig. 4(c) and (g) are restored by focusing on the area of interest in Fig. 4(b) and (f) , respectively. By comparing Fig. 4(c) and (g) , we find that while the area of interest remains the same between the two images, a considerable amount of blurring is observed in the non-interest area, in the images with reduced resolution.
This is more evident in the restored images shown in Fig. 4(d) and (h) , where the focus is on the non-interest area. However, a sharpness difference in the non-interest area has no significant effect when viewers pay attention only to the area of interest. On the other hand, the computational cost increases in proportion to the depth image resolution. Therefore, the size and the resolution of the non-interest area affect the computational cost of the CGH directly.
PPT Slide
Lager Image
Example of region of interest (ROI) process: (a–d) the case of not reducing the depth resolution of non-ROI, (e–h) the case of reducing the depth resolution of non-ROI into 1/4. (a, d) Depth maps, (b, f) generated computer-generated holograms, (c, g) reconstructed images from (b) and (f) by focusing ROI, (d, h) reconstructed images from (b) and (f) by focusing non-ROI.
IV. EXPERIMENTAL RESULTS
- A. Experiment Environment
As mentioned before, in this study, we use both the disparity value and the converted depth value via the matching method to author the digital hologram contents. To restore an original image from the digital hologram, a simulation technique was employed. Experimental images provided in the Middlebury site were used for the depth information of the matching method. Note that these images were acquired using Canon G1. Two test images of a baby and flower pots are shown in Fig. 5 . Here, Fig. 5(a) and (e) are left images, while Fig. 5(b) and (f) are right images.
PPT Slide
Lager Image
Examples of generated computer-generated holograms with disparity maps: (a–d) baby stereo images, (e–h) Flowerpots stereo images. (a, e) Left images, (b, f) right images, (c, g) disparity maps, (d, h) reconstructed image of the generated digital hologram for the extracted objects.
Fig. 5(c) and (g) display the disparity information provided in the Middlebury site. In addition, Fig. 5(d) and (h) are images restored by the simulation method, from the digital holograms created using Eq. (1), as a specific object is extracted from the image. In order to acquire the depth information directly by using a depth camera of the TOF method, a SR4000 depth camera of the MESA Imaging Company (Zurich, Switzerland) was used to take a picture. Fig. 6 shows an example of how to create holograms with the depth camera by using the depth information shot and then restore them by using the simulation method.
PPT Slide
Lager Image
Computer-generated hologram generation and reconstruction with depth map by a depth camera: (a) depth map and (b) reconstructed image.
- B. Authoring Results
- 1) Manipulation of Depth Information by Using the Stereo Matching Method
Sections of the object were extracted from the depth information converted from the disparity information, acquired using the matching method. The result was then manipulated to have the position to move, thereby creating digital holograms and restoring the results. As shown in the example in Fig. 7 , the distance movement was shown since movement without distance change is a simple coordinate transformation in the position movement. The distance in Fig. 7(a) from the object center is about 80 cm, while in Fig. 7(d) , this changes to 100 cm. Fig. 7(b) and (c) show the images restored from the holograms created from Fig. 7(a) , while Fig. 7(e) and (f) show the images restored from the holograms created from Fig. 7(d) . Among them, Fig. 7(b) and (e) have their restoration distances set to the object center in Fig. 7(a) , while Fig. 7(c) and (f) have their restoration distances set to the object center in Fig. 7(b) . That is, they refer to a distance from the restored images to the corresponding object.
- 2) Manipulation of Depth Information Acquired with Depth Camera
The depth images shot with the depth camera of the TOF method are also manipulated using the same procedure as the depth information obtained by the matching method. Fig. 8 shows an example of the manipulation of the depth information acquired with the TOF depth cameras. The procedure employed is the same as that shown in Fig. 7 .
PPT Slide
Lager Image
Depth map manipulation example for stereo matching method: (a, b) depth map and (c, d) reconstructed image.
PPT Slide
Lager Image
Depth map manipulation for time-of-flight camera method: (a) depth map and (b) reconstructed image.
V. CONCLUSIONS
In this paper, in order to create new digital hologram contents, we utilized depth information acquired using a depth camera of the TOF method and the stereo matching method. Furthermore, we combined the contents by analyzing the characteristics of the creation of depth information of such contents and created depth information by changing the object positions. In addition, the linearity of the hologram of each individual object in the hologram area was verified with experiments and each hologram of an object was created using the verified linearity characteristics, thereby compositing newly acquired holograms with the existing holograms to create new contents.
Acknowledgements
This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2012S1A5A2A03034349).
BIO
Scott Park received his M.S. degree in 2004 from the Department of Electronic Materials Engineering, Kwangwoon University, Seoul, Korea. He is currently a senior manager of DDR IP Division, Uniquify Inc., San Jose, CA, USA. His current research interests include digital system design, digital testability and design-for-test, digital embedded systems for wired and wireless communication, and design of digital signal processors.
Hyun-Jun Choi received his M.S. and Ph.D. degrees in 2005 and 2009, respectively, from the Department of Electronic Materials Engineering, Kwangwoon University, Seoul, Korea. He was a research professor at the Realistic Media Institute at Kwangwoon University. He was an assistant professor in the Department of Information and Communication Engineering at Anyang University in Anyang, Korea, from 2010 to 2011. He is currently an assistant professor with the Department of Electronic Engineering, Mokpo National Maritime University, Mokpo, Korea. His research interests include optical image processing and 3D displays
Moon-Seok Kim received his M.F.A. and Ph.D. degrees in 1998 and 2004, respectively, from Department of Applied Art, Hanyang University, Seoul, Korea. He was an assistant professor at Department of Advertising Design, Seoul University, Seoul, Korea. He is now an associated professor at Department of Cultural Industry in Northeast Asia, Kwangwoon University, Seoul, Korea, and Editor-in-Chief of Articles in Korea Digital Design Council (KODDCO). His research interests include graphic design, media design, and media contents design.
Dong-Wook Kim received his B.S. and M.S. degrees from the Department of Electronic Engineering, Hangyang University, Seoul, Korea, in 1983 and 1985, respectively, and his Ph.D. degree from the Department of Electrical Engineering, Georgia Institute of Technology, Atlanta, in 1991. He is currently Professor and Dean of Academic Affairs at Kwangwoon University, Seoul. His current research interests include digital system design, digital testability and design-for-test, digital embedded systems for wired and wireless communication, and design of digital signal processors.
Young-Ho Seo Dr. Seo received his M.S and Ph.D. degrees in 2000 and 2004, respectively, from Department of Electronic Materials Engineering, Kwangwoon University, Seoul, Korea. He was a researcher at Korea Electrotechnology Research Institute (KERI) from 2003 to 2004. He was a research professor in Department of Electronic and Information Engineering, Yuhan College, Buchon, Korea, in 2005. He was a professor at Department of Information and Communication Engineering, Hansung University, Seoul, Korea, from 2006 to 2007. He is now an assistant professor at the College of Liberal Arts, Kwangwoon University, Seoul, Korea, and a director of the research institute in Ten Technology Inc. His research interests include 2D/3D digital image processing and contents security.
References
Javidi B. , Okano F. 2002 Three-Dimensional Television, Video, and Display Technologies Springer Heidelberg
Choi H. J. , Seo Y. H. , Jang S. W. , Kim D. W. 2012 “Analysis of digital hologram rendering using computational method,” Journal of Information and Communication Convergence Engineering 10 (2) 205 - 209    DOI : 10.6109/jicce.2012.10.2.205
Seo Y. H. , Lee Y. H. , Yoo J. S. , Kim D. W. 2012 “Hardware architecture of high-performance digital hologram generator on the basis of a pixel-by-pixel calculation scheme,” Applied Optics 51 (18) 4003 - 4012    DOI : 10.1364/AO.51.004003
Yoshikawa H. 2001 “Fast computation of Fresnel holograms employing difference,” Optical Review 8 (5) 331 - 335    DOI : 10.1007/s10043-001-0331-y
Shimobaba T. , Ito T. , Masuda N. , Ichihashi Y. , Takada N. 2010 “Fast calculation of computer-generated-hologram on AMD HD5000 series GPU and OpenCL,” Optics Express 18 (10) 9955 - 9960    DOI : 10.1364/OE.18.009955