This paper proposes a method to manipulate digital hologram contents by manipulating and/or synthesizing the depth information. To synthesize digital holograms themselves in order to create new digital hologram contents. This paper uses both the depth information obtained by converting the disparity information by using a stereo matching method and that obtained by taking pictures with a depth camera. In addition, assuming that digital holograms are created using the computer-generated holography method, we propose a technique for authoring and compositing hologram contents by using either the changes in the three-dimensional positions of objects in the hologram or by combining the objects with other contents by means of changes in the depth information. Further, more than one digital hologram was synthesized to form a hologram. The reconstructed result from the synthesized hologram also contained all the objects in each digital hologram before synthesis at the same positions and distances.
Recently, with the increasing interest in three-dimensional (3D) images, there has been a demand for solving the problems of discomfort, dizziness, etc., related to a stereo or multi-view 3D image. Therefore, research on holograms, which can realize perfect 3D imaging, has become more active. However, the high cost of generating holograms is a challenge
In order to make the contents of digital holograms continuous and widespread, rather than using them as entertainment for a one-off event, it is necessary to diversify these contents. Therefore, we need to author hologram contents efficiently in order to reduce the cost and effort of content development as well as the authoring time, and to create hologram convergence contents that are applicable to various industrial fields
. Accordingly, a study on how to author hologram contents, such as producing hologram videos using one sheet of holograms, has been conducted previously. Currently, the development of a 3D broadcasting technology using digital holography is underway at MIT in the USA, at NHK and ATR in Japan, and at HHI in Germany.
In this paper, we discuss a measure to secure digital hologram contents. In addition, assuming that digital holograms are created using the computer-generated hologram (CGH) method, we propose a technique for authoring and compositing hologram contents by using either the changes in the 3D positions of the objects in the hologram or by combining the objects with other contents by using the changes in the depth information.
The rest of this paper is organized as follows: Section II introduces the CGH method, and Section III explains the proposed algorithm. The experimental results of the proposed algorithm are described in Section IV, and Section V presents the conclusion.
II. COMPUTER-GENERATED HOLOGRAM
A system for digital holograms uses electronic equipment such as a charge-coupled device (CCD) camera instead of an optical one to record the interference pattern of holography and transmit it as a video signal. The image is reconstructed at the receiver side by illuminating a laser beam to the received interference pattern uploaded on a spatial light modulator.
This section describes the CGH calculation method
and the method using a recursive addition system. The CGH generating equation is defined as Eq. (1):
indicates a particular point on the hologram or the 3D object, respectively,
denotes the wave number of the reference wave defined as 2π/λ,
represents the pixel pitch of the hologram, and (
) and (
) represent the coordinates of the hologram and the 3D object, respectively.
shows a sample coordinate array system for the 3D object and the digital hologram for the application of the CGH method, where the 3D object (light source) consists of 2 × 2 [pixel
], and the digital hologram is captured as an 4 × 4 [pixel
] image. To generate a digital hologram with this setup, the calculation of Eq. (1) must be carried out 256 (2 × 2 × 8 × 8) times.
Computer-generated hologram operation of the conventional method.
III. PROPOSED ALGORITHM
In this section, a method for creating new digital hologram contents by a manipulation of the depth information acquired using a matching method and time-of-flight (TOF) cameras, is introduced.
- A. Manipulation of Depth Information Acquired Using a Stereo Matching Method
shows a schematic view of the vertical and horizontal movements of objects while acquiring disparity information by using matching methods. First, for the vertical movement shown in
, which are the disparity values at positions
, can be expressed by Eq. (2):
The two equations can be combined to form Eq. (3):
This implies that when objects are moved from position
or vice versa, a disparity value should be adjusted to satisfy Eq. (3).
In addition, the size of an object projected by two cameras can be written as
position. Therefore, the size of an object according to depth shall follow Eq. (4), which is similar to Eq. (3):
Thus, when objects move vertically, the sizes of the moved objects and their disparity values are inversely proportional to distance and can be computed using Eqs. (3) and (4). In the case of the horizontal movement, as shown in
, the disparity and the sizes before and after the movement remain the same. Thus, the sizes and the disparity of objects do not change when the objects move horizontally.
Changes in disparity information and size by object movement for stereo matching method: (a) vertical movement and (b) horizontal movement.
- B. Manipulation of Depth Information Acquired with TOF Cameras
shows the changes in the sizes of the depth information acquired with TOF cameras. It can be seen that the same object moves vertically. The thickness of the object does not change before and after the movement; therefore, this can be defined as Eqs. (5)–(7), as given below:
The depth information of the TOF method for the vertical movement of objects can be easily obtained using Eq. (7).
In addition, the sizes of the object on the screen,
, before and after the movement can be defined by Eq. (8).
denotes the actual size of the object. Thus, the relationship between the sizes displayed on the screen before and after the movement can be expressed by Eq. (9).
Depth and object size obtained by time-of-flight depth camera.
In conventional 3D videos, a viewer’s attention is restricted to only a small area. In fact, it was reported that viewer can recognize 3D effect only in the limited area that they focus on directly, while the surrounding areas are actually seen as 2D effect. This is also true of holograms.
As mentioned above, considerable computation is required to create holograms, and with region of interest (ROI) processing, the computation time can be reduced.
shows a hologram image in which the depth information of the areas of interest and non-interest are the same, and a hologram image in which the resolution of the depth information in the non-interest area is reduced by 1/4.
show the original image and the depth image in which the resolution of the non-interest area is reduced by 1/4, while
show the results when CGHs of
are created. The images in
are restored by focusing on the area of interest in
, respectively. By comparing
, we find that while the area of interest remains the same between the two images, a considerable amount of blurring is observed in the non-interest area, in the images with reduced resolution.
This is more evident in the restored images shown in
Fig. 4(d) and (h)
, where the focus is on the non-interest area. However, a sharpness difference in the non-interest area has no significant effect when viewers pay attention only to the area of interest. On the other hand, the computational cost increases in proportion to the depth image resolution. Therefore, the size and the resolution of the non-interest area affect the computational cost of the CGH directly.
Example of region of interest (ROI) process: (a–d) the case of not reducing the depth resolution of non-ROI, (e–h) the case of reducing the depth resolution of non-ROI into 1/4. (a, d) Depth maps, (b, f) generated computer-generated holograms, (c, g) reconstructed images from (b) and (f) by focusing ROI, (d, h) reconstructed images from (b) and (f) by focusing non-ROI.
IV. EXPERIMENTAL RESULTS
- A. Experiment Environment
As mentioned before, in this study, we use both the disparity value and the converted depth value via the matching method to author the digital hologram contents. To restore an original image from the digital hologram, a simulation technique was employed. Experimental images provided in the Middlebury site were used for the depth information of the matching method. Note that these images were acquired using Canon G1. Two test images of a baby and flower pots are shown in
are left images, while
are right images.
Examples of generated computer-generated holograms with disparity maps: (a–d) baby stereo images, (e–h) Flowerpots stereo images. (a, e) Left images, (b, f) right images, (c, g) disparity maps, (d, h) reconstructed image of the generated digital hologram for the extracted objects.
display the disparity information provided in the Middlebury site. In addition,
are images restored by the simulation method, from the digital holograms created using Eq. (1), as a specific object is extracted from the image. In order to acquire the depth information directly by using a depth camera of the TOF method, a SR4000 depth camera of the MESA Imaging Company (Zurich, Switzerland) was used to take a picture.
shows an example of how to create holograms with the depth camera by using the depth information shot and then restore them by using the simulation method.
Computer-generated hologram generation and reconstruction with depth map by a depth camera: (a) depth map and (b) reconstructed image.
- B. Authoring Results
- 1) Manipulation of Depth Information by Using the Stereo Matching Method
Sections of the object were extracted from the depth information converted from the disparity information, acquired using the matching method. The result was then manipulated to have the position to move, thereby creating digital holograms and restoring the results. As shown in the example in
, the distance movement was shown since movement without distance change is a simple coordinate transformation in the position movement. The distance in
from the object center is about 80 cm, while in
, this changes to 100 cm.
show the images restored from the holograms created from
show the images restored from the holograms created from
. Among them,
have their restoration distances set to the object center in
have their restoration distances set to the object center in
. That is, they refer to a distance from the restored images to the corresponding object.
- 2) Manipulation of Depth Information Acquired with Depth Camera
The depth images shot with the depth camera of the TOF method are also manipulated using the same procedure as the depth information obtained by the matching method.
shows an example of the manipulation of the depth information acquired with the TOF depth cameras. The procedure employed is the same as that shown in
Depth map manipulation example for stereo matching method: (a, b) depth map and (c, d) reconstructed image.
Depth map manipulation for time-of-flight camera method: (a) depth map and (b) reconstructed image.
In this paper, in order to create new digital hologram contents, we utilized depth information acquired using a depth camera of the TOF method and the stereo matching method. Furthermore, we combined the contents by analyzing the characteristics of the creation of depth information of such contents and created depth information by changing the object positions. In addition, the linearity of the hologram of each individual object in the hologram area was verified with experiments and each hologram of an object was created using the verified linearity characteristics, thereby compositing newly acquired holograms with the existing holograms to create new contents.
This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2012S1A5A2A03034349).
Scott Park received his M.S. degree in 2004 from the Department of Electronic Materials Engineering, Kwangwoon University, Seoul, Korea. He is currently a senior manager of DDR IP Division, Uniquify Inc., San Jose, CA, USA. His current research interests include digital system design, digital testability and design-for-test, digital embedded systems for wired and wireless communication, and design of digital signal processors.
Hyun-Jun Choi received his M.S. and Ph.D. degrees in 2005 and 2009, respectively, from the Department of Electronic Materials Engineering, Kwangwoon University, Seoul, Korea. He was a research professor at the Realistic Media Institute at Kwangwoon University. He was an assistant professor in the Department of Information and Communication Engineering at Anyang University in Anyang, Korea, from 2010 to 2011. He is currently an assistant professor with the Department of Electronic Engineering, Mokpo National Maritime University, Mokpo, Korea. His research interests include optical image processing and 3D displays
Moon-Seok Kim received his M.F.A. and Ph.D. degrees in 1998 and 2004, respectively, from Department of Applied Art, Hanyang University, Seoul, Korea. He was an assistant professor at Department of Advertising Design, Seoul University, Seoul, Korea. He is now an associated professor at Department of Cultural Industry in Northeast Asia, Kwangwoon University, Seoul, Korea, and Editor-in-Chief of Articles in Korea Digital Design Council (KODDCO). His research interests include graphic design, media design, and media contents design.
Dong-Wook Kim received his B.S. and M.S. degrees from the Department of Electronic Engineering, Hangyang University, Seoul, Korea, in 1983 and 1985, respectively, and his Ph.D. degree from the Department of Electrical Engineering, Georgia Institute of Technology, Atlanta, in 1991. He is currently Professor and Dean of Academic Affairs at Kwangwoon University, Seoul. His current research interests include digital system design, digital testability and design-for-test, digital embedded systems for wired and wireless communication, and design of digital signal processors.
Young-Ho Seo Dr. Seo received his M.S and Ph.D. degrees in 2000 and 2004, respectively, from Department of Electronic Materials Engineering, Kwangwoon University, Seoul, Korea. He was a researcher at Korea Electrotechnology Research Institute (KERI) from 2003 to 2004. He was a research professor in Department of Electronic and Information Engineering, Yuhan College, Buchon, Korea, in 2005. He was a professor at Department of Information and Communication Engineering, Hansung University, Seoul, Korea, from 2006 to 2007. He is now an assistant professor at the College of Liberal Arts, Kwangwoon University, Seoul, Korea, and a director of the research institute in Ten Technology Inc. His research interests include 2D/3D digital image processing and contents security.
Three-Dimensional Television, Video, and Display Technologies
Choi H. J.
Seo Y. H.
Jang S. W.
Kim D. W.
“Analysis of digital hologram rendering using computational method,”
Journal of Information and Communication Convergence Engineering
DOI : 10.6109/jicce.2012.10.2.205
Seo Y. H.
Lee Y. H.
Yoo J. S.
Kim D. W.
“Hardware architecture of high-performance digital hologram generator on the basis of a pixel-by-pixel calculation scheme,”
DOI : 10.1364/AO.51.004003
“Fast calculation of computer-generated-hologram on AMD HD5000 series GPU and OpenCL,”
DOI : 10.1364/OE.18.009955