Augmented System for Immersive 3D Expansion and Interaction
Augmented System for Immersive 3D Expansion and Interaction
ETRI Journal. 2016. Mar, 38(1): 149-158
Copyright © 2016, Electronics and Telecommunications Research Institute (ETRI)
  • Received : August 20, 2015
  • Accepted : November 11, 2015
  • Published : March 01, 2016
Export by style
Cited by
About the Authors
Ungyeon Yang
Nam-Gyu Kim
Ki-Hong Kim

In the field of augmented reality technologies, commercial optical see-through-type wearable displays have difficulty providing immersive visual experiences, because users perceive different depths between virtual views on display surfaces and see-through views to the real world. Many cases of augmented reality applications have adopted eyeglasses-type displays (EGDs) for visualizing simple 2D information, or video see-through-type displays for minimizing virtual- and real-scene mismatch errors. In this paper, we introduce an innovative optical see-through-type wearable display hardware, called an EGD. In contrast to common head-mounted displays, which are intended for a wide field of view, our EGD provides more comfortable visual feedback at close range. Users of an EGD device can accurately manipulate close-range virtual objects and expand their view to distant real environments. To verify the feasibility of the EGD technology, subject-based experiments and analysis are performed. The analysis results and EGD-related application examples show that EGD is useful for visually expanding immersive 3D augmented environments consisting of multiple displays.
I. Introduction
Commercial 3D stereoscopic visualization technologies, such as 3DTV and head-mounted displays, are limited in terms of how well they can represent a natural sense of 3D space when they have a stationary screen of fixed size and limited field of view. In addition, the binocular stereoscopic principle has a fundamental convergence and accommodation conflict problem. Therefore, we suggest an augmented system, expanded 3D (E3D), which seamlessly combines multiple heterogeneous 3D displays into a single visualization space. E3D requires a specialized wearable display to visually connect between multiple 3D displays.
We develop an eyeglasses-type display (EGD) that is lightweight and that is specifically designed for close-range user interaction; for example, when a virtual object is outside of a natural 3D stereoscopic visualization zone between a stationary 3D screen and the user (for example, expanding horizontal space or field of view), or when 3D content is visualized within around 1 m from the user (such as in a close-range interaction); that is, within the reach of the user (for example, depth-directional expansion).
Historically, the development goals of a head-mounted display, which is closely related with heads-up display technology, are focused on obtaining a wide field of view and showing a high image resolution when presenting visual immersion and natural-scene perceptions to users in a virtual environment. However, these days, there is increasing demand for devices that allow a user to have direct interaction with virtual objects, such as 3D holographic user interaction in scientific-fiction movies, as opposed to passive participation in terms of immersive visualization.
In the case of producing a situation in which users interact naturally with virtual objects, because it is important to express the virtual presence of the user’s own self to feel the interplay, an indirect interaction metaphor (for example, an avatar or 3D user interface widget) and motion capture–based real-time control methods have been applied to virtual interaction environments. However, in the case of a closed view–type HMD-based virtual environment, current technology has not yet completed the user’s perception of presence at an ideal level. As an alternative to the previous problem, if we use a see-through-type HMD, then we can implement the demanded augmented reality scenario more easily because users can watch their own body and interact with virtual objects at the same time. Figure 1 shows our concept — an augmented system for a more natural 3D interaction.
PPT Slide
Lager Image
Concept of augmented system for heterogeneous displays.
II. Related Works
Current 3D display hardware that is based on the principle of binocular disparity has a fundamental weakness at visualizing a natural depth perception because of the convergence and accommodation mismatching problem. In a previous study [1] [5] , a layered multiple-display architecture that fuses the visualization areas from homogeneous and heterogeneous displays into one connected space was proposed.
Mendiburu described the development and presentation of stereoscopic 3D contents for 3DTV and 3D Cinema [6] . In addition, Shibata and others suggested a “safe zone” for 3D visualization with a “zone of comfort” regarding visual perception with various display environments [7] . By using a multiple-display platform and continuously linking multiple comfort zones for each display, our technique complements a single 3D display, which has a limited comfort zone for 3D viewing.
An augmented reality technology that enables the overlaying of computer-generated visual 2D or 3D information onto a 3D real-world view has long been developed to interact with combined virtual and real 3D objects [8] , [9] . To achieve a combined virtual and real interaction, a consumer-type augmented reality display device, which is a lightweight optical see-through head-mounted display similar to a light glasses-type wearable display, is needed [10] . However, the current version of a glasses-type wearable display has a low resolution (about 640 × 480) and a small field of view (about 15 degrees diagonally).
In augmented reality research, many perceptive and cognitive issues of the human eyes have been studied because incorrectly rendered stereoscopic images cause inconvenient visual fatigue and unstable viewing conditions [11] [13] . To date, many incorrect stereoscopic visual factors have been investigated, such as inconsistent depth budgets; fast forced convergence; potential accommodation and convergence conflicts; and color and contrast mismatching. To overcome the limitation of stereoscopic visual displays and reach the goal of our augmented system, the perception issues should focus on the near visual field because users interact well with virtual objects located within arm’s reach. At such near-field distances, most of the previous works have investigated perceptual depth matching judgments, where the depth of a tested virtual object is matched with a corresponding reference object.
However, an augmented system has to be manipulated with an appropriate measure of depth perception because it can only measure the depth perception of one object relative to that of another object. Bingham and Pagano described a depth perception measurement [14] . To increase the interaction accuracy in an augmented reality environment, many visual studies based on visual distance have been introduced. At medium-field distances of about 2 m to 10 m, depth perception has been widely studied for both virtual and augmented reality [15] , [16] , but has only recently been studied at near-field reaching distances of 20 cm to 60 cm [17] , [18] .
We exploited an experiment environment similar to that of Tresilian and Williams [19] . In this experiment environment, the participants performed both matching and reaching tasks by sliding finger-like physical objects to the proper position. The experiment was intended to elucidate the perceptive properties of various types of stereoscopic display hardware.
In Section III, the concept of our augmented system is explained using two 3D visual environment scenarios. Section IV covers, in detail, the hardware specifications of our novel EGD. In Section V, stereoscopic visual perception experiments for verifying the usability and validity of our designed EGD are described. Section VI provides some interesting 3D content platforms using the proposed EGD. Finally, some concluding remarks are given in Section VII.
III. 3D Visual Expansion
In the near future, consumers expect realistic and interactive 3D display environments, such as the one illustrated in Fig. 2 , whereby, new 3D display technologies will be developed by many display-related manufacturers. However, current advertisements for 3DTVs and 3D screens include exaggerated fictional scenes. The effects of a realistic approach or contact between users and extremely protruded 3D objects are almost impossible through current visual technologies. The more personal experiences consumers obtain regarding 3D products and services, the more they become aware that there are, in fact, many differences between those situations that are displayed in commercials and existing practical technologies. At the same time, researchers can understand the need for deeply studying human factors, such as 3D visual perception and safety, when watching 3D displays.
PPT Slide
Lager Image
Future 3D display environment. In this space, viewers can watch more realistic nature and interact with virtual objects.
According to the changes in displays owned by a single home, such as TVs, projectors, tablets, and mobile phones, and thanks to the development of wireless communication technologies, consumers are able to continuously obtain seamless content service by related heterogeneous displays. Therefore, they can overcome spatial and temporal restrictions with a single display. For example, Mills and others [20] showed a horizontal expansion of a small TV-screen to the surrounding space, and demonstrated an “ N -screen” service with multiple devices operating via cloud computing. The E3D display platform [1] has a more advanced goal than previous 2D works; it seeks to combine disconnected 3D visual spaces into one space to serve a seamless user experience.
By combining the comfort zone of various displays located in the user’s surrounding environment, the E3D platform can split and control the area of visualization space to preserve the natural 3D stereoscopic view around a user’s performance area. The extreme effect of depth perception, which is impossible with a single device, and the expression of objects located outside a single display’s viewing frustum can then be embodied.
Figure 3 shows a typical application model of the E3D platform. If two comfortable zones from a spatial display and a head-attached display are coupled, in a depth-expansion scenario, then a virtual moving 3D object can be seen and approaches the user within the safe viewing area. In addition, in a field-of-view expansion scenario, a protruding 3D object can fly in every direction within the surrounding area of the user. The E3D concept seems to be a simple idea, but is hard to realize for practical applications, because it requires an optimized solution in a three-axis relationship among the 3D display hardware, 3D rendering software, and human 3D visual factor.
PPT Slide
Lager Image
Two E3D display techniques. These expansion scenarios are able to present exaggerated 3D scenes.
We designed the wearable EGD to have optical properties, as shown in Figs. 4 and 5 , to visualize 3D objects within the user’s close-range interaction space shown in Fig. 1 . Without loss of generality, general 3D stereoscopic screens have a zone of comfort for 3D stereoscopic viewing. Therefore, we will derive the following optical design processes to obtain the maximum visualization and interaction spaces within a close range of the user’s body for a natural interaction.
According to the experimental ergonomic methods for analyzing the human characteristics of motion [21] , we can obtain the personal working space where the user’s hands can naturally reach, as shown in Fig. 4 . Therefore, using data taken from the national standard database of anthropometry from the Korea Agency for Technology and Standards, we calculated an average arm length (73 cm for ages 30 to 50), which was then designated as the standard length of a user’s arm in our study.
In addition, in accordance with the standard theory of 3D computer graphics, because a display outputs a virtual image rendered by projecting a finite-sized zone (viewing frustum), as shown in Fig. 4 , we may define the overlapping area suitable for visualizing an interaction with the target object within a close range. Based on previous studies, we heuristically set the distance of the virtual screen at 43 cm to obtain the maximum range between the positive and negative parallax within the 3D comfort zone. We can then obtain a natural visualization and close-range interaction space between around 30 cm to 60 cm from the user’s eye.
PPT Slide
Lager Image
Heuristic process used to obtain comfortable 3D visualization zone for EGD during close-range interaction.
As shown in Fig. 5 , the EGD platform is a mirror-like optical see-through system that has a simple structure comprising an image panel, a half-mirror, and a semitransparent aspherical concave half-mirror. Thus, the lower number of lenses for the optics system decreases the chromatic aberrations. We added a prism-based refraction part to obtain a higher density than air; as a result, it increased the diagonal FOV to 56° and minimized the volume of the optics. EGD has a function for adjusting the Interpupillary distance (IPD) because it has a narrow exit pupil diameter (EPD), as shown in Fig. 5 .
PPT Slide
Lager Image
Optics design metrics for EGD.
In general, when we implement a 3D stereoscopic display system, we apply a simple binocular camera model, which has a simple IPD factor for binocular disparity and locates two cameras as in a mutually parallel position. However, this method has a fundamental problem in that two virtual images are not fully superimposed when generating 3D effects. Thus, there is a typical way to compensate this problem; for the hardware, the micro–display panel is shifted inward to the center between the two eyes. At the same time, for the software, an off-axis projection model is applied for virtual cameras to make fully 100% overlapped virtual images at the position of the virtual screen. However, previous complementary methods are only suitable for visualizing 3D stereoscopic images at a far distance, such as more than 2 m away from the user. At such a distance, the convergence factor of the human visual system can be relatively ignored. However, in the case of our study, because the target space for a 3D visualization was set to within 1 m, our optics system was designed to follow the natural human factor for close viewing, and thus for both hardware and software, the convergence factor was reflected in the EGD design.
To apply the characteristic convergence of the human eye for close viewing, we rotated the yaw axis of the optics system to 4.200°, as shown in Fig. 6 . When using the toe-in style of the optical model in this manner, if we implement on-axis projection model–based virtual cameras to create virtual images, then a keystone effect is generated as a side effect because of the difference between the virtual screen and the two optical image planes. As a method to solve this side effect in hardware, we tilt the micro–display panel in a direction opposite to that of the convergence, as in Fig. 6 . In the same way, we implemented an image-rendering module as a tilt-shift lens model between the 100% stereo-overlapped virtual screen and two micro–display panels.
PPT Slide
Lager Image
Binocular configuration of EGD.
Figure 7 shows the developed EGD, which has an optical see-through system with stereo filters for external stereoscopic displays, a 9-DOF sensor for head tracking, and two USB 2.0 cameras. The OLED micro–display panel has a UWXGA resolution of 1,920 × 1,200. To support a mobile scenario, two full-HD resolution images for displays, two VGA resolution images (from the cameras), and 9-DOF sensor data are transferred using wireless communication modules, which use wireless HDMI, UWB, and Bluetooth technologies. In addition, for a natural augmented reality display, the brightness of the micro–display panel is controlled automatically by an ambient light sensor in response to the lighting condition, or manually for use in both indoor and outdoor environments.
PPT Slide
Lager Image
External appearance of EGD hardware.
V. Usability and Verification of EGD
In this section, 3D stereoscopic visual experiments were conducted to verify the usability of the developed EGD hardware. Three subject-based experiments are performed to analyze how the viewing expansion methods work on users’ visual perception. At first, our previous work for realizing the depth expansion scenario is summarized. Next, the field-of-view expansion usability analysis and user-perceived depth control methods are explained.
- 1. Depth Expansion Usability Testing
A depth expansion scenario was analyzed in our previous work [1] . EGD-wearing users are watching 3D virtual objects protruding from the front 3DTV to their EGD. In our experiments, we refer to the proper transitional position of virtual objects between an EGD and the 3DTV. Under a combined 3D stereoscopic display environment, as shown in Fig. 8 , inner 3D virtual objects and space contents should be systematically connected so as to give a continuous depth expansion to viewers. Twenty subjects participate in the test. The analysis results reveal that the subjects felt an unnaturalness of the contents when switching devices at distances of 1 m or 2 m, and even more so at a distance of 0 m.
PPT Slide
Lager Image
Experimental environment for depth expansion.
- 2. Field-of-View Expansion Usability Testing
The second experiment was conducted for the field-of-view expansion scenario, as shown on right side of Fig. 3 . A 3D virtual object is extruded from a 3DTV and is translated to another 3DTV. As shown in Fig. 9 , a virtual soccer ball moves from a 55 " 3DTV on the left to a 42 " 3DTV on the right. The invisible space between the 55 " 3DTV and the 42 " 3DTV is a marginal space. Because of this space, users cannot see momentarily the flowing virtual soccer ball. This implies that any EGD usage criteria for the invisible space should be presented. The experimental testing consists of five movement velocity levels, which are proportional to the physical distance between the two 3DTV displays.
PPT Slide
Lager Image
Experimental environment for field-of-view expansion.
Level 1 ignores the physical distance. In other words, as soon as the ball disappears on the left display, it appears on the right display with no delay. At level 2, the ball moves at the normal velocity based on the physical distance between the two displays. The velocities at levels 3, 4, and 5 are each as much as 10% slower than the previous level. The subjects watched each of the five different contents in random order. After watching each one, they filled out our questionnaire, which included three questions for each type of content asking the subjects to rate the appearance, naturalness, and predictability of the perception on a scale of zero to ten. Forty subjects participated in the experiment. In the questionnaire, one of the three questions for each type of content is regarding typical 3D image viewing; another asks about the visual naturalness according to the movement velocity; and the other is regarding whether the subjects predicted the ball’s appearance on the other display through its continuous movement.
Figure 10 shows the average and standard deviations of the naturalness and predictability of a visual scene at each velocity level. Based on a statistical analysis, an ANOVA test showed that the naturalness of each of the five contents was significantly different ( F = 5.425, p < 0.001), as was the predictability ( F = 3.641, p < 0.003). The analysis results indicate that the subjects felt a naturalness and predictability of the contents when at the same physical velocity. For the 20% slower velocity content, the subjects felt a strong unnaturalness. This analysis was able to confirm that the viewer’s perception of object movement between the two given displays depends on the velocity, and that the error range of the velocity must be 20% lower than real physical motion. As a result, if the velocity of movement is greater than 20%, the gap between the physical displays must be supplemented by another display device (for example, our EGD). In terms of aspects of predictability, the 20% slower velocity content also had an effect upon users. Through two simple user experiments, we have shown the potential for use of our EGD in an E3D display scenario. In addition, our augmented system with EGD demonstrates that 3D visual expansion schemes help users interact with visual objects.
PPT Slide
Lager Image
(a) Naturalness and (b) predictability of visual scene in expanded field-of-view scenario.
- 3. Measuring and Controlling for User Perception
To verify how differently users wearing EGD perceive 3D depth on other displays for the 3D visual expansion schemes, two experiments were conducted. These experiments allowed us to observe how the viewing expansion methods impact upon on a user’s visual perception and interaction. First, to study the potential advantages of the combined 3DTV and EGD interaction environment, we conducted a simple scenario-based qualitative pilot test in terms of visual preference and virtual immersion. Users watched a scene of an approaching butterfly on three different display environments, a 3DTV only, EGD only, and a 3DTV-to-EGD combination. A total of 29 non-experts on EGD hardware environments participated in the experiment, the median age of which is 26 years. In terms of virtual immersion, a 3DTV-to-EGD environment showed the best rating (58.6%, 17 persons), and 3DTV was the best in regard to visual preference (72.4%, 21 persons). The results indicate that the combined 3DTV-to-EGD environment is able to give users a more immersive virtual view and interactive quality.
For easy-to-use and accurate interaction in a combined 3DTV and EGD display environment, the virtual objects displayed on the 3DTV and EGD should provide the same feeling of depth, as described in Section III. Depth perception depends on the personal visual properties of the user; nevertheless, we intend to examine the display-optimized measures under the conditions of the EGD-wearing user. To carry out the experiment for the depth expansion method, as shown in Fig. 11 , we designed a small 3D visual theater, in which EGD-wearing users can locate a physical 3D positioning bar corresponding to the location of the perceived virtual object on the screen. The subjects watched a virtual ball, which was shown on both the 3DTV and EGD. They then moved the physical slider to the perceived position of the virtual ball. The virtual ball was rendered at a near-field distance of 20 cm to 90 cm.
PPT Slide
Lager Image
Experimental environment for measuring user’s depth perception.
Figure 12 shows a regression analysis ( F = 73.712, p < 0.001) between the given distance of the virtual ball and the subject’s perceived distance. The results indicate that the perceived distance on the 3DTV and EGD are different. The perceived depth of the 3DTV is relatively accurate, but the EGD shows linear differences. Based on this result, EGD is less sensitive to changes in depth (the EGD slope is smaller than that of the 3DTV). The characteristics of the EGD were used as reference control variables to perceive the same object depth between the 3DTV and EGD.
PPT Slide
Lager Image
Analysis results of depth perception on 3DTV and EGD.
In general, the hardware properties and software 3D rendering method of each 3D display incur differences in perceived 3D depth. By controlling the 3D camera rendering parameters, the visual system is able to provide a proper 3D perceived feeling of depth. Figure 13 shows three different 3D camera rendering techniques. The non-adjusting method in Fig. 13(a) only considers the IPD, and the off-axis screen and toe-in methods in Figs. 13(b) and 13(c) , respectively are able to simulate human eye movements.
PPT Slide
Lager Image
3D camera rendering methods: (a) non-adjusting, (b) off-axis screen, and (c) toe-in methods.
To investigate the proper 3D rendering method for our developed EGD, a user satisfaction test regarding the level of comfort, clarity, and feeling of depth was carried out for each rendering method studied. Figure 14 shows that the toe-in rendering method is the best for our EGD device. The results show the relevance between the rendering method and design of the binocular optics of the EGD device for near-distance visualization, as shown in Fig. 6 . Because our EGD hardware rotates two display panels in a direction opposite to that of convergence (of the human eye), the panels are able to simulate the overlapping region without implementing an off-axis screen rendering method. Therefore, for a commercial 3DTV applying an off-axis screen rendering method, our EGD exploits the toe-in rendering method.
PPT Slide
Lager Image
Satisfaction test results for different 3D rendering methods.
Most importantly, in various 3D display environments, such as our visual expansion scenarios in Fig. 3 , EGD minimizes the differences in perceived 3D depth to increase the feeling of immersion and interaction accuracy. The gap in the z -axis directional depth position, or the difference between a real object and the corresponding rendered virtual object, should be as a small as possible. To search the exact depth perception characteristics of EGD, positioning-task experiments were conducted in accordance with each rendering method. Figure 15 shows the “positioning gap” influencing the perceived difference in depth. The positioning gap indicates the difference in depth-direction between the position of the rendered virtual object and the perceived position based on the user’s positioning task. For example, in the case of the toe-in rendering method at 20 cm, the user perceives an object that is located at 4.66 cm further than 20 cm.
PPT Slide
Lager Image
Difference in perceived depth between real and virtual objects according to each rendering method (unit: cm).
Ideally, differences in perceived depth should be near zero. In addition, our EGD should provide a more exact depth perception at a distance of 20 cm to 50 cm because the EGD was designed for near-field visualization. To visualize an object at a more accurate depth position, we performed a t -test and regression analysis ( F = 17.523, p < 0.001) on the difference data for the toe-in rendering method. Through a data analysis of the toe-in rendering method, a regression line is derived as z out = 3.86 z in − 9.54. The equation of the relevant approximation line is able to infer the perceived depth error dependent on the object’s z -axis direction distance from the user. Conversely, by positioning an object at the perceived depth location, the EGD environment is able to improve the accuracy of the user interaction. Objects of interest for interaction and transition among 3D displays are controlled and visualized using the approximation model.
VI. Interactive Applications
- 1. Virtual Camera Module for EGD
To easily produce 3D content or systems using our developed EGD, our camera module is implemented for the widely used commercial 3D authoring tool, Unity3D. Figure 16 shows the rendering camera models for one EGD and one 3DTV. 3DTV faces the front (off-axis screen method), whereas EGD is viewed from slightly toward the right (toe-in method). The resultant images take a side-by-side 3D stereoscopic image format. Users are able to place multiple 3D stereo cameras in a virtual environment. The implemented camera-scripting module renders stereoscopic 3D images corresponding to each camera characteristic.
PPT Slide
Lager Image
EGD camera control scripting module and gizmos for authoring visual contents.
- 2. Immersive 3D Expansion and Interaction Systems
Figures 17 and 18 show two examples of an augmented system. The first example is a virtual golf system. The virtual golf configuration is composed of a 100 " front 3D projection screen, an IR-vision-based motion-capture system, and an EGD. The application was modified from the source codes of a commercial 3D screen golf system. In the case of a typical 3D display–based virtual golf system, if a player stands in front of the hole cup on the putting green, then the front screen must output an excessive negative parallax or the virtual image of the hole cup may be out of the view frustum between the player’s eye and the front screen. As shown on the left side of Fig. 17 , at the same time when the user’s interaction and visualization space are separated, the player is under pressure from a constrained situation. Thus, the player must see the visual feedback on the front screen and interact with the ball at the bottom. However, a defective user experience can be improved by applying an EGD for naturally showing what is in front of the user’s feet.
PPT Slide
Lager Image
Virtual golf system: left side shows typical virtual putting interaction, which separates view direction and working space. On right side, because user wears an EGD, more natural interaction and expanded views are provided.
Figure 18 shows a field of view expansion–based system setup for experiencing close-range interaction with the EGD applied over the effects of E3D display technologies. A 150 " 3D projector screen and an 84 " 3DTV show a flower garden in a demonstration room, where a subject wearing the EGD can watch butterflies seamlessly moving within several comfort zones of different 3D stereoscopic displays. As shown in Figs. 18(b) and 18(c) , a virtual camera rig was used to display the third-person point of view to outside observers. The user can watch the butterflies protrude from the outer 3D projector screen and move seamlessly into the EGD, or fly from the EGD to another outer screen. As shown in Fig. 18(d) , the subject can control the moving path of a group of butterflies using hand gestures.
PPT Slide
Lager Image
Expanded 3D interaction system with EGD.
VII. Conclusion and Future Work
In this study, we developed a novel wearable display, an EGD, which has a new optics system that can visualize direct interaction with 3D objects within close range of the user’s body. In addition, we presented pilot tests to verify the feasibility of the EGD and stereoscopic 3D augmented display platform. We have a goal to use these technologies in various close-range interactive 3D environments, such as in an information appliance environment or a digital theme park. Therefore, given a large number of user groups who have various ergonomic parameters, we need to find a way to take advantage of the stable multiple 3D display platform and EGD. Thus, we carried out a number of subject-based experiments to optimize the parameters for 3D visualization.
For future studies, the EGD may more conveniently support an optical see-through function to fuse multiple visual images from heterogeneous 3D displays without a delay. In addition, to apply the acquired knowledge from user studies to various practical situations, a pre-visualization tool needs to be developed to show the optimal parameter setup for diverse configurations of heterogeneous displays of the target systems.
Subject-based experiments require a larger experimental population to obtain more robust analysis results. However, in a practical situation, it is very hard to collect an evenly distributed subject group with various visual characteristics. Therefore, as in the case of Fig. 19 , we designed a new stereo camera system to emulate the human visual system, which will be able to substitute a subject as a standard, allowing more quantitative measurements to be taken.
PPT Slide
Lager Image
Stereo camera system for quantitative measurements.
This work was supported by the R&D program of Ministry of Science, ICT & Future Planning (MSIP) and Institute for Information & communications Technology Promotion (IITP) (10039923, Development of Live 4D contents platform technology based on expansion of realistic experiential space).
Corresponding Author
Ungyeon Yang received his BS degree in computer science and engineering from Chungnam National University, Daejeon, Rep. of Korea, in 1997. He received his MS and PhD degrees from Pohang University of Science and Tennology, Rep. of Korea, in 2000 and 2003, respectively. Since 2003, he has been a principal researcher with ETRI. His research interests include information visualization, 3D user interfaces, human factors, and multimodal user interaction in the field of virtual/mixed reality and ergonomics.
Nam-Gyu Kim received his BS degree in computer science from the Korea Advanced Institute of Science and Technology, Daejeon, Rep. of Korea, in 1995. He received his MS and PhD degrees in computer science and engineering from Pohang University of Science and Technology, Rep. of Korea, in 1996 and 2005, respectively. He joined the Advanced Telecommunications Research Institute International, Kyoto, Japan, in 2000 and then went on to work for Korea Telecommunication Research Center, Daejeon, Rep. of Korea, in 2006. Since 2009, he has been an associate professor with the Department of Game Engineering, Dong-Eui University, Busan, Rep. of Korea. His research interests include 3D human–computer interaction in games and multimedia systems; computer vision; and visual information processing in the field of virtual and augmented reality.
Ki-Hong Kim received his BS and MS degrees in electrical engineering from Kyungpook National University, Daegu, Rep. of Korea, in 1994 and 1996, respectively. In 2007, he receive his PhD degree in electrical engineering from the Korea Advanced Institute of Science and Technology, Daejeon, Rep. of Korea. Since 1996, he has been with ETRI, where he is working as a principal researcher. His main research interests include biosignal processing, speech signal processing, 3D sound, human–computer interaction, and virtual reality
Yang U.Y. 2013 “Mixed Display Platform to Expand Comfortable Zone of Stereoscopic 3D Viewing,” ETRI J. 35 (2) 352 - 355    DOI : 10.4218/etrij.13.0212.0245
Yang U.Y. “Seamlessly Expanded Natural Viewing Area of Stereoscopic 3D Display System,” IEEE Int. Conf. Consum. Electron. Las Vegas, NV, USA Jan. 11–14, 2013 167 - 168
Yang U.Y. 2012 Expandable 3D Stereoscopic Display System Korean Intellectual Property Office application no. 2012-0018690
Kim H.M. 2012 “Dual Autostereoscopic Display Platform for Multi-user Collaboration with Natural Interaction,” ETRI J. 34 (3) 466 - 469    DOI : 10.4218/etrij.12.0211.0331
Lee G.A. , Yang U.Y. , Son W.H. “Layered Multiple Displays for Immersive and Interactive Digital Contents,” Int. Conf. Entertainment Comput., LNCS Cambridge, UK Sept. 20–22, 2006 4161 123 - 134
Mendiburu B. , Pupulin Y. , Schklair S. 2012 “3D TV and 3D Cinema: Tools and Process for Creative Stereoscopy,” Focal Press, Elsevier Inc. Waltham, MA, USA 129 - 194
Shibata T. 2012 “The Zone of Comfort: Predicting Visual Discomfort with Stereo Displays,” J. Vis. 11 (8) 1 - 29
Azuma R. 2001 “Recent Advances in Augmented Reality,” IEEE Comput. Graph. Appl. 21 (6) 34 - 47
Zhou F. , Duh H. , Billinghurst M. “Trends in Augmented Reality Tracking, Interaction and Display: A Review of Ten Years of ISMAR,” Proc. IEEE/ACM Int. Symp. Mixed Augmented Reality Cambridge, UK Sept. 15–18, 2008 193 - 202
2013 Google Glass Wikimedia Foundation, Inc.
Yano S. 2002 “A Study of Visual Fatigue and Visual Comfort for 3D HDTV/HDTV Images,” Displays 23 (4) 191 - 201    DOI : 10.1016/S0141-9382(02)00038-0
Watt S.J. 2005 “Focus Cues Affect Perceived Depth,” J. Vis. 5 (7) 834 - 862    DOI : 10.1167/5.8.834
Hoffman D.M. 2008 “Vergence-Accommodation Conflicts Hinder Visual Performance and Cause Visual Fatigue,” J. Vis. 8 (3) 1 - 33
Bingham G.P. , Pagano C.C. 1998 “The Necessity of a Perception–Action Approach to Definite Distance Perception: Monocular Distance Perception to Guide Reaching,” J. Experimental Psychology: Human Perception Performance 24 (1) 145 - 168    DOI : 10.1037/0096-1523.24.1.145
Jones J.A. “Peripheral Visual Information and its Effect on Distance Judgments in Virtual and Augmented Environments,” Proc. ACM SIGGRAPH Symp. Appl. Perception Graph. Vis. Toulouse, France Aug. 27–28, 2011 29 - 36
Williams M.M. , Tresilian J.R. 2000 “Ordinal Depth Information from Accommodation,” Ergonomics 43 (3) 391 - 404    DOI : 10.1080/001401300184486
Singh G. “Depth Judgment Measures and Occluding Surfaces in Near-Field Augmented Reality,” Proc. Symp. Appl. Perception Graph. Vis. Los Angeles, CA, USA July 23–24, 2010 149 - 156
Singh G. “Depth Judgment Tasks and Environments in Near-Field Augmented Reality,” IEEE Virtual Reality Conf. Singapore Mar. 19–23, 2011 241 - 242
Tresilian J. , Williams M. 1999 “A Curious Illusion Suggests Complex Cue Interactions in Distance Perception,” J. Experimental Psychology: Human Perception Performance 25 (3) 677 - 687    DOI : 10.1037/0096-1523.25.3.677
Mills P. 2011 BBC Research & Development, Surround Video, White Paper 208
Pheasant S. , Haslegrave C.M. 2006 “Bodyspace: Anthropometry, Ergonom., and the Design of Work,” 3rd ed. CRC Press, Taylor & Francis Group Abingdon, UK