Advanced
New Implementation and Test Methodology for Single Lens Stereoscopic 3D Camera System
New Implementation and Test Methodology for Single Lens Stereoscopic 3D Camera System
Journal of Broadcast Engineering. 2014. Sep, 19(5): 569-577
Copyright © 2014, The Korean Society of Broadcast Engineers
  • Received : July 14, 2014
  • Accepted : September 10, 2014
  • Published : September 30, 2014
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
상일 박
sangilparkmail@gmail.com
성근 유
영화 이

Abstract
From the year 2009, 3D Stereoscopic movies and TV have been spotlighted after the huge success of a movie called “AVATAR”. Moreover, most of 3D movies & contents are created by mixing real-life shots & virtual animated pictures, such as “Robocop 3”, “Transformer 4” as shown in 2014. However, the stereoscopic 3D video film shooting with a traditional stereoscopic rig camera system, takes much more time to set the rig system and adjust the system setting for proper film making which necessarily resulting in bigger cost. In fact, these problems have depreciated the success of Avatar as decreasing demand for 3D stereoscopic video shooting. In this paper, inherent problems of traditional stereoscopic rig camera system are analyzed, and as a solution for the problems, a novel implementations of single-lens optical stereoscopic 3D camera system is suggested. The new system can be implemented to a technology for separating two lights when even those lights passing through in the same optical axis. The system has advantages of adjusting the setting and taking video compared with traditional stereoscopic 3D rig systems. Furthermore, this system can acquire comfortable 3D stereoscopic video because of the good characteristics of geometrical errors. This paper will be discussed the single-lens stereoscopic 3D camera system using rolling shutters, it will be tested geometrical errors of this system. Lastly, other types of single lens stereoscopic 3D camera system are discussed to develop the promising future of this system.
Keywords
Ⅰ. INTRODUCTION
Stereoscopy is a technique which applies the biological phenomenon that brain recomposes stereoscopy images which are occurred when two eyes with some distance watch one object and it’s called binocular disparity [1] . At this point, inter-ocular distance is a distance between two eyes and the distance is the major depth cue for depth perception. Two eyes focus and adjust the angle of eyeballs to obtain a clear vision as seeing an object. That procedure is called accommodation and convergence respectively [1] . When shooting a stereoscopic video, cameras is installed with inter-axial distance to make binocular disparity [2] . Thus, two cameras perform accommodation and convergence which are the same as what two eyes do. While showing stereoscopic videos to two eyes respectively using various methods, a brain recomposes stereoscopic images as if people actually watch an object directly for real and feels depth perception. A camera rig system consists of a slider bar and two cameras which are put on the bar and the distance and angles of cameras are controlled to take Stereoscopic 3D video.
The camera rig system has been a major equipment to shoot stereoscopic videos [3] . However, there are problems when accommodation and convergence are performed. The lenses of two camera have to be synchronized in accommodation and the whole angles of cameras have to be changed in convergence. The problems explained above interfere a perfect matching with two images and cause viewers to feel visual fatigue [2] . In chapter 2, inherent problems in traditional stereoscopic 3D camera systems will be analyzed. Single lens stereoscopic 3D camera has been proposed to resolve those problems [6] . One method which has one optical axis and uses mirrors to make binocular disparity is also suggested. However, it’s technically difficult to divide left and right images spatially having one optical path. Also, several electronic matters should be considered to synchronize left and right images. In chapter 3, pros and cons of single lens stereoscopic 3D camera are discussed and system realization depending on the number of image sensors is explained. To figure out characteristics of the embodied single lens stereoscopic 3D camera system, a suitable test procedure is necessary to evaluate performance of 3D cameras. In chapter 4, it is suggested that several evaluation items considering factors of single lens stereoscopic 3D camera system and what the test method is. Furthermore, the reason why subjective quality test is needed will be explained
Ⅱ. INHERENT PROBLEMS IN TRADITIONAL STEREOSCOPIC 3D CAMERA SYSTEMS
Traditional rig system consists two cameras put on a parallel slider bar which can perform taking stereoscopic movies after alignment. The distance of two lenses optical-axis are separated as much as human eyes distance: 6.5cm [2] . When taking movies using large aperture lens or big cameras, it cannot be possible to put on the rig in parallel. A rig system that mounts at 90 degrees Cameras has been suggested to solve the problems. Fig. 1 shows various types of rig systems. Accommodation are performed in rig system, one lenses movement is should be synchronized with the other lens. Also heavy camera units should be moved at an angle to perform convergence. The traditional rig system using two cameras and two lenses has inherent problems that are inevitable. More complicated problem is optical axis mismatch with lenses and image sensor. Opticalaxis error can be occurred when two cameras are same model and two lenses are the same. When two cameras and two lenses are zooming, this error is emphasized. Fig. 2 shows this type of problem. When two images are different, human`s brain chooses better quality image between two and then human perceives prior image [1] . That is called retinal rivalry. When it’s repeated, viewers feel visual fatigue and are bothered to be immersed. So newly designed single lens stereoscopic camera system which has one optical-axis will be needed.
PPT Slide
Lager Image
리그 시스템의 다양한 종류들 Fig. 1. Various types of rig systems
PPT Slide
Lager Image
줌 렌즈에서의 광축과 이미지 센서의 중심 불일치 Fig. 2. Optical axis and center of image sensor mismatch on zoom lens
Ⅲ. IMPLEMENTATION FOR STEREOSCOPIC 3D CAMERA SYSTEM USING SINGLE LENS OPTICAL SYSTEMS.
As discussed above, single lens stereoscopic 3D camera systems has been suggested. To get better image acquisition, stereoscopic image should be acquired by some rules.
  • •The lens axes of the camera lenses are parallel.
  • •The distances between the two cameras, between the right and left images, and between the pupils are the same.
  • •The field angle of the camera lenses and the view angle of the image are the same[5].
To satisfy those rules, the lights passing through lenses should be separated as much as the binocular disparity and be transmitted to an object lens and be transmitted to each image sensors using mirrors or prisms. To implement this system, the entrance of pupil of the refractive optical system has to be moved in front of the refractive optical system to reduce the size of the reflector using relay lens [6] . Also each light should be passing through one optical pathway. However each light should not be mixed up with. Inherently, optical combiner/splitter are needed.
A suggested Temporal C/S is implemented with a rolling shutters. As shown Fig. 3 , the rolling shutter is placed in front of the lenses and opened sequentially and closed to make the left and right images. Therefore, the image sensor can acquire the stereoscopic image. This method can be applied well when dividing the left image and right image in temporal domain. However, a mechanical devices including rolling shutters has to be designed very precisely. Moreover, the rolling shutter makes problems of vibration and noise. The noise and vibration can cause the motion as the images in temporal domain are split. According to this, twice faster frame-rate are desired to acquire normal frame-rate and twice faster performance of cameras are also needed. However, it is need to doubled shutter speed so that it can not be achieved in diminished light environment commonly. This single lens system have many advantages compared with traditional rig system.
  • •Alignment: after factory alignment can be performed, further adjustment is almost not needed.
  • •Adjustment: Inter-axial distance and convergence are adjusted by changing angle of small mirrors.
  • •Optical-axis mismatch between image sensor and lens: each light can pass through just one optical pathway.
PPT Slide
Lager Image
회전하는 셔터를 사용하는 시분할 C/S를 채택한 단일렌즈 양안시 카메라 Fig. 3. Single lens stereoscopic camera with temporal domain C/S using rolling shutters
IV. A SUGGESTION ABOUT QUALITY TEST METHODS FOR NEWLY DESIGNED SINGLE LENS STEREOSCOPIC 3D CAMERA SYSTEM.
The traditional camera quality test contains numerous test items which are about color quality and sharpness, distortion. Especially stereoscopic 3D image is consist of two consecutive frames, geometric error can be calculated by comparing two frames. So some test items are added to the traditional camera quality test for stereoscopic 3D camera.
  • •Alignment error
  • •Keystone distortion.
  • •Size error
  • •Rotational error
This test item represents geometric errors on stereoscopic 3D camera. Following this test items are described in Table 1 . A test for single lens stereoscopic camera using the previous Rolling Shutter had been taken with the test items above. The following is the method of a test. As shown in Fig. 4 ., Single lens system is set up to a camera which has zoom lens. The camera can take HD videos which has Fig. 3 . Single lens stereoscopic camera with temporal domain C/S using rolling shutters1080 pixels of horizontal resolution with 60 interlace mode. The view angle of the camera is 7.8mm-164mm. The central axes of the camera and a target board are matched precisely and the video is taken with the distance of 3M, 2M, 1.5M. To obtain images for the test, the camera captures videos by zooming in on each of three circles which have different sizes on the target board. The pattern of target board is shown in Fig. 5 . After taking videos, each right and left images is sorted to even and odd field and saved as static image files. As the resolution of the saved images are halved, interpolation is conducted to obtain twice larger horizontal resolution. As shown in Fig. 6 , the final resulting images is comprised of right and left images with 1920X1080 resolution in one display. To test items of Fig. 7 , errors between image layers are measured with a graphic tool by the unit of 1/100 sub-pixel. Finally, the measured items are converted to result values according to the equation of Table. 1 . Table. 2 and Table. 3 show the end result and average end result respectively.
공식들
PPT Slide
Lager Image
Table 1. Equations
PPT Slide
Lager Image
회전하는 셔터를 적용한 단일렌즈 양안시 카메라 시스템 Fig. 4. The single lens stereoscopic camera system enabled rolling shutters
PPT Slide
Lager Image
타겟 차트의 모양 Fig. 5. The figure of target chart
PPT Slide
Lager Image
양안시 영상에서 획득한 테스트 이미지 Fig. 6. The test image captured from stereoscopic videos
PPT Slide
Lager Image
양안 영상을 테스트하기 위한 테스트 기준 Fig. 7. The test measurement methods for stereoscopic videos
에러에 대한 각각의 결과값
PPT Slide
Lager Image
Table 2. The Result of each items error
결과값(평균)
PPT Slide
Lager Image
Table 3. Total results(average)
The most notable result is that there were less than average 1% errors occurred in most test items even though it was taken with various view-angles and distances. Furthermore, the test drew that Single lens stereoscopic 3D camera system contributes to complete the test faster comparing to the previous rig system requires much time to set up such as aligning.
The tested camera is a prototype using rolling shutters. However, the test results are promising good qualities on stereoscopic 3D camera system. though human factor are not considered on this test result. So, subjective quality test will be needed for stereoscopic 3D camera system [4] .
V. FUTURE WORKS
Temporal C/S with rolling shutter has a numerous advantages comparing to the other traditional rig systems, As the rolling shutter system has many problems, advanced Temporal C/S which doesn't require mechanical shutters are designed to overcome those problems. As shown Fig. 8 The devices above are adopted a Liquid Crystal shutters which can control opacity by electric signals instead of using rolling shutters with high rotation speed rate. This method is robust to noise and vibration which are caused by mechanical devices. Nevertheless, reduced temporal resolution and shutter speed are still remained as a problem. Also, total intensity of light are reduced in half when a light passing through the Liquid Crystal Shutters.
PPT Slide
Lager Image
액정 셔터를 사용한 시분할 C/S를 채택한 단일렌즈 양안시 카메라 Fig. 8. Single lens stereoscopic camera with temporal domain C/S using Liquified Crystal Display shutters
A Spatial C/S has been suggested to overcome the obstacles explained above. The Spatial C/S can be multiplexing left and right images by using polarization of lights. Fig. 9 is shown more detailed structure. The left and right entrance can filter light through polarization filters. In a general case, two lights are mixed or interfered with each other when two different light passed through in same optical axis. However, the two lights can not be mixed or interfered by using polarizing filters. After two different lights are passed to optical lenses when zooming or focusing, the lights are transferred straightly to each left and right image sensors by polarizing filters. Two view images can be acquired at the same time by using this method.
PPT Slide
Lager Image
편광 필터를 사용한 공간분할 C/S를 채택한 단일렌즈 양안시 카메라 Fig. 9. Single lens stereoscopic camera with spatial domain C/S using polarizing filters
The Temporal C/S reduced temporal resolution. In a case of 3D display using Shutter Glasses, using Temporal C/S gives more natural images if time synchronization is well performed. The quantity of light is reduced in half when applying Spatial C/S. That's because the light is passed through the polarization filter and integrated. However, there's an advantage of resolution for the reason both the temporal and spatial resolution reduced. Thus, using Polarized glasses is suitable for the Spatial method when watching 3D. Fig. 10 . shows a comparison with spatial and temporal C/S's sequences.
PPT Slide
Lager Image
공간분할과 시간분할 C/S의 순서 a) 공간분할, b) 시간 분할 Fig. 10. The Sequences of Spatial and Temporal C/S a) Spatial, b) Temporal
Ⅴ. CONCLUSION.
In this paper, a single lens stereoscopic camera was introduced and compared to the traditional two-camera rig system. This new camera system has temporal domain optical combiner and splitter using a rolling (rotating) shutter. Since this new system utilizes only one main camera lens for left and right light source, it is inherent to have a good focusing and zooming picture quality. Inevitably, traditional stereoscopic 3D camera (rig) system having two optical axes causes viewers to have visual fatigue and it leads to the limitation of broad distribution of 3D contents. As shown in this paper, one optical axis system can be a solution for visual fatigue. Also various quality tests proved that suggested single lens stereoscopic camera system is more suitable for UHD (4K and 8K) stereoscopic 3D.
Also two more advanced structures are introduced for the future work. Both structures have different optical combiner and splitter (C/S). The first one is with temporal domain combiner with two set of LCD shutters instead of a rotating shutter. And the other one is with spatial domain combiner and splitter without any shutters, however, it requires two CCD (or CMOS) sensors on the back. All of these proposed structures are with one main camera lens which are immune to have focusing and zooming problems
BIO
박 상 일
- 1977년 2월 : 연세대학교 전자공학과 졸업(공학사)
- 1983년 8월 : Kansas State University 전기전자공학과 졸업(석사)
- 1987년 2월 : University of New Mexico 전기전자공학과 졸업(박사)
- 1987년 1월 ~ 1988년 5월 : University of Pittsburgh 전자공학과 조교수
- 1988년 5월 ~ 1995년 5월 : MotorolaDSP SemiconductorDesign Manager
- 1995년 5월 ~ 2006년 11월 : 삼성전자 임원(반도체, 비서실, 본사기획실등)
- 2006년 11월 ~ 2012년 10월 : 스카이레이크인큐베스트투자팀 부사장
- 2009년 8월 ~ 2012년 10월 : 방송통신위원회 차세대 방송 PM
- 2012년 10월 ~ 현재 : 서울과학기술대학교 전자IT미디어공학과 교수
- 주관심분야 : 차세대 방송, 실감방송, 클라우드 플랫폼, 의료 전기전자
유 성 근
- 2013년 2월 : 서울과학기술대학교 전자IT미디어공학과 졸업(공학사)
- 2013년 2월 ~ 현재 : 서울과학기술대학교 미디어IT공학과 석사과정
- 주관심분야 : 차세대 방송, 실감방송, 클라우드 플랫폼
이 영 화
- 1969년 2월 : 동아대학교 기계공학과 졸업(공학사)
- 1964년 8월 ~ 1969년 6월 : 금성사 전자설계실 연구원
- 1972년 3월 ~ 1996년 2월 : 삼환전자 대표
- 1998년 10월 ~ 현재 : (주)아솔 대표
- 주관심분야 : 입체영상 획득, 입체영상 방송
References
An S. M. , Ramesh R. , Lee Y. S. , Chung W. Y. 2011 “Interaxial distance and convergence control for efficient stereoscopic shooting using horizontal moving 3D camera rig" ICMVIPPA 59 (408) 2176 - 2181
Ukai K. , Howarth. P. A. 2008 “Visual fatigue caused by viewing stereoscopic motion images: Background, theories, and observations” Health and Safety Aspects of Visual Displays 29 (2) 106 - 116
AVID 2011 “How to shoot, capture and edit 3D? The practise...” http://learn.avid.com /content/_Europe_PTG_cours-eware/NPI/Introduction%20to%20Stereoscopic%203D%20for%20ALEX/3_How%20to%20shoot%20capture%20edit%203D.pdf
Onural L. , Sikora T. , Ostermann J. , Smolic A. , Civanlar M. R. , Watson J. 2006 "An assessment of 3DTV technologies" Proceedings of the NAB Broadcast Engineering Conference
Yamanoue H. , Okui M. , Oksno F. , Yuyama I. 2001 "Development of a Compact 3-D HDTV Camera with Zoom Lens" Journal of the Optical Society of Korea 5 (2) 49 - 54    DOI : 10.3807/JOSK.2001.5.2.049
Park S. , Yoo S , Lee Y , Lee T 2013 "New broadcasting technology based on single lens video capturing system for 3D/UHD applications" ICT Convergence (ICTC), 2013 International Conference on. IEEE 506 - 511