Advanced
Feasibility of Using an Automatic Lens Distortion Correction (ALDC) Camera in a Photogrammetric UAV System
Feasibility of Using an Automatic Lens Distortion Correction (ALDC) Camera in a Photogrammetric UAV System
Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography. 2015. Dec, 33(6): 475-484
Copyright © 2015, Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • Received : October 10, 2015
  • Accepted : December 12, 2015
  • Published : December 31, 2015
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Hohyun Jeong
Member, Dept. of Spatial Information Engineering, Pukyong National University (E-mail:skyeyes82@pukyong.ac.kr)
Hoyong Ahn
Member, Dept. of Spatial Information Engineering, Pukyong National University (E-mail:hyahn85@pukyong.ac.kr)
Jinwoo Park
Dept. of Spatial Information Engineering, Pukyong National University (E-mail:swat018@gmail.com)
Hyungwoo Kim
Dept. of Spatial Information Engineering, Pukyong National University (E-mail:kalituma@pukyong.ac.kr)
Sangseok Kim
Member, Dept. of Industry and Academic Cooperation, Dongeui Institute of Technology (E-mail:kimss@dit.ac.kr)
Yangwon Lee
Dept. of Spatial Information Engineering, Pukyong National University (E-mail:modconfi@pknu.ac.kr)
Chuluong Choi
Corresponding Author, Member, Dept. of Spatial Information Engineering, Pukyong National University (E-mail:cuchoi@pknu.ac.kr)
Abstract
This study examined the feasibility of using an automatic lens distortion correction (ALDC) camera as the payload for a photogrammetric unmanned aerial vehicle (UAV) system. First, lens distortion for the interior orientation (IO) parameters was estimated. Although previous studies have largely ignored decentering distortion, this study revealed that more than 50% of the distortion of the ALDC camera was caused by decentering distortion. Second, we compared the accuracy of bundle adjustment for camera calibration using three image types: raw imagery without the ALDC option; imagery corrected using lens profiles; and imagery with the ALDC option. The results of image triangulation, the digital terrain model (DTM), and the orthoimage using the IO parameters for the ALDC camera were similar to or slightly better than the results using self-calibration. These results confirm that the ALDC camera can be used in a photogrammetric UAV system using only self-calibration.
Keywords
1. Introduction
Estimating the interior calibration parameters of a camera from sequences of images is an important problem for a broad range of applications ( Hartley, 2008 ). Almost all camera lenses suffer from some distortion, which reduces the positional accuracy of image points located on the image plane. This is one of the oldest computer vision problems and although much has been solved, some questions remain ( Kukelova and Pajdla, 2011 ).
Lens distortion may include symmetric radial distortion, decentering distortion, affinity distortion, and non-orthogonal deformation ( Fraser, 1997 ). Radial distortion is symmetrical relative to the principal point of the image and is calculated with the polynomial of odd powers against the radial distance ( Wolf ., 2000 ). Decentering distortion is the error that occurs when the lens centers are not concordant; it has less effect on distortion than does symmetric radial distortion. Affinity distortion and non-orthogonal deformation are typically very small. Consequently, lens distortion has recently been considered in terms of symmetric radial and decentering distortion ( Fraser, 1997 ; Kim ., 2013 ).
Several methods have been proposed for correcting the radial distortion of lenses causing moderate distortion, including the radial model by Brown (1971) , the rational model by Claus . (2005) , and the division model by Lenz (1987) . Automatic approaches for correcting radial distortion have been attempted ( Gonzalez-Aguilera ., 2011 ; Lee ., 2013 ; Cai, 2014 ). Additionally, intelligent cameras with automatic lens distortion correction have been introduced widely. A smart camera can correct lens distortion automatically as users shoot, and users are never required to load lens profiles. Therefore, the use of smart cameras allows a photogrammetric unmanned aerial vehicle (UAV) system to be developed at much lower cost and with much faster calibration than earlier systems. However, no research has considered the accuracy of an automatic lens distortion correction (hereafter, ALDC) camera in this context.
This study assessed the feasibility of using an ALDC camera as the payload for a photogrammetric UAV system. The study’s primary contributions can be summarized as follows:
1. Three types of imagery were used for lens calibration of the ALDC camera: raw imagery without the ALDC option (Type 1, only remove chromatic and vignette); imagery corrected using lens profiles (Type 2); and imagery with the ALDC option (Type 3).
2. We examined the effects of ALDC on the accuracy of aerial triangulation and a digital terrain model (DTM) with the photogrammetric UAV system.
2. Methodology
- 2.1. Camera calibration
An ALDC camera can estimate lens distortion using its lens firmware and then correct the lens distortion automatically. In this study, the Samsung SMART NX camera was chosen as the ALDC camera. It has a focal length of 16 mm, but when taking an image with the ALDC option, the field of view was 71.65° by 51.41° (wide by high). With the raw image without the ALDC option, this increased to 72.62° by 52.09°. The raw image was 1.3% larger than the image with the ALDC option. As shown in Fig. 1 , the actual picture element of the Samsung SMART NX ALDC is 20.50 megapixels, but the four corners and four edges of the wide angle lens produced barrel distortion.
PPT Slide
Lager Image
Lens distortion before and after correction: (a) after removing only chromatic and vignette using lens profiles; and (b) the result with auto lens distortion correction
Most cameras in photogrammetric UAV systems have inexpensive lenses, which can cause lens distortion and reduce the accuracy of DTMs. Accordingly, the most important issue in using a camera for photogrammetric purposes is to correct the distortion, because it shifts the image points locally and causes errors in their position ( Lee ., 2012 ). Lens distortion is divided into radial and decentering. The distortion can be described as follows:
PPT Slide
Lager Image
PPT Slide
Lager Image
where xc and yc are the corrected image points, drx and dry are the ( x , y ) components of the radial lens distortion correction, and dpx and dpy are the ( x , y ) components of the decentering lens distortion correction. Radial lens distortion ( dr ) is described as:
PPT Slide
Lager Image
where, r 2 = x 2 + y 2 , K1, K2, and K3 are the parameters. Decentering lens distortion is:
PPT Slide
Lager Image
PPT Slide
Lager Image
where P1 and P2 are the parameters, r is in mm from the principal point, x and y are in mm from the principal point.
A Samsung NX camera can obtain two types of image: raw images without the ALDC option (Type 1) and images with the ALDC option (Type 2). Images corrected using lens profiles (Type 3) can be estimated using Adobe Lens Profile Creator, which can incorporate three factors: lens distortion, chromatic aberration, and vignette.
Camera calibration and the IO parameters were calculated using the PhotoModeler software, which is a photogrammetric software for image-based measurement and 3D modelling developed by Eos Systems Inc., particularly designed for the photogrammetric close range.
Calibration was accomplished using multi-sheet method that uses several sheets having ringed automatically detected (RAD) coded targets. Taking into account the focal length and CCD size/resolution of the camera, 16 sheets of A3-size sheets of RAD-coded targets were obtained (inner target diameter: 12.6 mm; height: 1.8 m; width: 1.8 m). These targets provide the ability to automatically mark, recognize, and reference targets in a scene ( Fig. 2 )
PPT Slide
Lager Image
Calibration sheet used in this study (Multi RAD-coded target)
- 2.2. Photogrammetric UAV experiment and field campaign
Photogrammetric UAV experiments were conducted at Pukyong National University in South Korea ( Fig. 3a ). The UAV for this experiment was an X8+ double motor octocopter, equipped with a Pixhawk autopilot and 3DR u-blox GPS with a compass. This was an appropriate site for evaluating the three types of image due to its flat geography.
PPT Slide
Lager Image
The test field for study area (a) Aerial view of the region in which the UAV images were taken. The central area (marked in red) is where the experiment was conducted (b) Flight course in this study (c) Distribution of the locations of GPS base stations, 3D laser scanners, and GCPs on the final orthoimages (d) TLS LiDAR DTM
The study area was selected to provide six flight courses (length 80 m, interval 10m) to take pictures of the size 100 m by 100 m. The time interval for image-taking was set to 5 to 6 seconds for over 60% of the overlap ( Fig. 3b ). The instant field of view (IFOV) angle for the upper side lap 60% and overlap 60% was set according to the camera specification. The targets for GCPs were designed with an interval of 7 m to 10 m, and the radius for targets was 8 cm.
Before the photogrammetric UAV experiment, we conducted a field campaign to determine the ground control point (GCP) using aerial signal targets. The 3-D coordinates of control points are very important in processing images and generating ortho-rectified images. They are used as the GCPs for image triangulation and are also used to triangulate 3-D laser scanning data. To determine the 3-D coordinates of the two base station in the test field, we used GPS. Using the two control points determined by GPS surveying, 96 GCPs were established using a Total Station ( Fig. 3c ). The scanning process was implemented to generate DTM ( Fig. 3d ).
GPS accuracy in the field experiment was horizontal 11 mm and vertical 16 mm, whereas the accuracy of 3-D laser scanning was horizontal and vertical 3 mm. The accuracy of total station surveying was horizontal 3.2 mm and vertical 1 mm. The accuracy of GPS surveying was slightly greater than the accuracy listed in the technical specifications, and vertical accuracy was worse than the horizontal accuracy. The 3-D laser scanner and total station accuracies were very good, within 3 mm in both vertical and horizontal directions ( Table 1 ).
Field survey equipment using and accuracy
PPT Slide
Lager Image
1) ppm : parts per million
More vibration occurred in the rotorcraft UAV than a fixed wing craft. Movement of the sensors should be avoided and the influence of vibrations has to be reduced ( Henri, 2009 ). A vibration proof gimbal was attached for ALDC camera on the lower side of the rotorcraft UAV ( Fig. 4 ). The electric rotorcraft UAV showed reduced higher frequency vibrations than a gasoline engine UAV; high frequency vibrations can be critical to UAV sensors and serious effects can result on magnetic sensors from the magnetic field created by the rotating motor. The magnetic compass on the UAV suffers significant interference from the motors and power wires, because magnetic interference is linearly related with the current drawn. It is technically possible to set up compass and motor calibration relative to the throttle.
PPT Slide
Lager Image
Research UAV & vibration reduction device (a) A front view of UAV with a vibration reduction device, (b) A 65mm offset between camera and UAV, (c) A top view of UAV, (d) The camera balsa frame
3. Result & Discussion
- 3.1. Camera calibration
Calibrations and IO orientation are calculated from the perspective geometric model with bundle adjustment ( Brown, 1971 ). This study compared The interior orientation (IO) parameters were calculated by Photomodeler for three sets of ALDC type. Multi RAD-coded targets sheet area used to correct for irregular lens distortion found on ALDC cameras.
Table 2 lists the calibrated focal length (mm), principal point x0, y0, radial distortion, and decentering distortion estimates. The overall coordinate RMSE for the ALDC camera images was 0.5750-0.6380 pixel and the RMS error for object coordinates was less than 0.1mm from the object plane (1.8m x 1.8 m). The maximum marking residual of the calibration quality of the ALDC camera was 0.8170-0.9687.
Interior orientation of each type ALDC camera by calibration
PPT Slide
Lager Image
Interior orientation of each type ALDC camera by calibration
The calibrated focal lengths (mm) in the Type 2, Type 1, and Type 3 images were 16.6176, 16.7895, and 16.806, respectively. Differences were observed in the principal point x0, y0 (mm): (−0.1758, −0.0545), (−0.1795, −0.0548), and (−0.2163, 0.0031), respectively. The Type 2 and Type 1 values were similar, but the Type 3 estimate was larger. This is likely because the Raw file differs in pixel width and height. The pixel size x (mm) was 4.3842-4.3848, and pixel size y (mm) was 4.3860; each ALDC camera was similar.
Generally, of the symmetric radial distortion and decentering coefficients K1, K2, K3, P1, and P2, K3 are seldom considered because of their small magnitudes ( Fraser, 1997 ). In this study (see Table 2 ), We estimated radial and decentering lens distortion with K1, K2, P1, and P2. The radial lens distortion ( Fig. 5a ) differed in some ways. Lens radial distortion was in the order Type 1 (21.6) > Type 2 (1.1) > Type 3 (0.6). ALDC Raw (Type 1) exhibited large lens radial distortion, but Type 2 (imagery corrected by lens profiles) and Type 3 (imagery with ALDC) radial distortion exhibited much less. Decentering lens distortion is shown in Fig. 5b . The lens decentering distortion was in the order Type 3 (5.55-12.46) > Type 2 (3.77-10.01) > Type 1 (0.65-0.94). The lens distortion ratio is shown in Fig. 5c . The x-axis is the Ratio = (Radius/Max Radius) and the y-axis is Decentering / (Decentering + Radial).
PPT Slide
Lager Image
Radial and decentering distortion curve for each camera Type Device (a) Radial Lens Distortion, (b) Decentering Lens Distortion, (c) Decentering distortion Ratio
In the ALDC RAW file, the effect of the decentering distortion ratio was very small (<4%); the radial distortion was mostly removed. However, the lens decentering distortion was increased in the ALDC images ( Fig. 5b and 5c ). Decentering lens distortion has largely been ignored in past research, but it is important with ALDC cameras. Thus, a small-format camera can get much better results when decentering and radial distortion are considered, rather than radial distortion only ( Karras ., 1998 ). That is, the accuracy of IO for an ALDC camera depends on how decentering lens distortion is considered.
- 3.2 Image block triangulation
Photogrammetric image processing of the data, consisting of tie point measurement, bundle adjustment, DTM, and orthoimage generation, was performed using ERDAS LPS software.
Manufacturer interior orientation (M.IO), including focal length and manufacturer pixel size, was used. For the principal point X0, Y0, the radial and decentering distortion parameter was “0”, and the self-calibration option was IO for focal length, and principal point x0, y0 standard deviation was ±0.001 mm for triangulation.
Most previous research has only used radial lens distortion, but the results of this study indicated that for lens calibration for interior orientation, decentering distortion is also important. Therefore, we analyzed C.IO was analyzed for accuracy considering decentering lens distortion. M.IO cannot consider radial and decentering in the IO parameter, so only a C.IO camera was analyzed.
The results of image triangulation for each Interior Orientation and camera type are shown in Table 3 .
The result of aerial triangulation's by interior orientation and camera type
PPT Slide
Lager Image
The result of aerial triangulation's by interior orientation and camera type
In the case of the considering both radial and decentering parameter (D&R), total image unit-weight RMSE (The total root mean square error for the triangulation) ALDC and ALDC RAW were 0.723 and 0.734 pixels, respectively, whereas considering only radial parameter, its RMSE showed 1.082 and 0.765 pixels, respectively.
When considering radial and decentering (D&R) parameter, depending on verification of the camera, the total image unit-weight RMSE showed better than those for the only radial (R) lens distortion were considered. However, C.IO ALDC exhibited higher levels of improvement with high decentering distortion ratios. The deviation of the distortion center from its true location under both lens radial and decentering distortion was equivalent to adding two additional decentering distortion terms ( Moumen, 2005 ). Wide-angle lens decentering distortion is larger and the C.IO (D&R) is more effective. Also, C.IO (D&R) can reduce the sum of C.IO (R) residues ( Wang ., 2008 ).
based on this results, if a No ALDC camera (e.g. Canon IXUS) was used considering only radial distortion, a ‘useful’ result can be derived. However, with an ALDC camera, one has to consider radial and decentering distortion. Indeed, decentering lens distortion is critical in ALDC.
- 3.3 DTM and orthoimage generation
Orthoimages for each ALDC and DEM generated by the automatic terrain extraction (ATE) functionality in LPS 9.2, are shown in Fig. 6 . Also, a comparison was made between TLS DTM regarding the same area and image-based DTM to assess differences between them ( Table 4 ). Based on the difference of DTM extraction from the images and TLS DTM accuracy was evaluated by difference mean and standard deviation C.IO ALDC were 0.0cm and 2.5cm, C.IO ALDC Raw were 0.3cm and 3.2cm, respectively, whereas The M.IO ALDC were 2.2cm and 3.8cm, M.IO ALDC Raw were 35.0cm and 16.5 cm, in order. Although differences appeared in the histograms of DTM extraction from images ( Fig. 7 ), TLS DTM, and Calibrated IO had similar distributions, but the manufacturer IO one was different.
PPT Slide
Lager Image
Orthorectified image and DTM (a) Calibrated interior orientation ALDC orthoImage, (b) Calibrated interior orientation ALDC DTM, (c) Manufacturer interior orientation ALDC raw orthoImage (d) Manufacturer interior orientation ALDC raw DTM
Extraction DTM and difference each DTM and TLS DTM statics (unit : m)
PPT Slide
Lager Image
1) standard deviation
PPT Slide
Lager Image
TLS DTM and difference histogram each type image DTM
When converting the altitude of a Sony nex7 and UAV to the same altitude from an archaeological survey result, the standard deviation was ±2.56-3.52 cm ( Flener ., 2013 ). When a swinglet (UAV) and a Canon IXUS 120 were used, the result from taking the picture DTM height difference mean and standard deviation were 1.2±2.4 cm and 2.8±6.8 cm at the same fight height ( Vallet ., 2011 ). Thus, C.IO ALDC, C.IO ALDC Raw, and M.IO ALDC are considered to be within the level of significance. The maximum absolute values are about 20 m, near building edges and in the central area covered by very high trees. In most areas (~90%), differences were in the range of −30 to 40 cm. Moreover, the average of the differences being close to zero shows the absence of vertical and horizontal biases for all DTM ( Moumen, 2005 ) that is, when we convert to the same flight altitude, 6.23 to 8.31 cm. With the estimated ~90% difference, C.IO ALDC Raw were in the range of −4 to 2 cm and C.IO ALDC were-4.5 to 2cm showed remarkably good results, whereas M.IO ALDC were in the range of −8 to 6cm was reasonably similar, and M.IO ALDC Raw were −60 to −0.15cm showed poor results.
In the cases using a calibrated IO, the point quality was sufficiently high to play an important part in achieving a good result. In the case of using the manufacturer IO, ALDC was relatively good in each field, but the other M.IO file was low for the DTM extraction ratio. Thus, to obtain high quality data, a highly accurately C.IO (D&R) is considered to be essential.
5. Summary and Conclusions
Since 2010, when cameras were first equipped with powerful CPUs, auto lens distortion correction (ALDC) has become possible and ALDC is now used with self-calibration in nonphotography applications. However, use of ALDC still requires a rigorous evaluation of the camera. Recently, the Samsung NX camera, a Smart ALDC camera, has added internal clock time synchronization as accurate as 1/1,000 s, and is equipped with all kinds of sensors, so it seems to have great potential; however, careful consideration of the Smart ALDC camera’s accuracy and problems remains important.
ALDC image lens distortion was outstanding in comparison with imagery corrected by lens profiles. However, when assessing nonmetric and small-format camera, ALDC Raw images have high lens decentering distortion that previous research has largely ignored, but that increases greatly in ALDC processes. Thus, when comparing C.IO (D&R), and C.IO (R), ALDC Raw images showed marginal differences, but there were large improvements in C.IO (D&R) in ALDC. Accordingly, when using C.IO (D&R), a much better result is expected. However, C.IO (R) showed similar or better results than M.IO ALDC and self-calibration, so removal of decentering lens distortion is very important.
The accuracy of the final products (orthoimages and DTM) can be greatly improved by careful calibration. However, although some errors still appeared, the M.IO ALDC and self-calibration settings can generally be used by nonphotogrammetry experts, except for cases demanding high accuracy of orthoimages and DTM.
Acknowledgements
This work was supported by a Research Grant of Pukyong National University (2014 Year).
References
Brown D.C. (1966) Decentering distortion of lenses Photogrammetric Engineering 32 444 - 462
Brown D.C. (1971) Close-range camera calibration Photogrammetric Engineering 37 (8) 855 - 866
Cai J. (2014) Automatic lens distortion correction using single images by finding reliable lines for rectification ICARCV Proceedings of the 2014 13th International Conference on Control, Automation, Robotics & Vision Singapore, Singapore 10-12 December 1015 - 1020
Claus D. , Fitzgibbon A.W. (2005) A rational function lens distortion model for general cameras IEEE Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition San Diego, USA 20-25 June 1 213 - 219
Flener C. , Vaaja M. , Jaakkola A. , Krooks A. , Kaartinen H. , Kukko A. , Alho P. (2013) Seamless mapping of river channels at high resolution using mobile lidar and UAV-photography Remote Sensing 5 6382 - 6407
Fraser C.S. (1997) Digital camera self-calibration International Archives of Photogrammetry and Remote Sensing 52 (4) 149 - 159
Gonzalez-Aguilera D. , Gomez-Lahoz J. , Rodriguez-Gonzalvez P. (2011) An automatic approach for radial lens distortion correction from a single image IEEE Sensors Journal 11 (4) 956 - 965
Hartley R. (2008) Multiple View Geometry in Computer Vision Cambridge University Press Cambridge, UK
Henri E. (2009) In UAV Photogrammetry, Ph.D. dissertation ETH Zurich Zurich, Switzerland 203 -
Karras G.E. , Mountrakis G. , Patias P. , Petsa E. (1998) Modeling distortion of super-wide-angle lenses for architectural and archaeological applications International Archives of Photogrammetry and Remote Sensing 32 (5) 570 - 573
Kim J. , Lee S. , Ahn H. , Seo D. , Seo D. , Lee J. , Choi C. (2013) Accuracy evaluation of a smartphone based technology for coastal monitoring Measurement 48 (1) 233 - 248
Kukelova Z. , Pajdla T. (2011) A minimal solution to radial distortion autocalibration IEEE Transactions on Pattern Analysis and Machine Intelligence 33 (12) 2410 - 2422
Lee S. , Kim J. , Jin C. , Bae S. , Choi C. (2012) Assessment of smartphone based technology for remote environmental monitoring and development Instrumentation Science and Technology 40 (6) 504 - 529
Lee T.Y. , Chang T.S. , Wei C.H. , Lai S.H. , Liu K.C. , Wu H.S. (2013) Automatic distortion correction of endoscopic images captured with wide angle zoom lens IEEE Transactions on Biomedical Engineering 60 (9) 2603 - 2612
Lenz R. (1987) Linsenfehlerkorrigierte Eichung von Halbleiterkameras mit Standardobjektiven für Hochgenaue 3D - Messungen in Echtzeit Springer-Verlag Berlin, Germany
Moumen A. (2005) Nonmetric calibration of camera lens distortion: differential methods and robust estimation IEEE Transactions on Image Processing 14 (8) 1215 - 1230
Vallet J. , Panissod F. , Strecha C. , Tracol M. (2011) Photogrammetric performance of an ultra light weight swinglet “Uav” ISPRS Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences Zurich, Switzerland 14-16 September XXXVIII-1 253 - 258
Vallet J. , Skaloud J. , Koelbl O. , Merminod B. (2000) Development of a helicopter-based integrated system for avalanche mapping and hazard management International Archives of Photogrammetry and Remote Sensing 33 565 - 572
Wang J. , Shi F. , Zhang J. , Liu Y. (2008) A new calibration model of camera lens distortion Pattern Recognition 41 (2) 607 - 615
Wolf P. R. (1983) Elements of Photogrammetry Mcgrawhill New York, USA
Wolf P.R. , Dewitt B.A. , Wilkinson B. (2000) Elements of Photogrammetry: with Applications in GIS 3rd Edition McGraw-Hill New York, USA