Advanced
Parameterized Modeling of Spatially Varying PSF for Lens Aberration and Defocus
Parameterized Modeling of Spatially Varying PSF for Lens Aberration and Defocus
Journal of the Optical Society of Korea. 2015. Apr, 19(2): 136-143
Copyright © 2015, Optical Society of Korea
  • Received : November 11, 2014
  • Accepted : February 02, 2015
  • Published : April 25, 2015
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Wang Chao
University of Chinese Academy of Sciences, Beijing 100039, China
wc100714@163.com
Chen Juan
Changchun University of Industry, Changchun 130033, China
Jia Hongguang
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Dong Nanhu Road 3888, Changchun, Jilin 130033, China
Shi Baosong
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Dong Nanhu Road 3888, Changchun, Jilin 130033, China
Zhu Ruifei
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Dong Nanhu Road 3888, Changchun, Jilin 130033, China
Wei Qun
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Dong Nanhu Road 3888, Changchun, Jilin 130033, China
Yu Linyao
University of Chinese Academy of Sciences, Beijing 100039, China
Ge Mingda
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Dong Nanhu Road 3888, Changchun, Jilin 130033, China
Abstract
Image deblurring by a deconvolution method requires accurate knowledge of the blur kernel. Existing point spread function (PSF) models in the literature corresponding to lens aberrations and defocus are either parameterized and spatially invariant or spatially varying but discretely defined. In this paper, a parameterized model is developed and presented for a PSF which is spatially varying due to lens aberrations and defocus in an imaging system. The model is established from the Seidel third-order aberration coefficient and the Hu moment. A skew normal Gauss model is selected for parameterized PSF geometry structure. The accuracy of the model is demonstrated with simulations and measurements for a defocused infrared camera and a single spherical lens digital camera. Compared with optical software Code V, the visual results of two optical systems validate our analysis and proposed method in size, shape and direction. Quantitative evaluation results reveal the excellent accuracy of the blur kernel model.
Keywords
I. INTRODUCTION
Many factors can contribute to the undesired blurriness of a photograph. While blur sources such as camera shake, subject motion and defocus have been studied well, and researchers have proposed effective solutions to restore the corresponding photos, degradations due to the camera optical system have received little attention. This is particularly inappropriate because optical degradations affect every photograph and cannot be easily removed. This problem is wellknown in the photography community as “soft corners” or “coma aberration”, and it is a discriminating factor between entry level lenses and professional-grade equipment.
When blur is undesirable, one can deblur an image using a deconvolution method, which requires accurate knowledge of the blur kernel. In applications where blur is desirable and essential, such as shape from defocus, it is still necessary to recover the shape and size of the spatially varying blur kernel. The point-spread function (PSF) is a quantization form of blur kernel, which describes how a point of light is redistributed over a local area of the sensor in an imaging system.
The key insight of our work is that a novel parameterized model will be developed and presented for a spatially varying PSF due to Seidel aberrations and defocus in an imaging system. A parameterized spatially varying model can provide a regularization mechanism, which constrains each individual blur estimate and constrains the relationship between neighboring blur estimates.
In this article, Section II describes related work on the spatially varying PSF model in other literature. Section III describes the estimation algorithm developed by Simpkins for the spatially varying PSF model constructed from Seidel aberrations and optical system parameters. [1 , 7 , 10 , 13] The main contributions presented in this paper include comprehensive simulation of a proposed skew normal PSF model. Section IV and Section V presents results of a PSF model estimation experiment on a defocused infrared telescope system and a single lens digital camera with optical aberration. We compare the simulated results with standard PSF model generated from optical software Code V. Section VI presents conclusions and discusses the directions for future work.
II. RELATED WORK
Blur kernel models in the literature tend to fall into one of two categories: parameterized and spatially invariant or spatially varying and discretely defined. The downside of the spatially invariant model is that both simulation and experiments have demonstrated the need for PSF to have a spatial dependency. The downside of the discretely defined model is that it has many degrees of freedom for a PSF of reasonable size and requires a larger dataset for accurate kernel estimation. Additionally, there is no well-motivated mechanism to accurately interpolate the PSF for any particular point (x, y) in the image plane: other works on spatially varying blur have instead relied on ad-hoc interpolation of discretely defined PSFs corresponding to known points on the image plane.
Many techniques have been proposed to estimate and remove optical blur due to optical aberration, motion, and defocus. (See [1 , 3 , 4 , 6 , 7 , 9 , 10 , 13 , 14] and references therein.) Joshi et al. [3] estimate PSFs from edges in the image. They describe how to use a printed pattern to calibrate a camera at a given aperture and focal length and show that they can restore images taken later with the same parameters. The PSF can present spatially varying blur, which lacks a parameterized model. E. Kee et al. [6] introduce a method to estimate the blur kernel densely over the image and across multiple aperture and zoom settings. Simpkins and Stevenson [1 , 7 , 10 , 13] have developed methods using geometrical optics to construct a parameterized model for a spatially varying PSF due to lens aberrations and defocus.
A similar approach based on Seidel aberrations and optical system parameters is selected in this paper. The major improvement brought by our work is that we use our measures to build a parametric model of the spatially-varying optical blur. Our results show that, with our model, PSF models are built for a defocused infrared telescope and for a single lens digital camera with optical aberration.
III. PSF MODEL FORMATION
- 3.1. Seidel Aberration Model
The Seidel aberration model describes the path of a single ray of light passing through a lens. A derivation will then be presented of a composite model, which will be used as the proposed ray tracing model to account for lens aberrations and defocus.
Figure 1 shows the Seidel aberrated ray tracing procedure. The Seidel aberration model describes where a ray leaving from a point P 1 ’ on the exit pupil should arrive on the focused-image plane (at aberrated arrival point P 1 ), given an ideal arrival point P 1 * . The model is based on a third-order Taylor series expansion of Snell’s law and assumes a spherical lens surface which is symmetric about the optical axis. A complete presentation of the Seidel model is given by Born and Wolf [2] .
PPT Slide
Lager Image
Seidel aberrated ray tracing model.
The Seidel aberration model requires only five parameters corresponding to the aberration coefficients representing spherical aberration (B), astigmatism (C), field curvature (D), radial distortion (E), and coma (F). [5]
The Seidel model describes the arrival point of the aberrated ray on the focused-image plane (the conjugate plane of the object plane), which is not necessarily the plane of the image sensor. In order to take defocus into account, the Gaussian thick-lens geometric optics model with offset apertures can be used to follow the path of a ray traveling from the object at point P 0 to the image sensor at point P Img in Fig. 2 . In this model, the lens is represented by a pair of equivalent refractive planes, which are separated along the optical axis by a distance T Lens .
PPT Slide
Lager Image
The thick-lens ray tracing model with defocus consideration.
In the common case of a lens containing many optical elements, the thick-lens model with offset apertures is an approximation to the overall effect of the lens. Therefore T Lens , Z Enter , and Z Exit may be the effective thickness and offsets of the thick-lens approximation and not the actual lens thickness or the locations of the physical lens pupils [16 , 20] .
M is the magnification from entrance to exit pupils. Z 1 and Z 2 are the distances of the focused-image and the image planes, respectively, to the exit pupil of the system, and P 1 * is the nonaberrated arrival point of the ray on the focused image plane. These simplifying parameters are computed from the thick-lens model as Eq. (1) - (3), these parameters are the base of PSF model formation.
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
- 3.2. Skew Normal PSF Model
The Hu moment has a clear analogy to the moments of a joint bivariate probability distribution function. Following this analogy, the Hu moment is extended to the concept of a normalized central moment μ’ pq . Calculating this integral for p, q∈{0; 1; 2; 3}, the normalized second- and third-order central moments can be solved as functions of the aberration coefficients, the aperture radius, the focus and object distances, and the location in the image plane [Eq. (4) - (10)].
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
where the constants S 1 ,…,S 10 are introduced to make the central moments polynomial functions of the point P 1 * (x 1 * ,y 1 * ) across the focused image plane, and the constants are determined from the optical parameters of the camera as Eq. (11) - (20).
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
Because of the characteristic of the normalized central moment, the central moment and Seidel coefficients can be united in PSF model construction. The proposed model of S 1 - S 10 takes a set of predicted Hu moments and produces the corresponding estimate of the surface of the instantaneous PSF. There are two candidate models for the instantaneous PSF that will be evaluated in this article: the elliptical Gaussian model and the bivariate skew-normal distribution. The elliptical Gaussian model is specified entirely by the second-order central moments [8] .
PPT Slide
Lager Image
Unfortunately, the Gaussian PSF model cannot represent skew (it requires all four of the third-order moments to be zero). This is a major shortcoming of the Gaussian model, because with a lens that exhibits coma (represented by Seidel aberration coefficient “F”), the resulting lens-effect PSF can display significant skew. Interestingly, it is shown in Eq. (4) - (10) and (11) - (20) that if the coma coefficient F goes to zero, all third-order moments also go to zero, regardless of the values of the other four Seidel aberration coefficients. To allow for nonzero skew, we select the bivariate skew-normal distribution model shown in Eq. (22)
PPT Slide
Lager Image
where γ 1 =i/k 1 and γ 1 =j/k 2 . The skew-normal model has five parameters (κ 1 , κ 2 , ω, α 1 , and α 2 ), which affect the shape of the PSF surface, and a normalizing parameter λ SN . If α 1 and α 2 are chosen to be zero, this model reduces to the Gaussian model.
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
PPT Slide
Lager Image
IV. DEFOCUSED INFRARED TELESCOPE
- 4.1. Results of Focused Infrared Telescope
To test the proposed method, the PSF model is estimated with a real infrared telescope optical system. An infrared optical system with four spherical lenses is selected whose basic specification is listed in Table 1 [22 - 24] . There is no reflector in this camera, and every refracted surface figure is spherical. The glass material, silicon and zinc selenide are very common optical materials in an infrared telescope.
Parameters of infrared telescope optical system
PPT Slide
Lager Image
Parameters of infrared telescope optical system
Figure 3 shows the 2D ray tracing model and 3D construction of the infrared system.
PPT Slide
Lager Image
Structure of infrared optical system: (a) 2D model, (b) 3D model.
First, we estimate the PSF model from the focused infrared camera. The standard PSF model is generated by optical design software Code V, shown in Fig. 4 . There is no optical blur in the focused situation. Visual results indicate that both of the blur kernels are very small without optical blur shape.
PPT Slide
Lager Image
PSF model of focused infrared optical system: (a) simulated PSF by proposed method, (b) PSF generated by Code V.
A correlation coefficient is used as a metric to evaluate the results of PSF models. Corr is the computation of the correlation coefficient of matrix A and matrix B. The computation algorithm of Corr is shown in Eq. (29).
PPT Slide
Lager Image
where
PPT Slide
Lager Image
is the average value of matrix A, and similarly for parameter
PPT Slide
Lager Image
and matrix B. When the correlation coefficient value is close to 1, results indicate that there is a positive linear relationship between the two matrices [11] .
Compared with the simulated PSF and standard PSF from Code V, Fig. 5 shows the correlation coefficient distribution with different field angles, and the average correlation coefficient is 0.8003.
PPT Slide
Lager Image
Proposed PSF model correlation coefficient distribution of focused system.
- 4.2. Results of Defocused Infrared Telescope
In this optical system, 0.2 mm defocused distance from perfect image plane is adjusted to estimate the PSF model in defocused condition. Results are shown in Fig. 6 . There is symmetrical optical blur without coma component in any field angle.
PPT Slide
Lager Image
PSF model of defocused infrared optical system: (a) simulated PSF by proposed method, (b) PSF generated by Code V.
In defocused condition, comparing the simulated PSF and standard PSF, Fig. 7 shows the correlation coefficient distribution with different field angles, and the average correlation coefficient is 0.8479. All the correlation coefficients are higher than 0.75 in focused and defocused systems. Results show that this method can accurately estimate PSF in a defocused optical system.
PPT Slide
Lager Image
Proposed PSF model correlation coefficient distribution of defocused system.
V. SINGLE LENS CAMERA WITH OPTICAL ABERRATION
To test the optical aberration characteristic, a single lens optical system designed for digital camera or mobile phone camera is selected. To reduce the system length, there is only one single spherical lens in the optical system. However, correction of coma, astigmatism, field curvature, distortion and chromatic aberration must rely on stacking up additional lens elements. Spherical aberration can be avoidable by using an aspheric lens element. Because an imaging system using this lens can be manufactured massively at a very low cost, and its inherent defect produces very strong optical aberrations, it is a perfect optical system to test the spatially varying PSF model for optical aberration. Figure 8 shows the 2D ray tracing model and 3D construction of the one spherical lens optical system of the digital camera. All of the parameters of the single lens optical system are given in Table 2 .
PPT Slide
Lager Image
Structure of single lens optical system: (a) 2D model, (b) 3D model.
Parameters of single lens camera optical system
PPT Slide
Lager Image
Parameters of single lens camera optical system
In Fig. 9 , Cartesian PSFs model generated by Code V are segmented into several regions with different field angles 0° to 10°. PSF models in the Cartesian domain vary from region to region, not only in size, but also in shape and direction, resulting in a spatially variant distribution. Fig. 10 shows the shape and direction comparison of the simulated PSF surface with standard PSF from Code V.
PPT Slide
Lager Image
PSF contour of single lens generated by Code V: (a) field angle 0°, (b) field angle 2°, (c) field angle 4°, (d) field angle 6°, (e) field angle 8°, (f) field angle 10°.
PPT Slide
Lager Image
PSF model of single lens: (a) simulated PSF by proposed method, (b) PSF generated by Code V.
In Fig. 9 and Fig. 10 , diffraction phenomenon can be simulated from optical software Code V and Zemax. In classical physics, the diffraction phenomenon is described as the interference of waves according to the Huygens-Fresnel principle. In the proposed method, the PSF model is constructed from Seidel aberration coefficients based on the Taylor series of Snell’s law. The simulated PSF model cannot present diffraction phenomenon due to the difference of geometrical optics principles and physical optics principles.
As mentioned above, the correlation coefficient is used as a metric to evaluate the results of aberration PSF. The average value is higher than 0.9, the distribution is shown in Fig. 11 . As part IV proved, the correction coefficient of Fig. 11 goes higher as the field angle get larger, because the outer field is more defocused. The results show that the proposed model can provide good accuracy of the blur kernel model for optical aberration and for a defocused system.
PPT Slide
Lager Image
Proposed PSF model correlation coefficient distribution of aberration.
At last, in order to verify the truth of constructed PSF model, image restoration by different numbers of segmented regions was implemented. The blurred polar images were segmented into different numbers of locally invariant regions and restored by applying different numbers of polar PSFs [19] . The image restoration effects on an apartment building, car and observing tower are shown in Fig. 12 , captured by 5D mark III with EF 100 mm f/2 USM.
PPT Slide
Lager Image
Image restoration effect: (a) original image of apartment building, (b) blurred image of apartment building, (c) restored image of apartment building, (d) original image of car, (e) blurred image of car, (f) restored image of car, (g) original image of tower, (h) blurred image of tower, (i) restored image of tower.
VI. CONCLUSION
Motivated from an established theoretical framework in physics, we conclude that the proposed spatially varying model is able to accurately describe the second- and third-order Hu moments of a set of spatially varying PSF across the image plane. The model has been validated by means of the simulation results of an optical system with defocus and optical aberration
There are two imperfect characteristics to the proposed method. First, when any essential parameter of the optical system is inaccurate, a big deviation error will probably appear in PSF model formation. Estimating blur kernel based on blur image can make up this deficiency and increase the robustness. Second, because of the difference of geometrical optics and physical optics, the simulated PSF model cannot present diffraction phenomenon. In the future we will try to integrate these two principles to provide a more accurate PSF model.
Acknowledgements
This research is supported by the Arming Research Foundation under Grant NO. 51301060207.
References
Simpkins D. J. , Stevenson R. L. 2014 “Parameterized modeling of spatially varying optical blur,” Journal of Electronic Imaging 23 013005 -
Born M. , Wolf E. 1999 Principles of Optics Cambridge University Press New York, USA
Joshi N. , Szeliski R. , Kriegman D. 2008 “PSF estimation using sharp edge prediction,” Proc. IEEE Conf. Computer Vision and Pattern Recognition Anchorage, Alaska 24 - 26
Brauers J. , Seiler C. , Aach T. 2010 “Direct PSF estimation using a random noise target,” Proc. SPIE 7537 75370B -
Wang J. , Guo B. H. , Sun Q. , Lu Z. W. 2012 “Third-order aberration fields of pupil decentered optical systems,” Opt. Express 20 11652 - 11658
Kee E. , Paris S. , Chen S. , Wang J. 2011 “Modeling and removing spatially-varying optical blur,” Proc. IEEE Int. Conf. Computational Photography Anchorage, Alaska 11962826 -
Simpkins J. 2011 “Modeling and estimation of spatially-varying point spread functions due to lens aberrations and defocus,” Master’s Thesis University of Notre Dame
Hu M. 1962 “Visual pattern recognition by moment invariants,” IRE Trans. Inf. Theor. 8 179 - 187
Adur J. , Vicente N. , Zamboni J. D. , Izaguirre M. F. , Casco V. H. 2011 “A quantitative study of the quality of deconvolved wide-field microscopy images as function of empirical three-dimensional point spread functions,” J. Opt. Soc. Korea 15 142 - 147
Simpkins D. J. , Stevenson R. L. 2014 “A novel error metric for parametric fitting of point spread functions,” Proc. SPIE 9029 90290 - 90296
Chen L. , Lei J. , Romero J. 2014 “Quantum digital spiral imaging,” Light: Science & Applications 3 e153:1 - 4
Grossberg M. , Nayar S. 2004 “Modeling the space of camera response functions,” IEEE Transactions on Pattern Analysis and Machine Intelligence 26 1272 - 1282
Simpkins J. , Stevenson R. 2013 “A spatially varying PSF model for Seidel aberrations and defocus,” IS&T/SPIE Electronic Imaging 8666 86660F -
Simpkins D. J. , Stevenson R. L. 2012 “Robust grid registration for non-blind PSF estimation,” [IS&T/SPIE Electronic Imaging: Visual Information Processing and Communication III] Burlingame, California, USA 8305 83050 - 83058
Thompson K. 2005 “Description of the third-order optical aberrations of near-circular pupil optical systems without symmetry,” J. Opt. Soc. Am. A 22 11652 -
Aggarwal M. , Ahuja N. 2001 “A new imaging model,” International Conference on Computer Vision 82 - 89
Witte S. , Tenner V. T. , Noom D. W. E. , Eikema K. S. E. 2014 “Lensless diffractive imaging with ultra-broadband table-top sources: From infrared to extreme-ultraviolet wavelengths,” Light: Science & Applications 3 e163;1 - 8
Jiang Z. H. , Chao C. W. , Wei Q. 2012 “Wind tunnel experiment of supersonic conformal dome and its optical and structure characteristics,” Optics and Precision Engineering 20 1999 - 2005
Zhang Y. P. , Minema I. , Zimin L. G. , Ueda T. 2011 “Analysis of radially restored images for spherical single lens cellphone camera,” IEEE Sensors Journal 11 2834 - 2844
Booth M. J. 2013 “Adaptive optical microscopy: The ongoing quest for a perfect image,” Light: Science & Applications 3 e165 -
Jiang H. H. , Jia H. G. , Wei Q. 2012 “Analysis of zenith pass problem and tracking strategy design for roll-pitch seeker,” Aerospace Science and Technology 23 345 - 351
Jin H. , Lim J. , Kim Y. , Kim S. 2013 “Optical design of a reflecting telescope for CubeSat,” J. Opt. Soc. Korea 17 142 - 147
Wang C. , Jiang H. H. , Zhu R. F. , Yu L. Y. 2012 “Boresight error analysis and correction of conformal infrared seeker,” Acta Optica Sinica 33 0912001 -
Wang C. , Chen J. , Wei Q. , Yu L. Y. 2014 “Thermal noise analysis and correction of infrared conformal dome,” Acta Optica Sinica 34 0622001 -
Yu L. , Jia H. , Wei Q. 2013 “A design of mid-wave infrared integral catadioptric optical system with wide FOV,” J. Opt. Soc. Korea 17 142 - 147