Advanced
Scene-based Nonuniformity Correction Algorithm Based on Temporal Median Filter
Scene-based Nonuniformity Correction Algorithm Based on Temporal Median Filter
Journal of the Optical Society of Korea. 2013. Jun, 17(3): 255-261
Copyright ©2013, Optical Society of Korea
  • Received : January 01, 2013
  • Accepted : April 04, 2013
  • Published : June 25, 2013
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Lixiang Geng
School of Electronic Engineering and Optoelectronic Technology, NUST, Nanjing 210094, China, Key Laboratory of Photoelectronic Imaging Technology and System, Beijing Institute of Technology, Ministry of Education of China, Beijing 100081, China
genglixiang@gmail.com
Qian Chen
School of Electronic Engineering and Optoelectronic Technology, NUST, Nanjing 210094, China, Key Laboratory of Photoelectronic Imaging Technology and System, Beijing Institute of Technology, Ministry of Education of China, Beijing 100081, China
Weixian Qian
School of Electronic Engineering and Optoelectronic Technology, NUST, Nanjing 210094, China, Key Laboratory of Photoelectronic Imaging Technology and System, Beijing Institute of Technology, Ministry of Education of China, Beijing 100081, China
Yuzhen Zhang
Jiangsu Key of Spectral Imaging & Intelligent Sense, NUST, Nanjing 210094, China
Abstract
Scene-based nonuniformity correction techniques for infrared focal-plane arrays have been widely considered as a key technology, and various algorithms have been proposed to compensate for fixed-pattern noise. However, the existed algorithms’ capability is always restricted by the problems of convergence speed and ghosting artifacts. In this paper, an effective scene-based nonuniformity correction method is proposed to solve these problems. The algorithm is an improvement over the constant statistics method and a temporal median is utilized with the Gaussian kernel to estimate the nonuniformity parameters. Also theoretical analysis is conducted to demonstrate that effective ghosting artifacts elimination and superior convergence speed can be obtained with the proposed method. Finally, the performance of the proposed technique is tested with infrared image sequences with simulated nonuniformity and with infrared imagery with real nonuniformity. The results show the proposed method is able to estimate each detector’s gain and to offset reliably and that it performs better in increasing convergence speed and reducing ghosting artifacts compared with the conventional techniques.
Keywords
I. INTRODUCTION
Nowadays,the infrared imaging system is based on infrared focal-plane array (IRFPA) technology. However, the system has an inherent nonuniformity problem in that that the detectors in the array have different responses to the same input signal, causing the observed images to be degraded by fixed-pattern noise (FPN). In this case, many nonuniformity correction (NUC) techniques have been proposed to compensate for FPN. The NUC techniques include two classes that are reference-based and scene-based. Evidently, the scene-based methods are more attractive because the reference-based ones need to employ blackbody sources as uniform infrared targets while scene-based methods are conveniently updating the correction coefficients and avoiding scene interruptions.
So far, numerous scene-based nonuniformity correction (SBNUC) methods have been proposed. Assuming that each detector’s irradiance in the array has the same mean and variance, Y. M. Chiang and J. G. Harris proposed constant statistics (CS) methods [1 , 2] . References [2 , 3] aimed to increase convergence speed and reduce ghosting. S. N. Torres and M. M. Hayat presented a Kalman filter based method by a discrete-time Gauss-Markov process [4] . Another adaptive SBNUC technique is based on a neural network (NN) which is described in [5] . S. N. Torres, E. M. Vera improved the speed of the method by modifying the learning rate in the gain and offset parameter estimation process [6] . However, it is proved in reference [2] that the CS algorithm with appropriate coefficient has faster convergence rate and better performance than the LMS algorithm. Registration methods are proposed in [7 , 8] . But they all need complex registration algorithms and cannot work well when the nonuniformity is serious.
This paper puts forward a novel SBNUC algorithm called the med-CS method. This method is based on an improvement of the CS method. Due to the small influence of the median filter by an infrequent artifact and to the efficient capability for smoothing of impulsive-type noise, the proposed method utilizes the characteristics of median filter and applies it in the CS method to estimate the nonuniformity parameters. In the proposed algorithm, a temporal median is adopted combining with the Gaussian kernel. In this way, improvement in removal of ghosting artifacts and convergence speed can be achieved and the thorough analysis of effects on these two aspects is launched.
The remainder of the paper is organized as follows. In Section II the detector-level model and the readout amplifier model are given. Then the proposed algorithm is described and the superiority of the proposed algorithm is demonstrated. In Section III results obtained from simulated and real infrared data are presented. Finally, some conclusions are given in Section IV.
II. NONUNIFORMITY CORRECTION METHODS
- 2.1. Observation Model and Conventional CS-NUC Method
The relationship between the observed infrared signal response and the real world is generally nonlinear. In literature [9] ,the output pixel value of each detector is modeled nonlinearly as a high order polynomial. To simplify the problem formulation for SBNUC, the linear approximation model is adopted which contains the first-order of the polynomial. Then the output of a single pixel is given by:
PPT Slide
Lager Image
where, n is the frame index and a ( n ) and b ( n ) are respectively the real gain and offset of the detector. And x ( n ) stands for the real incident infrared signal collected by the respective detector. Without losing generality, x ( n ) can be assumed to have fixed mean and unity mean deviation. The expressions are now written as:
PPT Slide
Lager Image
Where, m is the temporal mean and s is the deviation of the observed signal. Then the aim of the NUC is transferred to estimate the gain s and the bias m . So the FPN can be corrected as:
PPT Slide
Lager Image
To estimate the gain and bias, the CS algorithm uses the mean of the measured output y ( n ), based on assumption that the temporal mean and standard deviation of each pixel is constant over time and space. Mean deviation is used instead of standard deviation to simplify the implementation and provide more robust estimation. Mean and mean deviation of y are calculated by the following recursive equations:
PPT Slide
Lager Image
CS is improved to better estimate changes of gain and offset with a fixed-length rectangular and exponentially shaped window.
PPT Slide
Lager Image
It can be seen from (5) that coefficient α controls the size of the window and it should be less than 1 but usually very close to 1.
- 2.2. The Proposed Med-CS NUC Method
The main drawback of the CS NUC method originates in the basic constraint that is related to the supposition that the scene is constantly moving. However, when the scene is not moving for a short while, the ghosting effect appears and the convergence rate will be affected. In this section, the proposed method takes a temporal median value with the Gaussian kernel and the ability of the method is analyzed.
- 2.2.1. Parameters estimation
In the med-CS NUC method, the parameters for each pixel are calculated by computing weighted means and variances based on medians from the values with length of L frames. And the form of the weighted means expression is similar to the bilateral filter in [10] as follow
PPT Slide
Lager Image
The weighted variance is expressed as:
PPT Slide
Lager Image
The weights are computed from a Gaussian distribution centered at ymed
PPT Slide
Lager Image
where σ is the standard deviation parameter defining the Gaussian kernel. It can be observed that the weights depend on differences in intensity value between yn ( i , j ) and ymed ( i , j ). The ymed ( i , j ) is the median value of the pixel ( i , j ) over L frames
PPT Slide
Lager Image
where the weights ωk ( i , j ) for each pixel location are used to minimize the effect of outliers (values far from the median ymed ( i , j )). The weight is controlled by σ which controls the decay of the weight factor as the range parameter σ varies.
- 2.2.2. Analysis of convergence
An assumption that the parameters of the nonuniformity are constant is employed, which means the offsets of sensors are considered to be fixed over time. According to the derivation in the literature [3] the mean-square error ( MSE ) of the recursive nonuniformity parameters estimation with the conventional CS method can be written as
PPT Slide
Lager Image
where σy is the standard deviation of y ( n ). To explain intuitionisticly, ΔMSE ( n ) is defined as the difference of MSE between the adjacent frames as follow
PPT Slide
Lager Image
From (11), it can be seen that if σy ( n ) is equal to σy ( n -1), the MSE of the conventional CS-method decays as n goes to infinity and ΔMSE ( n ) shrinks to zero with recursive coefficient α2 . However, the actual scene consists of multiple parts with different deviations. So the ΔMSE will oscillate when σy changes and the convergence will be influenced. In this section, the superiority of the med-CS method in estimating the average of scene which contains different deviations is demonstrated.
The CS method is based on the assumption that every pixel has the same distribution and the temporal mean and standard deviation of each pixel is constant over time and space. But the distribution tendency characteristic of each pixel over time has not been considered. The mean value that the CS method adopted is an appropriate measure of the central tendency. For instance the measurements of each pixel in a period of time has a Gaussian distribution. In fact, the temporal statistics of each pixel approximate mixture distributions rather than following a single distribution strictly. The mixture distribution contains more than two distributions. Regularly, there is more than three times difference among the deviations of the different distributions. Consequently, the deviation of the measurements cannot be constant over time and it will bring the problem according to (11) which was analyzed previously. To solve this problem a median value is utilized and proved to be effective by Monte Carlo simulation.
The median method, however, should have more perfect performance and this hypothesis is verified using Monte Carlo simulation. For this study, several groups of statistics were examined. Without loss of generality, an assumption is employed that the statistics consist of two Gaussian distributions with different deviations.
PPT Slide
Lager Image
The simulation consists of 10,000 samples and each sample contains N values, where N is set to 50 and 500. As in (12), each sample is made up of two kinds of different distribution values whose deviations are σ and ζσ . The coefficient ζ is set to 3, 5 and 10 and the proportion coefficient λ is set from 0% to 100% with interval of 10%. The mean and median of samples are calculated and their mean-square errors ( MSE ) are calculated respectively by:
PPT Slide
Lager Image
The result of the simulation is showed in Table 1 , Table 2 . Table 1 is generated by 10,000 samples with 50 values per sample, Table 2 is generated by 10,000 samples with 500 values per sample. The curves in Fig. 2 (a) are drawn with the data in Table 1 and Fig. 2 (b) is drawn with the data in Table 2 .
The results display that the MSE of the mean method performs significantly worse than the median method especially when distributions are various. The proportion coefficient λ expresses the diversity of the sample distribution. The sample distribution is unitary when the coefficient λ is approaching 0% or 100% and various when the coefficient λ is approaching 50%. The mean method performs well when the distribution is unitary but the median method has the obvious advantage when the distribution is various like the behavior of the blue line while λ is approaching 50%.
Simultaneously, the coefficient ζ indicates the degree of the difference between the distributions. When the coefficient ζ is from 3 to 10 the difference between the two kinds of MSE becomes larger, the greater the difference of the distributions’ deviations changes, the more outstanding the median method performs.
Finally, comparing Table 1 to Table 2 or comparing Fig. 2 (a) to Fig. 2 (b), it shows that the more sufficient the amount of data is, the maore superior the MSE is. However, the general statistical characteristic of the data is constant.
From the above analysis the median method has a remarkable effect on eliminating interference of deviation variation. The median value is an accepted measure of the various tendencies
MSE of mean and median from Monte Carlo simulation for various samples (50 values per sample) from the different deviations observed distributions
PPT Slide
Lager Image
MSE of mean and median from Monte Carlo simulation for various samples (50 values per sample) from the different deviations observed distributions
MSE of mean and median from Monte Carlo simulation for various samples (500 values per sample) from the different deviations observed distributions
PPT Slide
Lager Image
MSE of mean and median from Monte Carlo simulation for various samples (500 values per sample) from the different deviations observed distributions
PPT Slide
Lager Image
(a) A simulated sequence of scene signals of single pixel for 500 frames, (b) The results of estimating the simulated signals’ average and the green curve indicates the conventional CS method, the blue curve indicates the med-CS method.
PPT Slide
Lager Image
MSE of mean and median from Monte Carlo simulation for various samples from the different deviations observed distributions. And (a) is corresponding to the Table 1, (b) is corresponding to the Table 2.
of a group of values even when their distribution is non- Gaussian and the median method is valid even when applied to a small number of samples. Moreover the median method is less sensitive to the variation of sample distribution. Therefore, the med-CS using a temporal median is more robust and suitable to convergence than the conventional CS method.
III. RESULTS AND DISCUSSION
In this section, the proposed med-CS method is applied to both simulated and real data to correct the nonuniformity compared with the various SBNUC algorithms and the method’s capability is verified.
- 3.1. Applications to Simulated Nonuniformity
Several tests have been executed over an IR sequence of frames with simulated nonuniformity in this work. The deghosting capability has been demonstrated by visually comparing the frames obtained with the original CS method [1] and the improved CS method in [2] . Here, we call the method in [2] the gated-CS method because the algorithm sets a gated threshold to deghost.
As an evaluation criterion of correction performance, the Root Mean Square Error ( RMSE ) [8] is used to measure the difference between the true infrared image and the corrected image. Smaller values for the RMSE indicate better performance. The RMSE is calculated by:
PPT Slide
Lager Image
where X ( i , j ) is the ( i , j ) th pixel’s value of the true frame, while
PPT Slide
Lager Image
is the pixel’s value of the corrected frame and M × N is the number of detectors in the FPA.
The corrupted sequence with artificial nonuniformity is generated from a clear frame infrared video sequence with Simulated Nonuniformity added. The added FPN is composed of a synthetic gain with a unit-mean Gaussian distribution with standard deviation of 0.2, and a synthetic offset with a zero-mean Gaussian distribution with standard deviation of 15.
The CS method uses an exponential window parameter of α = 0.99 ( N ≈100). The gated-CS method uses the same α and a change threshold of T = 20 and the med-CS method uses the same frame length L = 100. The results in Fig. 3 are very typical of the numerous video sequences tested. In the first 200 frames the algorithms converge and then the RMSE curves oscillate because of ghosting artifacts. The curves in Fig. 3 show that the med-CS method significantly outperforms the original CS method and the gated-CS method and that the oscillation of the med-CS method is notably weaker than the other methods. Although the gated-CS performs well when the sequences have strong object higher than the threshold but burns in image with original CS method as in 350th frame, the med-CS gets the best result in removing ghosting artifacts when the burn-in object is lower than the change threshold of the gated-CS method as in the 630th and 780th frame.
PPT Slide
Lager Image
The RMSE curves of the three SBNUC algorithms. The green curve indicates the conventional CS method, the blue curve indicates the gated-CS method and the red curve indicates the med-CS method.
PPT Slide
Lager Image
The result of simulated nonuniformity correction (780th frame) with the three SBNUC algorithms, (a) the clear infrared image added with simulated nonuniformity, (b) corrected by the CS method, (c) corrected by the gated-CS method, (d) corrected by the med-CS method.
Figure 4 shows the images for the 780th frame. Fig. 4 (a) shows the true clear infrared image corrupted with simulated nonuniformity. The NUC results of original CS method, gated-CS and med-CS algorithms are shown in Fig. 4 (b) - (d). It is shown that the med-CS algorithm effectively generates much fewer ghosting artifacts than the other techniques.
- 3.2. Applications to Real Infrared Data
In this section, the original CS, gated-CS, and med-CS are applied to a noisy sequence collected by using a 320 × 256 HgCdTe IRFPA camera, operating in the 8 to 14 μ m range. The video contains 2000 frames captured at a rate of 25 fps .
PPT Slide
Lager Image
The result of true nonuniformity correction (379th frame) with the three SBNUC algorithms, (a) the clean infrared image with real FPN, (b) corrected by the CS method, (c) corrected by the gated-CS method, (d) corrected by the med-CS method.
The coefficients used in the experiment are: the original CS method uses an exponential window parameter of α = 0.992 ( N ≈125). The gated-CS method uses the same α and a change threshold of T = 25 and the med-CS method uses the same frame length L = 125.
To indicate good correction performance or examine whether there is the presence of artifacts, a visual evaluation is performed by watching a video sequence. Fig. 5 shows the original image and the corrected images using the original CS, gated-CS, and med-CS algorithm of the 379th frame. It is very noticeable that the med-CS compensates the FPN and performs the best over the sequence. Besides, it effectively generates much fewer ghosting artifacts than the other techniques.
IV. CONCLUSION
In this paper, based on the analysis of property of scene statistical distribution, a new NUC method called med-CS has been proposed, which adopts a temporal median filter combining with Gaussian kernel to estimate the NUC parameters. Taking advantage of the median filter’s characteristics properly, the proposed algorithm has the ability to improve the nonuniformity correction and eliminate ghosting artifacts efficiently. Experiments carried out with simulated nonuniformity and the real infrared data have shown that the proposed method offers the best performance compared with the conventional methods tested.
Acknowledgements
This study was supported by a grant from National Natural Science Foundation of China (61101199) and Natural Science Foundation of Jiangsu Province (BK2011699).
References
Harris J. G. , Chiang Y. M. (1997) “Nonuniformity correction using constant statistics constraint: analog and digital implementations” Proc. SPIE 3061 895 - 905
Harris J. G. , Chiang Y. M. (1998) “Minimizing the ghosting artifact in scene-based nonuniformity correction” Proc. SPIE 3377 106 - 113
Qian W. , Chen Q. , Bai J. , Gu G. (2011) “Adaptive convergence nonuniformity correction algorithm” Appl. Opt. 50 1 - 10
Vera E. , Torres S. (2005) “Fast adaptive nonuniformity correction for infrared focal-plane array detectors” EURASIP J. Appl. Signal Process 13 106 - 117
Scribner D. A. , Sarkady K. A. , Caulfield J. T. , Kruer M. R. , Katz G. , Gridly C. J. (1990) “Nonuniformity correction for staring JR focal plane arrays using scene-based techniques” Proc. SPIE 1308 224 -
Torres S. N. , Vera E. M. , Reeves R. A. , Sobarzo S. K. (2003) “Adaptive scene-based non-uniformity correction method for infrared-focal plane arrays” Proc. SPIE 5076 130 - 139
Hardie R. C. , Hayat M. M. , Armstrong E. E. , Yasuda B. (2000) “Scene-based nonuniformity correction using video sequences and registration” Appl. Opt. 39 1241 - 1250
Zuo C. , Chen Q. , Gu G. , Sui X. (2011) “Scene-based nonuniformity correction algorithm based on interframe registration” J. Opt. Soc. Am. A 28 1164 - 1176
Narayanan B. , Hardie R. C. , Muse R. A. (2005) “Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture” Appl. Opt. 44 3482 - 3491
Zuo C. , Chen Q. , Gu G. , Qian W. (2011) “New temporal high-pass filter nonuniformity correction based on bilateral filter” Optical Review 18 197 - 202