Scene-based Nonuniformity Correction Algorithm Based on Temporal Median Filter

Journal of the Optical Society of Korea.
2013.
Jun,
17(3):
255-261

- Received : January 16, 2013
- Accepted : April 16, 2013
- Published : June 25, 2013

Download

PDF

e-PUB

PubReader

PPT

Export by style

Share

Article

Metrics

Cited by

TagCloud

Scene-based nonuniformity correction techniques for infrared focal-plane arrays have been widely considered as a key technology, and various algorithms have been proposed to compensate for fixed-pattern noise. However, the existed algorithms’ capability is always restricted by the problems of convergence speed and ghosting artifacts. In this paper, an effective scene-based nonuniformity correction method is proposed to solve these problems. The algorithm is an improvement over the constant statistics method and a temporal median is utilized with the Gaussian kernel to estimate the nonuniformity parameters. Also theoretical analysis is conducted to demonstrate that effective ghosting artifacts elimination and superior convergence speed can be obtained with the proposed method. Finally, the performance of the proposed technique is tested with infrared image sequences with simulated nonuniformity and with infrared imagery with real nonuniformity. The results show the proposed method is able to estimate each detector’s gain and to offset reliably and that it performs better in increasing convergence speed and reducing ghosting artifacts compared with the conventional techniques.
where,
n
is the frame index and
a
(
n
) and
b
(
n
) are respectively the real gain and offset of the detector. And
x
(
n
) stands for the real incident infrared signal collected by the respective detector. Without losing generality,
x
(
n
) can be assumed to have fixed mean and unity mean deviation. The expressions are now written as:
Where,
m
is the temporal mean and
s
is the deviation of the observed signal. Then the aim of the NUC is transferred to estimate the gain
s
and the bias
m
. So the FPN can be corrected as:
To estimate the gain and bias, the CS algorithm uses the mean of the measured output
y
(
n
), based on assumption that the temporal mean and standard deviation of each pixel is constant over time and space. Mean deviation is used instead of standard deviation to simplify the implementation and provide more robust estimation. Mean and mean deviation of
y
are calculated by the following recursive equations:
CS is improved to better estimate changes of gain and offset with a fixed-length rectangular and exponentially shaped window.
It can be seen from (5) that coefficient
α
controls the size of the window and it should be less than 1 but usually very close to 1.
L
frames. And the form of the weighted means expression is similar to the bilateral filter in
[10]
as follow
The weighted variance is expressed as:
The weights are computed from a Gaussian distribution centered at
y_{med}
where
σ
is the standard deviation parameter defining the Gaussian kernel. It can be observed that the weights depend on differences in intensity value between
y_{n}
(
i
,
j
) and
y_{med}
(
i
,
j
). The
y_{med}
(
i
,
j
) is the median value of the pixel (
i
,
j
) over
L
frames
where the weights
ω_{k}
(
i
,
j
) for each pixel location are used to minimize the effect of outliers (values far from the median
y_{med}
(
i
,
j
)). The weight is controlled by
σ
which controls the decay of the weight factor as the range parameter
σ
varies.
MSE
) of the recursive nonuniformity parameters estimation with the conventional CS method can be written as
where
σ_{y}
is the standard deviation of
y
(
n
). To explain intuitionisticly,
ΔMSE
(
n
) is defined as the difference of
MSE
between the adjacent frames as follow
From (11), it can be seen that if
σ_{y}
(
n
) is equal to
σ_{y}
(
n
-1), the
MSE
of the conventional CS-method decays as
n
goes to infinity and
ΔMSE
(
n
) shrinks to zero with recursive coefficient
α^{2}
. However, the actual scene consists of multiple parts with different deviations. So the
ΔMSE
will oscillate when
σ_{y}
changes and the convergence will be influenced. In this section, the superiority of the med-CS method in estimating the average of scene which contains different deviations is demonstrated.
The CS method is based on the assumption that every pixel has the same distribution and the temporal mean and standard deviation of each pixel is constant over time and space. But the distribution tendency characteristic of each pixel over time has not been considered. The mean value that the CS method adopted is an appropriate measure of the central tendency. For instance the measurements of each pixel in a period of time has a Gaussian distribution. In fact, the temporal statistics of each pixel approximate mixture distributions rather than following a single distribution strictly. The mixture distribution contains more than two distributions. Regularly, there is more than three times difference among the deviations of the different distributions. Consequently, the deviation of the measurements cannot be constant over time and it will bring the problem according to (11) which was analyzed previously. To solve this problem a median value is utilized and proved to be effective by Monte Carlo simulation.
The median method, however, should have more perfect performance and this hypothesis is verified using Monte Carlo simulation. For this study, several groups of statistics were examined. Without loss of generality, an assumption is employed that the statistics consist of two Gaussian distributions with different deviations.
The simulation consists of 10,000 samples and each sample contains
N
values, where
N
is set to 50 and 500. As in (12), each sample is made up of two kinds of different distribution values whose deviations are
σ
and
ζσ
. The coefficient
ζ
is set to 3, 5 and 10 and the proportion coefficient
λ
is set from 0% to 100% with interval of 10%. The mean and median of samples are calculated and their mean-square errors (
MSE
) are calculated respectively by:
The result of the simulation is showed in
Table 1
,
Table 2
.
Table 1
is generated by 10,000 samples with 50 values per sample,
Table 2
is generated by 10,000 samples with 500 values per sample. The curves in
Fig. 2
(a) are drawn with the data in
Table 1
and
Fig. 2
(b) is drawn with the data in
Table 2
.
The results display that the
MSE
of the mean method performs significantly worse than the median method especially when distributions are various. The proportion coefficient
λ
expresses the diversity of the sample distribution. The sample distribution is unitary when the coefficient
λ
is approaching 0% or 100% and various when the coefficient
λ
is approaching 50%. The mean method performs well when the distribution is unitary but the median method has the obvious advantage when the distribution is various like the behavior of the blue line while
λ
is approaching 50%.
Simultaneously, the coefficient
ζ
indicates the degree of the difference between the distributions. When the coefficient
ζ
is from 3 to 10 the difference between the two kinds of
MSE
becomes larger, the greater the difference of the distributions’ deviations changes, the more outstanding the median method performs.
Finally, comparing
Table 1
to
Table 2
or comparing
Fig. 2
(a) to
Fig. 2
(b), it shows that the more sufficient the amount of data is, the maore superior the
MSE
is. However, the general statistical characteristic of the data is constant.
From the above analysis the median method has a remarkable effect on eliminating interference of deviation variation. The median value is an accepted measure of the various tendencies
MSE of mean and median from Monte Carlo simulation for various samples (50 values per sample) from the different deviations observed distributions
MSE of mean and median from Monte Carlo simulation for various samples (500 values per sample) from the different deviations observed distributions
(a) A simulated sequence of scene signals of single pixel for 500 frames, (b) The results of estimating the simulated signals’ average and the green curve indicates the conventional CS method, the blue curve indicates the med-CS method.
MSE of mean and median from Monte Carlo simulation for various samples from the different deviations observed distributions. And (a) is corresponding to the Table 1 , (b) is corresponding to the Table 2 .
of a group of values even when their distribution is non- Gaussian and the median method is valid even when applied to a small number of samples. Moreover the median method is less sensitive to the variation of sample distribution. Therefore, the med-CS using a temporal median is more robust and suitable to convergence than the conventional CS method.
RMSE
)
[8]
is used to measure the difference between the true infrared image and the corrected image. Smaller values for the
RMSE
indicate better performance. The
RMSE
is calculated by:
where
X
(
i
,
j
) is the (
i
,
j
) th pixel’s value of the true frame, while
is the pixel’s value of the corrected frame and
M
×
N
is the number of detectors in the FPA.
The corrupted sequence with artificial nonuniformity is generated from a clear frame infrared video sequence with Simulated Nonuniformity added. The added FPN is composed of a synthetic gain with a unit-mean Gaussian distribution with standard deviation of 0.2, and a synthetic offset with a zero-mean Gaussian distribution with standard deviation of 15.
The CS method uses an exponential window parameter of
α
= 0.99 (
N
≈100). The gated-CS method uses the same
α
and a change threshold of
T
= 20 and the med-CS method uses the same frame length
L
= 100. The results in
Fig. 3
are very typical of the numerous video sequences tested. In the first 200 frames the algorithms converge and then the
RMSE
curves oscillate because of ghosting artifacts. The curves in
Fig. 3
show that the med-CS method significantly outperforms the original CS method and the gated-CS method and that the oscillation of the med-CS method is notably weaker than the other methods. Although the gated-CS performs well when the sequences have strong object higher than the threshold but burns in image with original CS method as in 350th frame, the med-CS gets the best result in removing ghosting artifacts when the burn-in object is lower than the change threshold of the gated-CS method as in the 630th and 780th frame.
The RMSE curves of the three SBNUC algorithms. The green curve indicates the conventional CS method, the blue curve indicates the gated-CS method and the red curve indicates the med-CS method.
The result of simulated nonuniformity correction (780th frame) with the three SBNUC algorithms, (a) the clear infrared image added with simulated nonuniformity, (b) corrected by the CS method, (c) corrected by the gated-CS method, (d) corrected by the med-CS method.
Figure 4
shows the images for the 780th frame.
Fig. 4
(a) shows the true clear infrared image corrupted with simulated nonuniformity. The NUC results of original CS method, gated-CS and med-CS algorithms are shown in
Fig. 4
(b) - (d). It is shown that the med-CS algorithm effectively generates much fewer ghosting artifacts than the other techniques.
μ
m range. The video contains 2000 frames captured at a rate of 25
fps
.
The result of true nonuniformity correction (379th frame) with the three SBNUC algorithms, (a) the clean infrared image with real FPN, (b) corrected by the CS method, (c) corrected by the gated-CS method, (d) corrected by the med-CS method.
The coefficients used in the experiment are: the original CS method uses an exponential window parameter of
α
= 0.992 (
N
≈125). The gated-CS method uses the same
α
and a change threshold of
T
= 25 and the med-CS method uses the same frame length
L
= 125.
To indicate good correction performance or examine whether there is the presence of artifacts, a visual evaluation is performed by watching a video sequence.
Fig. 5
shows the original image and the corrected images using the original CS, gated-CS, and med-CS algorithm of the 379th frame. It is very noticeable that the med-CS compensates the FPN and performs the best over the sequence. Besides, it effectively generates much fewer ghosting artifacts than the other techniques.

I. INTRODUCTION

Nowadays，the infrared imaging system is based on infrared focal-plane array (IRFPA) technology. However, the system has an inherent nonuniformity problem in that that the detectors in the array have different responses to the same input signal, causing the observed images to be degraded by fixed-pattern noise (FPN). In this case, many nonuniformity correction (NUC) techniques have been proposed to compensate for FPN. The NUC techniques include two classes that are reference-based and scene-based. Evidently, the scene-based methods are more attractive because the reference-based ones need to employ blackbody sources as uniform infrared targets while scene-based methods are conveniently updating the correction coefficients and avoiding scene interruptions.
So far, numerous scene-based nonuniformity correction (SBNUC) methods have been proposed. Assuming that each detector’s irradiance in the array has the same mean and variance, Y. M. Chiang and J. G. Harris proposed constant statistics (CS) methods
[1
,
2]
. References
[2
,
3]
aimed to increase convergence speed and reduce ghosting. S. N. Torres and M. M. Hayat presented a Kalman filter based method by a discrete-time Gauss-Markov process
[4]
. Another adaptive SBNUC technique is based on a neural network (NN) which is described in
[5]
. S. N. Torres, E. M. Vera improved the speed of the method by modifying the learning rate in the gain and offset parameter estimation process
[6]
. However, it is proved in reference
[2]
that the CS algorithm with appropriate coefficient has faster convergence rate and better performance than the LMS algorithm. Registration methods are proposed in
[7
,
8]
. But they all need complex registration algorithms and cannot work well when the nonuniformity is serious.
This paper puts forward a novel SBNUC algorithm called the med-CS method. This method is based on an improvement of the CS method. Due to the small influence of the median filter by an infrequent artifact and to the efficient capability for smoothing of impulsive-type noise, the proposed method utilizes the characteristics of median filter and applies it in the CS method to estimate the nonuniformity parameters. In the proposed algorithm, a temporal median is adopted combining with the Gaussian kernel. In this way, improvement in removal of ghosting artifacts and convergence speed can be achieved and the thorough analysis of effects on these two aspects is launched.
The remainder of the paper is organized as follows. In Section II the detector-level model and the readout amplifier model are given. Then the proposed algorithm is described and the superiority of the proposed algorithm is demonstrated. In Section III results obtained from simulated and real infrared data are presented. Finally, some conclusions are given in Section IV.
II. NONUNIFORMITY CORRECTION METHODS

- 2.1. Observation Model and Conventional CS-NUC Method

The relationship between the observed infrared signal response and the real world is generally nonlinear. In literature
[9]
，the output pixel value of each detector is modeled nonlinearly as a high order polynomial. To simplify the problem formulation for SBNUC, the linear approximation model is adopted which contains the first-order of the polynomial. Then the output of a single pixel is given by:
PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

- 2.2. The Proposed Med-CS NUC Method

The main drawback of the CS NUC method originates in the basic constraint that is related to the supposition that the scene is constantly moving. However, when the scene is not moving for a short while, the ghosting effect appears and the convergence rate will be affected. In this section, the proposed method takes a temporal median value with the Gaussian kernel and the ability of the method is analyzed.
- 2.2.1. Parameters estimation

In the med-CS NUC method, the parameters for each pixel are calculated by computing weighted means and variances based on medians from the values with length of
PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

- 2.2.2. Analysis of convergence

An assumption that the parameters of the nonuniformity are constant is employed, which means the offsets of sensors are considered to be fixed over time. According to the derivation in the literature
[3]
the mean-square error (
PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

MSE of mean and median from Monte Carlo simulation for various samples (50 values per sample) from the different deviations observed distributions

PPT Slide

Lager Image

MSE of mean and median from Monte Carlo simulation for various samples (500 values per sample) from the different deviations observed distributions

PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

III. RESULTS AND DISCUSSION

In this section, the proposed med-CS method is applied to both simulated and real data to correct the nonuniformity compared with the various SBNUC algorithms and the method’s capability is verified.
- 3.1. Applications to Simulated Nonuniformity

Several tests have been executed over an IR sequence of frames with simulated nonuniformity in this work. The deghosting capability has been demonstrated by visually comparing the frames obtained with the original CS method
[1]
and the improved CS method in
[2]
. Here, we call the method in
[2]
the gated-CS method because the algorithm sets a gated threshold to deghost.
As an evaluation criterion of correction performance, the Root Mean Square Error (
PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

- 3.2. Applications to Real Infrared Data

In this section, the original CS, gated-CS, and med-CS are applied to a noisy sequence collected by using a 320 × 256 HgCdTe IRFPA camera, operating in the 8 to 14
PPT Slide

Lager Image

IV. CONCLUSION

In this paper, based on the analysis of property of scene statistical distribution, a new NUC method called med-CS has been proposed, which adopts a temporal median filter combining with Gaussian kernel to estimate the NUC parameters. Taking advantage of the median filter’s characteristics properly, the proposed algorithm has the ability to improve the nonuniformity correction and eliminate ghosting artifacts efficiently. Experiments carried out with simulated nonuniformity and the real infrared data have shown that the proposed method offers the best performance compared with the conventional methods tested.
Acknowledgements

This study was supported by a grant from National Natural Science Foundation of China (61101199) and Natural Science Foundation of Jiangsu Province (BK2011699).

Harris J. G.
,
Chiang Y. M.
(1997)
“Nonuniformity correction using constant statistics constraint: analog and digital implementations”
Proc. SPIE
3061
895 -
905

Harris J. G.
,
Chiang Y. M.
(1998)
“Minimizing the ghosting artifact in scene-based nonuniformity correction”
Proc. SPIE
3377
106 -
113

Vera E.
,
Torres S.
(2005)
“Fast adaptive nonuniformity correction for infrared focal-plane array detectors”
EURASIP J. Appl. Signal Process
13
106 -
117

Scribner D. A.
,
Sarkady K. A.
,
Caulfield J. T.
,
Kruer M. R.
,
Katz G.
,
Gridly C. J.
(1990)
“Nonuniformity correction for staring JR focal plane arrays using scene-based techniques”
Proc. SPIE
1308
224 -

Torres S. N.
,
Vera E. M.
,
Reeves R. A.
,
Sobarzo S. K.
(2003)
“Adaptive scene-based non-uniformity correction method for infrared-focal plane arrays”
Proc. SPIE
5076
130 -
139

Citing 'Scene-based Nonuniformity Correction Algorithm Based on Temporal Median Filter
'

@article{ E1OSAB_2013_v17n3_255}
,title={Scene-based Nonuniformity Correction Algorithm Based on Temporal Median Filter}
,volume={3}
, url={http://dx.doi.org/10.3807/JOSK.2013.17.3.255}, DOI={10.3807/JOSK.2013.17.3.255}
, number= {3}
, journal={Journal of the Optical Society of Korea}
, publisher={Optical Society of Korea}
, author={Geng, Lixiang
and
Chen, Qian
and
Qian, Weixian
and
Zhang, Yuzhen}
, year={2013}
, month={Jun}