Infrared Target Extraction Using Weighted Information Entropy and Adaptive Opening Filter
Infrared Target Extraction Using Weighted Information Entropy and Adaptive Opening Filter
ETRI Journal. 2015. Oct, 37(5): 1023-1031
Copyright © 2015, Electronics and Telecommunications Research Institute (ETRI)
  • Received : August 10, 2014
  • Accepted : July 30, 2015
  • Published : October 01, 2015
Export by style
Cited by
About the Authors
Tae Wuk Bae
Hwi Gang Kim
Young Choon Kim
Sang Ho Ahn

In infrared (IR) images, near targets have a transient distribution at the boundary region, as opposed to a steady one at the inner region. Based on this fact, this paper proposes a novel IR target extraction method that uses both a weighted information entropy (WIE) and an adaptive opening filter to extract near finely shaped targets in IR images. Firstly, the boundary region of a target is detected using a local variance WIE of an original image. Next, a coarse target region is estimated via a labeling process used on the boundary region of the target. From the estimated coarse target region, a fine target shape is extracted by means of an opening filter having an adaptive structure element. The size of the structure element is decided in accordance with the width information of the target boundary and mean WIE values of windows of varying size. Our experimental results show that the proposed method obtains a better extraction performance than existing algorithms.
I. Introduction
In an infrared (IR) image, object extraction plays an important role in automatic IR target recognition and tracking systems [1] [3] . However, because of noise and a low signal-to-noise ratio, it is a challenging task to accurately extract the target region from an IR image; this is further hampered by the optical properties of the IR band, such as reflection, refraction, and diffraction between the visible and radio/microwave bands, and by the fact that IR is propagated at the speed of light [4] . Thresholding is widely used for segmentation when extracting a target region from an IR background. One of the most popular methods is that of Otsu [1] — a thresholding approach that is based on a one-dimensional (1D) intensity histogram, which maximizes the class variance between the background and target pixels. A two-dimensional (2D) intensity histogram, which consists of pixel intensity information and the local average intensity of the neighborhood pixels, can better describe the intensity distribution. Accordingly, a threshold method based on a 2D intensity histogram using the maximum value of the cluster variance (2D Otsu) was proposed [5] , which is superior to the 1D Otsu method when images are corrupted by noise. However, risk regarding an exponential increment of the computation time occurs. Fast recursive methods [6] [7] have been proposed to reduce the computational complexity of 2D Otsu.
Since the development of IR-guided missiles, various IR small target detection methods, such as the spatial filter–based method [8] [10] , morphology-based method [11] , and temporal-based method [12] [13] , have been researched with the aim of improving one’s ability to detect a missile early on. However, IR near-target detection techniques have not been researched. Therefore, this paper introduces a novel extraction technique for IR near-targets.
In IR images, objects with different IR radiation normally appear in the form of different changes in gray values. To describe the change in gray values on different objects more clearly, the concept of a “complex degree” was presented [14] [15] . To measure the complex degree of different IR images quantitatively, weighted information entropy (WIE) is introduced to describe different IR backgrounds [16] .
In this paper, an IR target extraction method using WIE and an adaptive opening filter is proposed for IR near-target extraction. The proposed method detects the boundary region of a candidate target region using the distribution of local variance WIE. A coarse target region is then estimated through a labeling of the target boundary region. In a coarse target region, a fine target shape is extracted by an opening filter with an adaptive structure element.
II. Related Work
- 1. WIE and Target Size
WIE was presented for improving an IR image [16] . Let S denote a set of gray values in an IR image with 256 gray levels; ps , the probability of gray value s occurring in set S ; and s the mean of gray values in the IR image, where the variance of the WIE of the image can be formulated as follows:
H(S)= s=1 255 (s s ¯ ) 2 × p s log p s ,
where ps log ps = 0 when ps = 0. Actually, a complex degree can be regarded as a concept that corresponds to the distribution of frequency components in an IR image [14] [15] . For instance, regions with little change in gray values in an image (low degree of complexity), such as a mild sky, mainly consist of low-frequency components in the image. Regions containing small targets in an image usually embody drastic changes in grey values (high degree of complexity); thus, they mainly consist of high-frequency components. Therefore, this naturally gives us an opportunity to design a spatial high-pass filter for preprocessing IR small-target images. For pixel x in an IR image, if there are m kinds of gray values, s 1 , s 2 , ... , sm , in neighborhood M of the pixel and the probabilities of each gray value in the neighborhood are p s1 , p s2 , ... , psm , respectively, then the local variance WIE value of pixel x can be defined as
H(x)= i=1 m ( s i s ¯ (x) ) 2 × p s i log p s i ,
where s ( x ) represents the mean of the gray values in neighborhood M . By calculating a local variance WIE value at each pixel in the IR image, a local variance WIE image is obtained. Figure 1 shows the results of a local variance WIE filter for two IR images with small (AIRPLANE1) and large targets (TANK1). In Fig. 1(a) , a WIE filter detects a small target (5 × 5 pixels) around a cloud; however, it is difficult to detect a large target (25 × 15 pixels) on the ground. As shown in the figure, there is a drawback in that it is difficult to accurately detect a large target because the WIE filter emphasizes only transient regions, such as a boundary region with spatially varying gray values, as opposed to flat regions. Small targets mainly consist of transient regions, similar to the features of a boundary or edge region. In the case of a large target, a WIE filter detects its boundary region. Therefore, it can be used as a target-position detector in the case of large targets.
PPT Slide
Lager Image
Local variance WIE results for (a) AIRPLANE1 and (b) TANK1.
- 2. Morphological Operator and Structure Element Size
Morphological detection is one of the main methods researched in the field of IR target detection. Along with the development of morphological research, morphological image processing, which is a special image-processing discipline, has gradually become a new trend in image processing, and a favorable tool in IR target detection [17] [19] . In IR target detection based on morphological filters, the critical technology is its design including its morphological operator and structuring element. The basic morphological operations are erosion, dilation, opening, and closing — upon which we can obtain some important compound operations with different characteristics by combining them [20] . The dilation and erosion of image f by subimage (structuring element) b , denoted as f b and f b , respectively, are
(fb)(s)=max[f(sx)+b(x)|       for (sx) D f  and  x D b ],
(fb)(s)=max[f(s+x)b(x)|       for (s+x) D f  and  x D b ],
where Df and Db represent the region of input image f and structure element b , respectively. The opening of image f by b , denoted as f b , is
The opening is simply the erosion of f by b , followed by a dilation of the result by b . The morphological top-hat transformation of an image, denoted as h , is defined as
This morphological operator is affected by the shape and size of the structure element. In a Matlab simulation of the experimental results of this paper, the shape of the employed structure element resembled that of a disk in the case of all the test images.
In IR images, intensity variation of edge or boundary regions of objects is spatially gradual; such variations follow a Gaussian distribution [21] . Therefore, most objects within an IR image are not sharp and contain acute edges or boundary regions. For this reason, the aforementioned disk shape was adopted for simulations.
Figure 2 shows the impact of an opening filter through structure elements of various sizes. For a simulation using Matlab, the strel function (for example, strel (‘disk,’ radius)) and structure element function were used with ‘disk’ with radii of 2, 7, and 10. We can see that the performance of the opening filter strongly depends on the size of the chosen structure element. Larger structure element sizes destroy the shapes of targets, as shown in Fig. 2 . On the other hand, smaller structure element sizes may cause clutter around a target. Thus, when detecting a target using a morphological operator, structure element selection seriously affects any resulting extraction results.
PPT Slide
Lager Image
Impact by structure element for (a) TANK1 and (b) TANK2.
III. Proposed IR Target Extraction Method Using WIE and Adaptive Opening Filter
To solve the limit of WIE according to the target size and impact based on the structure element in the morphological operator with regard to IR target detection, we proposed an IR target extraction method using WIE and an adaptive opening filter. The structure of the proposed method is shown in Fig. 3 . The proposed method has the following three procedures: 1) detection of the candidate target and its boundary region, 2) determination of the entire target region, and 3) use of an adaptive opening operator.
PPT Slide
Lager Image
Structure of proposed method.
- 1. Candidate Target Boundary Region Detection
Local variance WIE information is used to estimate the possible target position. An IR near-target (or large target) has very high WIE values in its boundary region. On the other hand, the inner region of an IR near-target has flat or uniform WIE values. Based on these features (of IR near-targets), we detect a candidate target boundary region and target region as follows:
  • 1) Obtain local variance WIE image,WI(i,j), using 5 × 5 pixel size centered on pixel position (i,j) for an IR imageI(i,j), (1 ≤i≤M, 1 ≤j≤N).
  • 2) Extract candidate target boundary region using threshold Thc=μ+ασ, whereμandσrepresent the mean and standard deviation ofWI, respectively, andα= 7. As shown inFig. 4, as the boundary region of a target has very high WIE values, the threshold value Thcfor detecting a candidate target boundary region is set to a high value. The thresholded WIE image (candidate target boundary region) by Thcis shown inFig. 5(a). Then, a blocked image of the candidate target boundary region,Ic(x,y), (1 ≤x≤M/8, 1 ≤j≤N/8) is made and integrated using an 8 × 8 pixel size, as shown inFig. 5(b).
  • 3) Detect pixels with very high target probability (very high WIE) using threshold Tht=β× max[WI], as shown inFig. 5(c). Mainly, these pixels exist within the target boundary region, and the pixels’ coordinates offer a clue as to the whereabouts of a target. The maximum value ofWI(i,j) is represented by max[WI], whereβ= 0.9. The value ofβis to be chosen through experimentation, as the boundary regions of targets usually have abnormal WIE values. Simply, to detect a target, pixel coordination with the max[WI] (not Tht) is needed. And, to cope with multiple targets, a threshold process is used. Then, a blocked image of pixels with very high target probability,It(x,y), is made with an 8 × 8 pixel size, as shown inFig. 5(d).
PPT Slide
Lager Image
(a) Original image and (b) its local variance WIE image.
PPT Slide
Lager Image
Candidate target boundary region detection: (a) thresholded local variance WIE image; (b) blocked image Ic(x, y) for (a); (c) detected pixels with very high target probability; and (d) blocked image It(x, y) for (c).
- 2. Target Region Selection
The procedure for extracting an entire target region is as follows:
  • 1) Assign the same label to connected blocks inIc(x,y), blocked image of candidate target boundary region. Compared withFig. 5(b),Fig. 6(a)is assigned a set of labels for connected blocks (Fig. 6(a)has a total of nine labels (nine colors)). This label set,Lc, can be defined as follows:
L c ={ L 1 , L 2 , L 3 ,    , L p },
  • where subscript c means a candidate target boundary region andprepresents the total number of labels. For example,p= 9 forFig. 6(a). In the same way, a label set forIt(x,y),Lt, is given by
L t ={ L 1 , L 2 , L 3 ,    , L q },
  • where subscript t represents a target boundary region with high target probability andqrepresents the total number of labels. For example,q= 1 forFig. 6(b).
  • 2) Find intersection label setLct, amongLcandLt, which is defined by
L ct = L c L t ={ L 1 ,    , L r }.
  • Extract only intersection labels {L1, ... ,Lr} inLc, as shown inFig. 6(c).Figure 5(c)showsr= 1 (Intersection label is 1).
  • 3) Fill in inner part of intersection labels to determine rough target region.
  • 4) Count onNLr, the number of filled intersection labels, shown inFig. 6(d).
PPT Slide
Lager Image
Target region selection: (a) assigned labels for Ic(x, y); (b) assigned label for It(x, y); (c) intersection label; (d) filled intersection-label.
If NLr > 3, then the detected target region is considered to be an IR near-target; otherwise, it is considered a small target, whereby it is detected by only the local variance WIE method (as mentioned in Section II-1).
- 3. Adaptive Opening Filter
In an opening filter, the size of the structure element is adaptively determined by examining the WIE value of a transient region, such as a target boundary. A larger structure element deforms the inner region of a detected rough target region due to the erosion procedure used by the opening filter. On the other hand, a smaller structure element causes an inexact detection of a target boundary region due to the dilation procedure used by the opening filter. Thus, the size of the structure element should be optimized through a consideration of the width information via the WIE values in a target boundary. For this, a variable window is used for the maximum WIE coordinate, ( i m , j m ), which is mainly detected in transient regions, such as a target boundary. The window’s size, w = (2 k + 1) × (2 k + 1), centered on ( i m , j m ), is increased by comparing the mean WIE of the window μ ( W w ) with a threshold (Th se = τ × WI ( i m , j m ), τ = 0.5), as shown in Fig. 7 . If a target boundary is wide, then the size of the window will be increased while maintaining the mean WIE value for the window. The width information of a target boundary with high WIE value is decided in accordance with the size of the variable window. Finally, the size of the structure element for the opening filter is decided by k , half of a window size, w = (2 k + 1) × (2 k + 1), to prevent a target boundary region from being deformed and to reject remaining clutter in the detected target block.
PPT Slide
Lager Image
Method to determine size of structure element.
IV. Experimental Results
Computer simulations were conducted on seven IR images with different backgrounds (TANK1, TANK2, AIRPLANE2, EXPLOSION, MAN1, DEER, and MAN2), as shown in Fig. 8 . Table 1 shows the size and normalized noise level for test IR images. The noise level was estimated by a recent paper [22] .
PPT Slide
Lager Image
Near targets for experiments: (a) TANK1, (b) TANK2, (c) AIRPLANE2, (d) EXPLOSION, (e) MAN1, (f) DEER, and (g) MAN2.
Sizes and noise levels for test images.
Images Size Normalized noise level
TANK1 320 × 240 0.4954
TANK2 360 × 240 0.3414
AIRPLANE2 480 × 360 0.1739
EXPLOSION 320 × 240 0.0439
MAN1 320 × 240 0.0693
DEER 320 × 240 0.0362
MAN2 320 × 240 0.1385
TANK1, TANK2, and AIRPLANE2 have high contrast, but high noise level; on the other hand, EXPLOSION, MAN1, DEER, and MAN2 have low contrast, but low noise level. To verify the performance of the proposed method, we compared the results with those of the Otsu method and Top-hat filter.
Figure 9 shows extraction results for seven IR images including IR near-targets, by 2D Otsu method, Top-hat filter, and the proposed method. The performance of the 2D Otsu method is not bad for EXPLOSION, DEER, and MAN2, which have high gray-level targets, compared to backgrounds. Top-hat filter also shows good performance in the case where a target is bigger than the surrounding clutter and the proper size of its structure element is selected based on high-contrast images, as shown in the results for TANK1, TANK2, EXPLOSION, and MAN2. However, in the cases of AIRPLANE2 (which includes a small target) and MAN1 and DEER (both of which include low-contrast targets), Top-hat filter cannot effectively discriminate a target with clutter. In addition, Top-hat filter creates clutter in addition to the target in MAN2, thus producing a high-contrast image, yet one with a complex background. The disadvantage of Top-hat filter is that its extraction results are heavily affected by the size of the structure element. We can confirm that the 2D Otsu and Top-hat filter methods do not properly extract fine target shapes nor effectively reject clutter. Figure 8 shows that the proposed method effectively rejects clutter and accurately extracts IR near-targets.
PPT Slide
Lager Image
Extraction results on seven IR images including IR near-targets: (a) original IR images, (b) thresholded local variance WIE images, (c) label images, (d) 2D Otsu method, (e) Top-hat filter, and (f) proposed method.
Table 2 shows the size of the structure element for an adaptive opening filter in the cases of seven test IR images. We know that both AIRPLANE2 and MAN1 have a wide target boundary region since, in both cases, a large structure-element size and window size (to represent the size of a transient region, such as the boundary region of a target) have been used.
Structure element sizes for test IR images.
Images Window size Shape of SE Size of SE
TANK1 7 × 7 Disk 3
TANK2 5 × 5 Disk 2
AIRPLANE2 15 × 15 Disk 7
EXPLOSION 7 × 7 Disk 3
MAN1 9 × 9 Disk 4
DEER 7 × 7 Disk 3
MAN2 5 × 5 Disk 2
For a quantitative comparison, two metrics — signal-to-clutter ratio gain (SCRG) and background suppression factor (BSF) [23] — are employed and defined as follows:
SCRG= (S/C) out (S/C) in ,     BSF= C in C out ,
where S is the signal amplitude and C is the clutter standard deviation for the input (in) and output (out) IR images. The performance results for the methods shown in Fig. 8 are listed in Table 3 . From both subjective and objective comparison results, we can clearly see that the proposed method has better target extraction and background suppression abilities for IR target detection when compared to the existing methods.
Objective performance comparison of proposed and existing methods.
2D Otsu 5.25 6.59 6.45 5.25 4.21 4.82 4.85 5.23 7.25 6.12 7.51 7.21 5.85 4.32
Top-hat 5.10 5.60 6.54 5.63 6.43 6.12 6.23 7.54 2.16 1.98 7.62 6.67 4.28 3.21
Proposed 9.35 10.01 8.21 9.24 9.05 10.53 9.26 10.95 9.65 9.52 10.87 11.98 9.63 10.32
Table 4 shows the computation times of the aforementioned methods for the given test IR images. The proposed method needs a little more computation time compared to 2D Otsu and Top-hat filter due to its slightly higher complexity. In the future, to reduce the computation time, we think that it is better to either use subsampling for an entire input image or to locally apply the WIE process after preprocessing.
Computation times of proposed and existing methods for given test IR images.
Images Computation time (s)
2D Otsu Top-hat filter Proposed
TANK1 1.12 0.99 1.59
TANK2 1.18 0.97 1.63
AIRPLANE2 0.90 0.97 1.78
EXPLOSION 1.17 1.01 1.49
MAN1 1.16 1.01 1.55
DEER 1.16 0.98 1.60
MAN2 1.17 1.00 1.64
V. Conclusion
IR target detection is an important technique in IR search and tracking, which is necessary for military applications to warn about incoming targets. Until now, various IR small-target detection techniques have been developed to cope with IR guided missile or small targets that are at great distances away. However, research about IR near-target detection techniques is lacking. Thus, this paper proposed a novel method using weighted information entropy and an adaptive opening filter for extracting the fine shape of IR near-targets. Because the proposed method was designed considering the features of a target boundary, it can effectively detect IR near-targets, unlike existing methods. However, it requires greater computation time due to algorithm complexity. Basically, target extraction should be applied in real time to quickly cope with enemy attacks. Thus, techniques for increasing the speed of computation of the proposed algorithm are necessary in future.
Tae Wuk Bae received his BS, MS, and PhD degrees in electrical engineering from Kyungpook National University, Daegu, Rep. of Korea, in 2004, 2006, and 2010, respectively. Since 2014, he has been working for Daegu-Gyeongbuk Research Center, ETRI, Daegu, Rep. of Korea. His research interests include medical devices and medical signal processing.
Hwi Gang Kim received his BS and MS degrees in electrical engineering from Kyungpook National University, Daegu, Rep. of Korea, in 2008 and 2010, respectively. Since 2014, he has been working for Daegu-Gyeongbuk Research Center, ETRI, Daegu, Rep. of Korea. His research interests include image processing, computer vision, and medical signal processing.
Young Choon Kim received his BS, MS, and PhD degrees in electronic engineering from Kyungpook National University, Daegu, Rep. of Korea, in 1991, 1993, and 1997, respectively. Since 1999, he has been with the Department of Information Communication & Security, Youngdong University, Rep. of Korea. His research interests include robot vision, infrared image processing, and infrared countermeasures.
Corresponding Author
Sang Ho Ahn received his BS, MS, and PhD degrees in electronic engineering from Kyungpook National University, Daegu, Rep. of Korea, in 1986, 1988, and 1992, respectively. Since 1995, he has been with the Electronic Engineering Department, Inje University, Gimhae, Rep. of Korea. His research interests include robot vision, infrared image processing, and infrared countermeasures.
de Jong A.N. “IRST and its Perspective,” Proc. SPIE Int. Symp. Opt. Sci., Eng., Instrum. San Diego, CA, USA July 9–14, 1995 206 - 213
Wolfe W.L. 1996 “Introduction to Infrared System Design,” SPIE Optical Engineering Press Washington, DC, USA 29 - 40
Chengjun L. , Ying W. , Zeling S. “A Small Target Detection Algorithm Based on Multi-scale Energy Cross,” IEEE Int. Conf. Robot., Intell. Syst. Signal Process. Hunan, China Oct. 8–13, 2003 1191 - 1196
Kim S.H. , Lee J.H. 2005 “Near-Infrared Light Propagation in an Adult Head Model with Refractive Index Mismatch,” ETRI J. 27 (4) 377 - 384    DOI : 10.4218/etrij.05.0104.0056
Ostu N. 1979 “A Threshold Selection Method from Gay-Level Hstograms,” IEEE Trans. Syst. Man Cybern. 9 (1) 62 - 66    DOI : 10.1109/TSMC.1979.4310076
Gong J. , Li L. , Chen W. 1998 “Fast Recursive Algorithm for Two Dimensional Thresholding,” Pattern Recogn. 31 (3) 295 - 300    DOI : 10.1016/S0031-3203(97)00043-5
Zhang T.X. 2006 “Fast Recursive Algorithm for Infrared Ship Image Segmentation,” J. Infrared Millimeter Waves 25 (4) 295 - 300
Bae T.-W. 2009 “An Efficient Two-Dimensional Least Mean Square (TDLMS) Based on Block Statistics for Small Target Detection,” J. Infrared, Millimeter Terahertz Waves 30 (10) 1092 - 1101    DOI : 10.1007/s10762-009-9530-6
Bae T.-W. , Zhang F. , Kweon I.-S. 2012 “Edge Directional 2D LMS Filter for Infrared Small Target Detection,” Infrared Physics Technol. 55 (1) 137 - 145    DOI : 10.1016/j.infrared.2011.10.006
Bae T.-W. , Sohng K.-I. 2010 “Small Target Detection Using Bilateral Filter Based on Edge Component,” Int. J. Infrared Millimeter Terahertz. Waves 31 (6) 735 - 743
Bae T.-W. 2011 “Recursive Multi-SEs NWTH Method for Small Target Detection in Infrared Images,” IEICE Electron. Exp. 8 (19) 1576 - 1582    DOI : 10.1587/elex.8.1576
Bae T.-W. 2010 “Small Target Detection Using Cross Product Based on Temporal Profile in Infrared Image Sequences,” Comput. Electr. Eng. 36 (6) 1156 - 1164    DOI : 10.1016/j.compeleceng.2010.05.004
Bae T.-W. 2011 “Small Target Detection Using Bilateral Filter and Temporal Cross Product in Infrared Images,” Infrared Physics Technol. 54 (5) 403 - 411    DOI : 10.1016/j.infrared.2011.06.006
Yang L. , Yang J. , Yang K. 2004 “Adaptive Detection for Infrared Small Target under Sea–Sky Complex Background,” Electron. Lett. 40 (17) 1083 - 1085    DOI : 10.1049/el:20045204
Yang L. , Yang J. , Ling J. 2005 “New Criterion to Evaluate the Complex Degree of Sea–Sky Infrared Backgrounds,” Opt. Eng. 44 (12) 126401 - 126406    DOI : 10.1117/1.2140588
Yang L. , Zhou Y. , Yang J. 2006 “Variance WIE Based Infrared Images Processing,” Electron. Lett. 42 (15) 857 - 859    DOI : 10.1049/el:20060827
Jackway P. 2000 “Improved Morphological Top-Hat,” Electron. Lett. 36 (14) 1194 - 1195    DOI : 10.1049/el:20000873
Zeng M. , Li J. , Peng Z. 2006 “The Design of Top-Hat Morphological Filter and Application to Infrared Target Detection,” Infrared Physics Technol. 48 (1) 67 - 76    DOI : 10.1016/j.infrared.2005.04.006
Bai X. , Zhou F. 2010 “Infrared Small Target Enhancement and Detection Based on Modified Top-Hat Transformations,” Comput. Electr. Eng. 36 (6) 1193 - 1201    DOI : 10.1016/j.compeleceng.2010.05.008
Gonzalez C. , Woods E. , Eddins L. 2004 Digital Image Processing Using MATLAB Pearson Prentice Hall FL, USA 519 - 560
Xiong Y. 1997 “An Extended Track-Before-Detect Algorithm for Infrared Target Detection,” IEEE Trans. Aerosp. Electron. Syst. 33 (3) 1087 - 1092    DOI : 10.1109/7.599339
Liu X. , Tanaka M. , Okutomi M. 2013 “Single-Image Noise Level Estimation for Blind Denoising,” IEEE Trans. Image Process. 22 (12) 5226 - 5237    DOI : 10.1109/TIP.2013.2283400
Hilliard C.I. “Selection of a Clutter Rejection Algorithm for Real-Time Target Detection from an Airborne Platform,” Proc. SPIE Signal Data Process. Small Targets Orlando, FL, USA Apr. 24–28, 2000 74 - 84