Advanced
Adaptive Object-Region-Based Image Pre-Processing for a Noise Removal Algorithm
Adaptive Object-Region-Based Image Pre-Processing for a Noise Removal Algorithm
KSII Transactions on Internet and Information Systems (TIIS). 2013. Dec, 7(12): 3166-3179
Copyright © 2013, Korean Society For Internet Information
  • Received : July 04, 2013
  • Accepted : November 16, 2013
  • Published : December 30, 2013
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Sangwoo Ahn
Department of Nanoscale Semiconductor Engineering, Hanyang University Seoul, Rep. of Korea
Jongjoo Park
Department of Nanoscale Semiconductor Engineering, Hanyang University Seoul, Rep. of Korea
Linbo Luo
Department of Science and Technology, China University of Geosciences, Wuhan, P. R. China
Jongwha Chong
Department of Nanoscale Semiconductor Engineering, Hanyang University Seoul, Rep. of Korea

Abstract
A pre-processing system for adaptive noise removal is proposed based on the principle of identifying and filtering object regions and background regions. Human perception of images depends on bright, well-focused object regions; these regions can be treated with the best filters, while simpler filters can be applied to other regions to reduce overall computational complexity. In the proposed method, bright region segmentation is performed, followed by segmentation of object and background regions. Noise in dark, background, and object regions is then removed by the median, fast bilateral, and bilateral filters, respectively. Simulations show that the proposed algorithm is much faster than and performs nearly as well as the bilateral filter (which is considered a powerful noise removal algorithm); it reduces computation time by 19.4 % while reducing PSNR by only 1.57 % relative to bilateral filtering. Thus, the proposed algorithm remarkably reduces computation while maintaining accuracy.
Keywords
1. Introduction
A s digital imaging techniques proliferate, there is demand for high-quality images from sources ranging from digital broadcasting systems to smartphones. However, the continuing miniaturization and increasing resolution of CMOS image sensors (CIS) comeswith the problem of increasing noise. A great deal of research has focused on noise removal techniques [1] .
Noise removal algorithms aim to remove image noise while maximally maintaining edges, texture, and color. Noisy images can be symbolized as follows:
PPT Slide
Lager Image
where o ( x , y ) represents the noisy image, i ( x , y ) represents the noise-free natural image, and n ( x , y ) represents the noise; x and y represent the two-dimensional pixel coordinates. In the context of Eq. (1), the purpose of the noise removal algorithm can be thought of as estimating the image as close to i ( x , y ) as possible by removing n ( x , y ) from o ( x , y ).
Among the numerous possible approaches, this paper proposes an effective algorithm for pre-processing based on region-based noise removal. The region-based noise removal strategy targets the human visual system, which gives more attention to well-focused objects and bright regions rather than blurred and dark regions [2] . Accordingly, the region-based noise removal method segments a noised image into three types of regions: well-focused regions that include objects, bright regions without objects, and dark regions. According to the characteristics of each region, one of three noise removal algorithms is applied: a highly accurate algorithm with high computational complexity is applied to well-focused regions, a moderately accurate algorithm with moderate computational complexity is applied to bright regions, and a low-accuracy algorithm with low computational complexity is applied to dark regions. Thus, the computational complexity and processing time can be reduced without loss of noise removal accuracy as perceived by the human visual system.
Our research aims to develop efficient methods for the pre-processing system of a noise removal algorithm; to this end, the present work employs a novel method based on adaptive higher-order statistics (HOS).
In Section 2, we review previous related work in noise removal and segmentation. Section 3 describes the proposed algorithm, focusing on pre-processing filters for effective segmentation. Section 4 presents simulations in which the pre-processing filter is applied to database images and nature images and evaluated in terms of computation speed and noise removal accuracy.
2. Related Work
- 2.1 Image Segmentation
There are many approaches for segmenting images. Among them, the higher-order statistics (HOS) method is widely used to determine which regions are focused well or poorly. This method gives outstanding results for Gaussian noisy images [3 4] . The HOS method accentuates the difference between neighboring pixels using fourth-order statistical analysis on the red, blue, and green channels of input images, which are calculated as follows for each channel (using the green channel as an example).
PPT Slide
Lager Image
where η ( x , y ) are the neighboring pixels centered at ( x , y ), IG ( s , t ) is the green channel of the input image,
PPT Slide
Lager Image
is the mean value of the neighboring pixels, Nη is the size of set η ( x , y ), and DSF is a downscaling factor experimentally set to 300.
PPT Slide
Lager Image
are the fourth-order statistics on the green channel, and HOS ( x , y ) is the final result representing the disparity value.
According to previous research, HOS methods suffer from some limitations. One issue is that current implementations of the HOS method do not consider the distance between neighboring pixels. Thus, we can improve the effectiveness by using the distance between pixels as in the bilateral filter.
- 2.2 Noise Reduction
Noise reduction is a widely researched topic in image processing. In this paper, we do not improve the accuracy of the noise reduction algorithm but instead propose a pre-processing system to improve the efficiency of noise reduction by means of state-of-the-art algorithms. In this section, we introduce the three noise reduction algorithms used in the proposed system.
  • • Median filter: The median filter is an old and widely used noise reduction algorithm. From the left-top block of the input image (in vertical scanning order), the center pixel of each block is replaced by the median value of that block. This method very quickly reduces the noise ratio of a noisy image but damages the quality of the image.
  • • Bilateral filter: The bilateral filter[5–6]replaces the center pixel of a block with a calculated pixel, which is a weighted average that takes into account the spatial and tonal distances between pairs of pixels in the block. The weighted average gives preference to the information of similar pixels such as spatially close and those with similar tone. The bilateral filter efficiently reduces noise without losing edge information. The bilateral filter can be expressed as follows:
PPT Slide
Lager Image
where F [ I ] p is the restored value of pixel p , σ s is the standard deviation for the spatial distance of neighboring pixels, σ r is the standard deviation in the tonal distance between neighboring pixels, G represents the Gaussian function, Ip and Iq are the pixel intensities of pixels p and q , respectively, S represents the set of neighboring pixels centered at p , and Wp is the sum of the weights within the block. The bilateral filter is computationally complex but very useful for preserving edge information; thus, it is best applied to well-focused and object regions.
  • • Fast bilateral filter: The fast bilateral filter[7]reduces the computational complexity of the bilateral filter, losing some accuracy but decreasing computation time. The main contributor to reducing computational cost is the replacement of Gaussian filters with a lower-cost function that has a similar shape, for example:
PPT Slide
Lager Image
where F [ I ] p is the restored value of pixel p , Iq is the intensity value of pixel q , and S represents the neighboring pixel centered at pixel p . dq and vq are the spatial weight and tonal weight of pixel q , respectively, which are defined as follows:
PPT Slide
Lager Image
where gq is the spatial distance, fq is the tonal distance, and c is an adjustable constant experimentally set to 8. The shapes of these weights are similar to that of the Gaussian function and can replace it without great loss of accuracy.
3. Proposed Algorithm
The core algorithm proposed in this paper is the pre-processing filter that segments images into three different regions for effective image noise removal; this strategy has been researched extensively because of its ability to improve the accuracy and speed [8] .
The proposed pre-processing filter segments images into three types of regions related to the perception of the human visual system, and a different noise-removal algorithm is applied to each type as appropriate. The three region types used in the filter are dark regions, background regions, and object regions. First, object edges are not noticeable in dark regions, so preserving edges during noise removal is unnecessary. Second, noise is more conspicuous in background regions but does not attract the viewer’s main attention. Third, the object region attracts the most attention, and noise in this region has the worst impact on the human visual system; accordingly, clear noise removal and preservation of edge information is key. This characteristic-based region classification improves the noise-removal algorithm by reducing its computational cost and concentrating on the regions that are most likely to receive careful attention.
In this paper, we apply the median filter to the dark region, the fast bilateral filter to the background region, and the bilateral filter to the object region ( Fig. 1 ).
PPT Slide
Lager Image
Block diagram of the proposed algorithm.
- 3.1 Image Segmentation by Luminance
Image segmentation by luminance classifies image segments as dark regions and bright regions. In the proposed algorithm, dark regions are those in which it is too dark to recognize objects; dark regions are well-suited to the median filter, since it inexpensively removes noise and its tendency to degrade image quality is less important in these regions. Segmentation based on luminance level can be described as follows:
PPT Slide
Lager Image
where σ ( i , j ) represents the standard deviation within each block,
PPT Slide
Lager Image
represents the mean within each block, c ( i , j ) is the coefficient of each block, v ( i , j ) is the luminance intensity, x ( i , j ) represents the pixel intensity on pixel ( i , j ), α is the adjustment coefficient (experimentally set to 1.1), TB is the threshold value (set to 1/9 of the mean value), and L ( i , j ) is a binary classification in which 1 represents a bright region and 0 represents a dark region. This simple algorithm segments dimly lit regions easily and with low computational complexity.
- 3.2 Image Segmentation by Object and Background
The proposed segmentation method for object and background is based on the HOS [3] presented above in Eq. (2). The HOS refers to the disparity between each pixel and its neighboring pixels; however, this method has a weakness in that the mean does not contain sufficient information to disentangle objects in complex, noisy conditions. We solve this problem by supplementing the image gradient and spatial-tonal distances with a bilateral filter.
First, spatial distance weights represent similarity in location among pixels; closer points have higher weights and farther points have lower weights. Second, tonal distance weights represent the similarity in intensity among pixels; cognate pixels have low disparity and are given high weights. Third, gradient weights are calculated from the disparity between each pixel and its top, bottom, left, and right neighboring pixels; if these gradients are less than a predefined threshold, the pixel is judged to be in the same region as its neighboring pixels, and the results of the segmentation algorithm can be applied to the pixels as a group. The benefits of this algorithm are reduction in the computational complexity and improvement in the accuracy of segmentation because error points of HOS due to noise are eliminated. These three weights are implemented as follows:
PPT Slide
Lager Image
where wsd ( s , t ) and wtd ( s , t ) are the spatial and tonal distance weights between pixel ( x , y ) and its neighboring pixel ( s , t ), respectively. S is the set of neighboring pixels, and wG ( x , y ) is the binary weight of image gradients on pixel ( x , y ). TG is the threshold value; we set this value to 20 according to experimental calculations, but it can be changed in accordance with images.
The proposed HOS method using the weights mentioned above is implemented as follows:
PPT Slide
Lager Image
where mR (2) ( x , y ), mG (2) ( x , y ), and mB (2) ( x , y ) are the proposed second-order statistics on the red, green, and blue channels, respectively; wG ( x , y ), wsd ( s , t ), and wtd ( s , t ) are the weights mentioned above; IR ( s , t ) and IR ( x , y ) are the intensity values on the ( s , t ) and ( x , y ) pixels, respectively; η ( x , y ) is the set of neighboring pixels centered at ( x , y ); and C is an adjustment constant set to 100 in this paper. ASOS ( x , y ) is the result of the proposed adaptive second-order statistics (ASOS) and is highest in the object region.
The binary ASOS value classifies image regions as object or background regions [9] ; here we use the convention that 1 represents the high-ASOS object region and 0 represents the low-ASOS background region.
4. Experimental Classification Results and Analysis
Simulations were carried out on the Tampere Image Database 2008 (TID 2008) [10] , a collection of low-depth-of-field images and nature images. TID 2008 contains 25 different images with various scenes, ensuring that simulations assess the accuracy of the proposed noise-removal algorithm in a variety of conditions. The images with low depth of field emphasize the performance of the proposed ASOS because their object and background regions are distinct. The nature images are taken without any effects and represent conditions found in a wide variety of images, thereby testing the algorithm’s general utility.
Our first simulation compared the noise reduction performance of our proposed ASOS to that of the original HOS on a noise-free image, an image with artificially added noise, and a naturally noisy image ( Fig. 2 ). These simulations were intended to test the performance of the proposed method in segmenting object and background regions; the ideal results are better performance by the proposed segmentation algorithm on the noisy images.
PPT Slide
Lager Image
Object/background segmentation by HOS and the proposed method. The first row is a noise-free image, the second row is the same image with artificially added noise, and the third row is a naturally captured image.
PPT Slide
Lager Image
PSNR Comparison: ASOS vs. HOS with increasing noise level.
This comparison verifies that the proposed method more accurately segments the images into object and background areas than the original HOS method for both the artificially noisy image and naturally noisy image, verifying its applicability as a pre-processing filter to improve the accuracy and efficiency of the region-based noise removal algorithm. In addition, we simulate the noise immunity of ASOS with the increasing noise level images shown in Fig. 3 . The figure above shows that the PSNR results of ASOS decreased more slowly than those of HOS, which indicates that ASOS is less sensitive to noise than HOS. PSNR results are calculated based on HOS and ASOS results for a noiseless image, respectively.
PPT Slide
Lager Image
Image, PSNR, and computation time results of the proposed algorithm (first row), the bilateral filter (second row), and the fast bilateral filter (third row).
Table 1 describes the numerical verification of the simulation results for LENA, BABOON, park, and flower in Fig. 5 . As shown in Table 1 , the proposed algorithm greatly reduces the computation time without loss of accuracy compared to the bilateral filter and fast bilateral filter. However, some of the results take too much computation time compared to the fast bilateral filter. Therefore, we need to focus on reducing the computational time of the proposed segmentation algorithm in order to reduce the time gap between the proposed method and the fast bilateral filter in future work.
In addition, we simulated a naturally captured image like that in Fig. 6 . With the results of the simulation, we verify that the proposed algorithm can be applied to the general images without low depth of field. The enlarged images in Fig. 6 show that the proposed algorithm can effectively maintain object edges with elimination of the background region. Faces on the enlarged images demonstrate that the proposed algorithm has the same results as the bilateral filter, and the wall surface shows the same results as the fast bilateral filter.
Performance measurement of the proposed algorithm
PPT Slide
Lager Image
Performance measurement of the proposed algorithm
PPT Slide
Lager Image
Test images.
PPT Slide
Lager Image
Noise removal from a low-light image.
Lastly, we compared the proposed algorithm with famous state-of-the-art algorithms. One is a current fast bilateral filter using trigonometric range kernels [11] , and the other is a current region-considered denoising algorithm via gradient histogram preservation [12] . Both algorithms have similarities to our approaches to fast denoising. Therefore, we demonstrate that the proposed algorithm works well with this simulation.
Fig. 7 shows that the proposed algorithm is more accurate than the traditional fast bilateral filter; however, the traditional region-considered denoising algorithm is surprisingly accurate. With regard to accuracy, the traditional region-considered denoising algorithm is superior, and the proposed algorithm shows better performance that the current fast bilateral filter. Nevertheless, with regard to computational time, the region-considered algorithm is extremely slow compared to other algorithms. As a result, the proposed algorithm is competitive with regard to computation time and accuracy.
PPT Slide
Lager Image
Comparison with state-of-the-art algorithms.
5. Conclusion
We proposed a pre-processing filter for noise removal which segments the image into object and background regions and applies different noise removal algorithms to each region based on its characteristics. Simulations demonstrated that the proposed algorithm reduces computational times relative to the bilateral filter while performing nearly as well in terms of PSNR. However, the proposed algorithm still requires relatively long computational times compared to the fast bilateral filter. Future work will focus on reducing the computational time of the segmentation.
BIO
Sangwoo Ahn received a B.S. degree in Department of Information and Communication Engineering from Hanyang University, Seoul, South Korea in 2010 and He finished the unified Master’s and Doctor’s course work in Department of Nanoscale Semiconductor Engineering from Hanyang University. He is currently Ph.D. candidate in Department of Nanoscale Semiconductor Engineering from Hanyang University, Seoul, South Korea. His research interests include image processing, image stabilization and de-focusing.
Jongjoo Park received a B.S. degree in Department of Information and Communication Engineering from Hanyang University, Seoul, Korea in 2011 and He received a M.S. degree in Department of Nanoscale Semiconductor Engineering from Hanyang University, Seoul, South Korea, 2013 and He is currently on the Doctor’s course in Department of Nanoscale Semiconductor Engineering from Hanyang University, Seoul, South Korea. His research interests include image processing, image stabilization, de-mosaicking and de-focusing.
Linbo Luo received his Ph.D. degree in Electronics and Computer Engineering from Hanyang University, Seoul, Korea in 2012. He is currently a researcher with the Department of Science and Technology, China University of Geosciences, Wuhan. His research interests include image warping, digital image stabilization for image sensor and their embedded implementation.
Jongwha Chong was born in Nonsan, Korea, on March 10, 1950. He received B.S. and M.S. degrees in Electronics Engineering from Hanyang University, Seoul, Korea, in 1975, and 1979 respectively. He received his Ph.D. degree in Electronics and Communication Engineering from Waseda University, Japan in 1981. Since 1981, he has been a professor at the Department of Electronics Engineering, Hanyang University, Seoul, Korea.
References
Oh B. , Wu P. , Xu D. , Kuo C. 2010 “Improved image denoising with adaptive nonlocal means (ANL-means) algorithm” IEEE Transactions on Consumer Electronics 56 (4) 2623 - 2630    DOI : 10.1109/TCE.2010.5681149
Vijaya G. , Vasudevan V. 2010 “A Simple Algorithm for Image Denoising Based on MS Segmentation” inProc. of International Journal of Computer Applications 2 (6) 9 - 15    DOI : 10.5120/674-947
Park J. , Kim C. 2006 “Extracting Focused Object from Low Depth-of-Field Image Sequences” inProc. of SPIE Visual Communications and Image Processing 6077 607710-1 - 607710-8    DOI : 10.4218/etrij.07.0106.0173
Gelle G. , Colas M. , Delaunay G. 1997 “Higher Order Statistics for Detection and Classification of Faulty Fabelts Using Acoustical Analysis” in Proc. of IEEE Signal Processing Workshop on Higher-Order-Statistics Jul 43 - 46    DOI : 10.1109/HOST.1997.613484
Tomasi C. , Manduchi R. 1998 “Bilateral Filtering for Gray and color Images” in Proc. IEEE Int. Conf. Computer. Vision 59 - 66    DOI : 10.1109/ICCV.1998.710815
Vinh T.Q. , Kim Y.C. 2010 “Edge-Preserving Algorithm for Block Artifact Reduction and its Pipelined Architecture” ETRI Journal 32 (3) 380 - 389    DOI : 10.4218/etrij.10.0109.0290
Luo L. , Chong J. 2012 “Real-Time Digital Image Stabilization for Cell Phone Cameras in Low-Light Enviroments without Frame Memory” ETRI Journal 34 (1) 138 - 141    DOI : 10.4218/etrij.12.0211.0338
Lee S. , Maik V. , Jang J. , Shin J. , Paik J. 2005 “Noise adaptive spatio-temporal filter for real-time noise removal in low light level images” IEEE Transaction on Consumer Electronics 51 (2) 648 - 653    DOI : 10.1109/TCE.2005.1468014
Cheng H.D. , Jiang X.H. , Wang J. 2002 “Color image segmentation based on homogram thresholding and region merging” Pattern Recognition 35 (2) 373 - 393    DOI : 10.1016/S0031-3203(01)00054-1
Ponomarenko N. , Lukin V. , Zelensky A. , Egiazarian K. , Carli M. , Battisti F. 2009 “TID2008 – A Database for Evaluation of Full-Reference Visual Quality Assessment Metrics” Advances of Modern Radioelectronics 10 30 - 45
Chaudhury K N , Sage D , Unser M 2011 “Fast O (1) bilateral filtering using trigonometric range kernels” IEEE Transaction on Image Processing 20 (12) 3376 - 3382    DOI : 10.1109/TIP.2011.2159234
Zuo W. , Zhang L. , Song C. , Zhang D. 2013 “Texture Enhanced Image Denoising via Gradient Histogram Preservation” in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)    DOI : 10.1109/CVPR.2013.159