Fast and Efficient Satellite Imagery Fusion Using DT-CWT Proportional and Wavelet Zero-Padding
Fast and Efficient Satellite Imagery Fusion Using DT-CWT Proportional and Wavelet Zero-Padding
Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography. 2015. Dec, 33(6): 517-526
Copyright © 2015, Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • Received : November 11, 2015
  • Accepted : December 12, 2015
  • Published : December 31, 2015
Export by style
Cited by
About the Authors
Yong-Hyun Kim
Member, Dept. of Civil and Environmental Engineering, Seoul National University (
Jae-Hong Oh
Corresponding Author, Member, Dept. of Civil Engineering, Chonnam National University (
Yong-Il Kim
Member, Dept. of Civil and Environmental Engineering, Seoul National University (
Among the various image fusion or pan-sharpening methods, those wavelet-based methods provide superior radiometric quality. However, the fusion processing is not only simple but also flexible, since many low- and high-frequency sub-bands are often produced in the wavelet domain. To address this issue, a novel DT-CWT (Dual-Tree Complex Wavelet Transform) proportional to the fusion method by a WZP (Wavelet Zero-Padding) is proposed. The proposed method produces a single high-frequency image in the spatial domain that is injected into the LRM (Low-Resolution Multispectral) image. Thus, a wavelet domain fusion can be simplified to spatial domain fusion. In addition, in the proposed DT-CWTP (DT-CWT Proportional ) fusion method, it is unnecessary to decompose the LRM image by adopting WZP. The comparison indicates that the proposed fusion method is nearly five times faster than the DT-CWT with SW (Substitute-Wavelet) fusion method, meanwhile simultaneously maintaining the radiometric quality. The conducted experiments with WorldView-2 satellite images demonstrated promising results with the computation efficiency and fused image quality.
1. Introduction
The satellite sensors typically supply LRM image and HRP (High-Resolution Panchromatic) image separately because of physical and technological constraints ( Thomas ., 2008 ; Wang ., 2005 ); which of that implies that the design of the HRM (High-Resolution Multispectral) sensor is limited ( Kim ., 2011a ; Zhang, 2004 ). Those constraints on the signal-to-noise ratio impose with the lower spatial resolution, but the desired spectral resolution is larger. Conversely, if and only if the highest spatial resolution is obtained whenever no spectral diversity is required ( Chen, 2012 ), the image fusion or the pan-sharpening aims to improve the geometric information of the original LRM image. However, we cannot obtain an ideal HRM image due to the abovementioned limitation ( Choi ., 2013 ; Wang ., 2005 ). Thus, to use an HRM image in various geospatial fields, an image fusion is a requisite with the efficient method in remote sensing.
Among the many image fusion methods that are currently available, the DWT (Discrete Wavelet Transform) methods provide superior fused images that maintain the radiometric information of the LRM image better than CS (Component-Substitution) fusion methods, such as the intensity-hue-saturation transform and principal component analysis ( Wang ., 2005 ). Although the DWT-based methods are spectrally consistent, there are two problems; firstly, DWT is a shift-variant, and exhibits artifacts due to the aliasing in fused image ( Ioannidou and Karathanassi, 2007 ; Thomas ., 2008 ); secondly, these fusion methods have a high computation cost and complexity compared with CS fusion methods ( Pradhan ., 2006 ). In particular, the latest satellite sensors for Earth observation are currently producing a nearly continual stream data. The explosion in the amount of collected information has required the development of computationally efficient techniques for transforming the massive amount of remote sensing data into scientific understanding.
The first problem can be overcome by the DT-CWT, which is nearly shift-invariant and directionally selective in two or more dimensions of making these methods particularly suitable for image fusion ( Ioannidou and Karathanassi, 2007 ; Selesnick ., 2005 ). In a practice, the second problem results in additional computational power and a longer wait time for the fused imagery. Thus, to address this issue, this paper proposes the DT-CWTP fusion method to solve the second problem - i.e., the high computation cost and complexity of the fusion method. In other words, the aim of the study is to present a fusion technique enabling an easier implementation of improved wavelet-based image fusion.
There are many studies on the DT-CWT in the remote sensing fields. Those superresolution imaging ( Celik and Tjahjadi, 2010 ) and despeckling of synthetic aperture radar data were proposed ( Ranjani and Thiruvengadan, 2010 ). In image fusion, the DT-CWT fusion method with SW method ( Ioannidou and Karathanassi, 2007 ) and a DT-CWT fusion method with an injection model have been proposed ( Renza ., 2011 ). However, the study focused on the SW method rather than AW (Additive-Wavelet) method. In the image fusion community, the AW-based fusion method is preferred because the preservation of the LRM image is critical ( Kim ., 2011b ; Núnez ., 1999 ; Thomas ., 2008 ). Additionally, these studies overlook the convenience of the fusion method in terms of algorithm flexibility.
In this study, we improve upon the DT-CWT fusion method to reduce the computation complexity compared to conventional methods and increase the flexibility of fusion method. To increase the radiometric quality of the fused image, the proposed method considers the WZP for conciseness and adopts the proportional injection model. To verify the efficiency and quality of the proposed method, experimental evaluations are conducted on the WorldView-2 data, and compared with the wavelet-based fusion methods. This paper organized as follows; in section 2, the DT-CWT image fusion and WZP briefly introduced, while the section3 proposed DT-CWTP fusion method; in section 4, those results and discussion are presented, when conclusions are drawn in section 5.
2. DT-CWT and Image Fusion
This section reviews the DT-CWT and related fusion methods with briefly discussions the characteristics of these methods. In addition, we focused on fusion schemes rather than mathematical formulations. It is assumed that those LRM and HRP images are a priori geometrically registered and superimposed. We enlarged the LRM image size to the HRP image size using bicubic interpolation.
Kingsbury (2001) introduced a new type of wavelet transform known as the DT-CWT, which exhibits a shift-invariant property and improves directional resolution compared with the DWT. The DT-CWT also yields the perfect reconstruction using two parallel decimated trees with real-valued coefficients generated at each tree ( Celik and Tjahjadi, 2010 ), so as provides a time frequency analysis of the signal by measuring its frequency contents at different times. The algorithm decomposed the input signal into two signals, or sub-bands, that represent the low- and high-frequency components, respectively. The DT-CWT with the complex wavelet function and complex scaling function decomposes an image into one complex scaling subband and six complex wavelet sub-bands at each decomposition level. The wavelet sub-bands are oriented in six dimensions and provide the directionality of the complex wavelet function ( Renza ., 2011 ). More detailed theoretical and application parts of DT-CWT can be seen in Selesnick . (2005) .
- 2.1 Perfect reconstruction with WZP
An image can be decomposed into a series of low- and high-frequency sub-bands using the specific wavelet transform. PR (Perfect Reconstruction) means that the synthesized final image is the same as the original input image. Among the many wavelet transforms, the DT-CWT supports the PR condition. This condition is important in signal and image processing because the recovered signal or image should not take the errors within the precision of the specific arithmetic implementation. This paper adapts the PR condition to image fusion.
The WZP was used to produce the super-resolved imagery by filling the unknown high-frequency sub-bands with zeros ( Temizel and Vlachos, 2005b ). In other words, an approximation to the unknown high-resolution image is generated using WZP. Using the available low-resolution image x of size m × n, the unknown high-resolution image
PPT Slide
Lager Image
is reconstructed by the zero-padding of high-frequency sub-band following to the inverse wavelet transform (IWT) as follows:
PPT Slide
Lager Image
where 0 m,n is an all-zero of size m × n ( Temizel and Vlachos, 2005a ). The WZP method can be conversely applied in image fusion. That is, the known low-frequency sub-bands can be filled with zeros. By performing the IWT, a single high-frequency image in the spatial domain is produced. Conversely, the known high-frequency sub-bands can be filled with zeros. By performing the IWT, a single low-frequency image in the spatial domain is produced, which approach can be applied to the DT-CWT. The low frequency or high frequency of the original image can be easily discarded. The PR conditions and the WZP method of DT-CWT are shown in Fig. 1 , which can be simply implemented as an AW method. This method can be naturally expanded for any wavelet transform that supports the PR condition.
PPT Slide
Lager Image
PR condition and WZP of the DT-CWT
- 2.2 DT-CWT image fusion and general image fusion framework
The primary objective of DT-CWT fusion method is the reduction of aliasing in DWT-based fusion methods and obtaining in the analytic transform to minimize shift dependence, which is possible to use complex wavelets for a complex-type filter bank. However, the currently proposed DT-CWT fusion methods are based on the SW method ( Ioannidou and Karathanassi, 2007 ; Renza ., 2011 ). The primary disadvantage of these methods is the requisite decomposition of the LRM image and the subsequent loss of the high frequency of the LRM image. The first problem increases the computation cost, and the second problem results in the loss of the radiometric and geometric information of the LRM image ( Kim ., 2011b ). The information that is visible in the LRM image can be missing in the HRP image and vice versa ( Thomas ., 2008 ). That is, SW-based methods could eliminate both the geometric and radiometric information of the LRM image. Thus, the AW-based fusion methods are preferable. Also, we have to develop an effective fusion method that is based on the AW method using DT-CWT with a fast computation speed. This task requires to maintain the LRM image during the fusion process, and inject the high-frequency, which is not included in the LRM image, by using a simple and effective approach that is the main concept of the proposed method.
Aiazzi . (2009) demonstrated a general fusion framework that can be defined as follows:
PPT Slide
Lager Image
where HRMi is the i th HRM image, LRMi is the resized i th LRM image, LRP is the low-resolution panchromatic image or intensity image, and ωn is the global/local fusion parameter. Also, Wang . (2005) present a similar general image fusion method, in which the mathematical model is as follows:
PPT Slide
Lager Image
where α is the modulation coefficient and ω is the signal difference between the HRP and LRP images. Parameter ω represents the geometric information between the high- and low-resolution levels. Thus, an important point in image fusion is the development of a modulation coefficient to mitigate the radiometric distortion and generation of geometric information to inject into the LRM image.
3. Proposed DT-CWTP Fusion Method
Firstly, we developed the DT-CWT fusion method as an AW method in the spatial domain by adopting the PR condition and WZP. After that, the proposed DT-CWTP is presented.
The procedure of proposed DT-CWT with AW fusion method can be summarized as follows:
1) Perform histogram matching between the HRP image and intensity image. The intensity image is obtained by averaging the LRM image.
2) Decompose only the histogram-matched HRP image.
3) The low-frequency sub-bands of the decomposed HRP image are filled with zeros.
4) An IDT-CWT (Inverse DT-CWT) is conducted to produce the WP (Wavelet Plane). The WP is the sum of the high frequencies and contains the detailed geometric information of the HRP image.
5) The WP is added to the resized LRM image, and finally, the fused HRM image is generated. Mathematically, the WP can be defined as follows:
PPT Slide
Lager Image
where LFHRP is the low frequency of the HRP image and HFHRP is the high frequency of the HRP image. Zeros is an all-zero matrix of the LFHRP image size. Thus, the AW fusion method can be simplified into single WP injection method in the spatial domain as follows:
PPT Slide
Lager Image
where the WP has the same size as the resized LRM and HRP images. This fusion method can produce the same type of fused imagery produced by the conventional AW methods. However, the proposed method is simpler and more flexible in that this method fuses in the spatial domain, which illustrated in Fig. 2 . Those previous studies used the AW method as the addition approach in the wavelet domain ( Amolins ., 2007 ).
PPT Slide
Lager Image
DT-CWT fusion method with the AW method
However, the proposed fusion method injects the same geometric information into each LRM image because this method does not yet possess the modulation coefficient of the general image fusion framework. The radiometric signature of each HRM image is not preserved, which implies that the fixed injection method can produce unnecessary or redundant information; the redundant information means that the geometric and radiometric information is not presented in the LRM image, but is presented in the fused image, and vice versa. Thus, some further improvements can be achieved by injecting the detailed geometric information using the modulation coefficient. Consequently, to preserve the geometric and radiometric information in the fused data, the proposed fusion method can be rewritten as follows:
PPT Slide
Lager Image
where L is the number of LRM bands. This method proportionally injects the geometric information into every pixel while considering the relative radiometric signatures of the LRM bands in the manner of the AWLP (AdditiveWavelet Luminance Proportional) ( Otazu ., 2005 ) and IAWP (Improved Additive-Wavelet Proportional) fusion methods ( Kim ., 2011b ). To obtain this weighting factor, the ratio between the LRM image and the mean value of all LRM images was calculated. This approach facilitates the injection of geometric details into the LRM image in a manner proportional to their original values. Thus, the fused image can preserve the radiometric angle between the original LRM and fused HRM images. This method is termed the DT-CWTP method. Importantly, there is no need to decompose the LRM image, however, only a simple addition operation was used. Thus, the proposed fusion method is more capable; and it has better computational efficiency than the conventional SW-based fusion methods. In addition, this fusion method can apply the local fusion parameter in the spatial domain rather than the wavelet domain ( Choi ., 2013 ). Moreover, the trade-off between radiometric and geometric information could be controlled using a modulation coefficient, as in LilloSaavedra and Gonzalo (2006) .
4. Results and Discussion
In this study, we used the WorldView-2 satellite imagery to verify our fusion method. In our experiments, the LRM image size was 512 × 512 pixels, while the HRP image size was 2048 × 2048 pixels. A Fig. 3 shows the WorldView-2 data, whereas study area covers a large of agricultural fields, urban areas, and river. We compared the proposed fusion method with DT-CWT with the SW method ( Ioannidou and Karathanassi, 2007 ). Additionally, the à trous wavelet-based fusion methods presented in Otazu ., (2005) and Kim ., (2011b) were compared with our method. Also, the fusion processing time has compared based on the computation cost. The reported times were measured in the MATLAB 2014b environment and in a quad-core 2.5 GHz personal computer platform by using the average elapsed time of 20 trials.
PPT Slide
Lager Image
WorldView-2 data
PPT Slide
Lager Image
Fused images comparison
The Table 1 demonstrates the elapsed time of each fusion method. The AWLP method was the fastest method among those compared methods. The IAWP method was two times slower than AWLP method, since the IAWP method must generate the LRP image by filtering with a Gaussian low-pass filter that of frequency response matches the shape of the modulation transfer function. The DT-CWT with the SW method was the slowest method because of the decomposition of the LRM image and the substitution process in the wavelet domain.
Elapsed time comparison
PPT Slide
Lager Image
Elapsed time comparison
The DT-CWT with the AW method and the proposed DT-CWTP method exhibit similar elapsed times that resemble those of the IAWP method. Above all, the proposed method is nearly five times faster than the DT-CWT with the SW method ( Ioannidou and Karathanassi, 2007 ). If the histogram matching were performed with each LRM image rather than intensity image, the elapsed time would be increased in proportion to the total number of bands in all of the fusion methods. On the other hand, the computation complexity or cost is affected by the image size, data read and write speed, pre- and post-processing, and speed of internal memory. Thus, further studies using more accurate mathematical model will be addressed in our future work. Nevertheless, the proposed fusion method is contributes to image fusion community and other image processing fields.
Objectively evaluate these fusion methods, the QNR (Quality with No Reference) index, as proposed by Alparone . (2008) , was used. The radiometric distortion is referred to as Dλ , and the geometric distortion index is referred to as Ds . The highest value of the QNR is one and obtained when the radiometric and geometric distortion are both zero. The QNR index is defined as
PPT Slide
Lager Image
A higher QNR value indicates that most of the radiometric and geometric information of the LRM and HRP images are incorporated during fusion processing. The main advantage of the QNR index is that, in spite of the lack of a reference data, the global quality of a fusion method can be assessed at the full scale of HRP image. Table 2 shows the performance comparisons of the fused images. The results indicate that the proposed DT-CWTP method provides a less distorted fused image compared with other fusion methods. Thus, as demonstrated by the elapsed time and QNR index, the proposed fusion method has low computational time requirements and is superior in fused image quality.
QNR indices
PPT Slide
Lager Image
QNR indices
In addition, we compare the subset WPs of the AWLP, IAWP, and DT-CWT fusion methods in Fig. 5 . The WP is the sum of high frequencies and the zero-mean image. The WP of AWLP method resembles the primary high frequency in Choi .(2013) ; that was clearly demonstrated that the primary high-frequency information maximizes the geometric clarity of fused image, while maintaining the radiometric distortion caused by injecting the excessive high-frequency information. The WP of IAWP and DT-CWT is similar but displays a subtle distinction in visual analysis. The WP of the DT-CWT accentuates the dominant edge information of the feature compared with that in the AWLP and IAWP methods.
PPT Slide
Lager Image
Comparison of subset WPs
5. Conclusions
The critical issue in image fusion is how much radiometric information is preserved while simultaneously increasing the geometric information. Additionally, if the fused data from specific fusion method are nearly identical with other fused data from a more computationally efficient method, the latter method is ideal for a large quantity of remote sensing data. To address these problems, we proposed the DT-CWTP fusion method, which is considered an improvement upon the AW and SW methods of DT-CWT fusion in the sense. The LRM image is not decomposed and injects the geometric information as a proportional method. In the experimental results using WorldView-2 data, the proposed method demonstrated the higher speed and the higher quality. The comparison indicates that the proposed fusion method is nearly five times faster than the DT-CWT with SW fusion method. The most significantly, the proposed fusion method increases flexibility because the wavelet domain fusion is simplified to the spatial domain by adopting the WZP method. This approach facilitates the general and less computationally intensive fusion method using the DT-CWT.
This research was supported by an NRF (National Research Foundation) grant funded by the South Korean government (NRF-2014R1A1A1001995).
Aiazzi B. , Baronti S. , Lotti F. , Selva M. (2009) A comparison between global and context-adaptive pansharpening of multispectral images IEEE Geoscience and Remote Sensing Letters 6 (2) 302 - 306
Alparone L. , Aiazzi B. , Baronti S. , Garzelli A. , Nencini F. , Selva M. (2008) Mutltispectral and panchromatic data fusion assessment without reference Photogrammetric Engineering and Remote Sensing 74 (2) 193 - 200
Amolins K. , Zhang Y. , Dare P. (2007) Wavelet based image fusion techniques-an introduction, review and comparison ISPRS Journal of Photogrammetry and Remote Sensing 62 (4) 249 - 263
Celik T. , Tjahjadi T. (2010) Image resolution enhancement using dual-tree complex wavelet transform IEEE Geoscience and Remote Sensing Letters 7 (1) 123 - 126
Chen C.H. (2012) Signal and Image Processing for Remote Sensing CRC Press, Taylor & Francis Group
Choi J. , Yeom J. , Chang A. , Byun Y. , Kim Y. (2013) Hybrid pansharpening algorithm for high spatial resolution satellite imagery to improve spatial quality IEEE Geoscience and Remote Sensing Letters 10 (3) 490 - 494
Ioannidou S. , Karathanassi V. (2007) Investigation of the dual-tree complex and shift-invariant discrete wavelet transforms on quickbird image fusion IEEE Geoscience and Remote Sensing Letters 4 (1) 166 - 170
Kim Y. , Eo Y. , Kim Y. , Kim Y. (2011a) Generalized IHS-based satellite imagery fusion using spectral response function ETRI Journal 33 (4) 497 - 505
Kim Y. , Lee C. , Han D. , Kim Y. , Kim Y. (2011b) Improved additive-wavelet image fusion IEEE Geoscience and Remote Sensing Letters 8 (2) 263 - 267
Kingsbury N. (2001) Complex wavelets for shift invariant analysis and filtering of signals Applied and Computational Harmonic Analysis 10 (3) 234 - 253
Lillo-Saavedra M. , Gonzalo C. (2006) Spectral or spatial quality for fused satellite imagery? A trade-off solution using the wavelet à trous algorithm International Journal of Remote Sensing 27 (7) 1453 - 1464
Núnez J. , Otazu X. , Fors O. , Prades A. , Pala V. , Arbiol R. (1999) Multiresolution-based image fusion with additive wavelet decomposition IEEE Transactions on Geoscience and Remote Sensing 37 (3) 1204 - 1211
Otazu X. , Fors O. , Núnez J. , González-Audícana M. (2005) Introduction of sensor spectral response into image fusion method. Application to wavelet-based methods IEEE Transactions on Geoscience and Remote Sensing 43 (10) 2376 - 2385
Pradhan P.S. , King R.L. , Younan N.H. , Holcomb D.W. (2006) Estimation of the number of decomposition levels for a wavelet-based multiresolution multisensor image fusion IEEE Transactions on Geoscience and Remote Sensing 44 (12) 3674 - 3686
Ranjani J.J. , Thiruvengadan S. (2010) Dual-tree complex wavelet transform based SAR despeckling using interscale dependence IEEE Transactions on Geoscience and Remote Sensing 48 (6) 2723 - 2731
Renza D. , Martinez E. , Arquero A. (2011) Quality assessment by region in spot images fused by means dual-tree complex wavelet transform Advances in Space Research 48 (8) 1377 - 1391
Selesnick I.W. , Baranjuk R.G. , Kingsbury N.G. (2005) The dual-tree complex wavelet transform IEEE Signal Process. Magazine 22 (6) 123 - 151
Temizel A. , Vlachos T. (2005a) Image resolution upscaling in the wavelet domain using directional cycle spinning Journal of Electronic Imaging 14 (4) 040501 - 040501-3
Temizel A. , Vlachos T. (2005b) Wavelet domain image resolution enhancement using cycle-spinning Electronics Letters 41 (3) 119 - 121
Thomas C. , Ranchin T. , Wald L. , Chanussot J. (2008) Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics IEEE Transactions on Geoscience and Remote Sensing 46 (5) 1301 - 1312
Wang Z. , Ziou D. , Armenakis C. , Li D. , Li Q. (2005) A comparative analysis of image fusion methods IEEE Transactions on Geoscience and Remote Sensing 43 (6) 1391 - 1402
Zhang Y. (2004) Understanding image fusion Photogrammetric Engineering and Remote Sensing 70 (6) 653 - 660