Fast and Efficient Satellite Imagery Fusion Using DT-CWT Proportional and Wavelet Zero-Padding

Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography.
2015.
Dec,
33(6):
517-526

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

- Received : November 27, 2015
- Accepted : December 31, 2015
- Published : December 31, 2015

Download

PDF

e-PUB

PubReader

PPT

Export by style

Article

Metrics

Cited by

TagCloud

Among the various image fusion or pan-sharpening methods, those wavelet-based methods provide superior radiometric quality. However, the fusion processing is not only simple but also flexible, since many low- and high-frequency sub-bands are often produced in the wavelet domain. To address this issue, a novel DT-CWT (Dual-Tree Complex Wavelet Transform) proportional to the fusion method by a WZP (Wavelet Zero-Padding) is proposed. The proposed method produces a single high-frequency image in the spatial domain that is injected into the LRM (Low-Resolution Multispectral) image. Thus, a wavelet domain fusion can be simplified to spatial domain fusion. In addition, in the proposed DT-CWTP (DT-CWT
Proportional
) fusion method, it is unnecessary to decompose the LRM image by adopting WZP. The comparison indicates that the proposed fusion method is nearly five times faster than the DT-CWT with SW (Substitute-Wavelet) fusion method, meanwhile simultaneously maintaining the radiometric quality. The conducted experiments with WorldView-2 satellite images demonstrated promising results with the computation efficiency and fused image quality.
a priori
geometrically registered and superimposed. We enlarged the LRM image size to the HRP image size using bicubic interpolation.
Kingsbury (2001)
introduced a new type of wavelet transform known as the DT-CWT, which exhibits a shift-invariant property and improves directional resolution compared with the DWT. The DT-CWT also yields the perfect reconstruction using two parallel decimated trees with real-valued coefficients generated at each tree (
Celik and Tjahjadi, 2010
), so as provides a time frequency analysis of the signal by measuring its frequency contents at different times. The algorithm decomposed the input signal into two signals, or sub-bands, that represent the low- and high-frequency components, respectively. The DT-CWT with the complex wavelet function and complex scaling function decomposes an image into one complex scaling subband and six complex wavelet sub-bands at each decomposition level. The wavelet sub-bands are oriented in six dimensions and provide the directionality of the complex wavelet function (
Renza ., 2011
). More detailed theoretical and application parts of DT-CWT can be seen in
Selesnick . (2005)
.
x
of size m × n, the unknown high-resolution image
is reconstructed by the zero-padding of high-frequency sub-band following to the inverse wavelet transform (IWT) as follows:
where 0
_{m,n}
is an all-zero of size
m
×
n
(
Temizel and Vlachos, 2005a
). The WZP method can be conversely applied in image fusion. That is, the known low-frequency sub-bands can be filled with zeros. By performing the IWT, a single high-frequency image in the spatial domain is produced. Conversely, the known high-frequency sub-bands can be filled with zeros. By performing the IWT, a single low-frequency image in the spatial domain is produced, which approach can be applied to the DT-CWT. The low frequency or high frequency of the original image can be easily discarded. The PR conditions and the WZP method of DT-CWT are shown in
Fig. 1
, which can be simply implemented as an AW method. This method can be naturally expanded for any wavelet transform that supports the PR condition.
PR condition and WZP of the DT-CWT
where
HRM_{i}
is the
i
th HRM image,
LRM_{i}
is the resized
i
th LRM image,
LRP
is the low-resolution panchromatic image or intensity image, and
ω_{n}
is the global/local fusion parameter. Also,
Wang . (2005)
present a similar general image fusion method, in which the mathematical model is as follows:
where
α
is the modulation coefficient and
ω
is the signal difference between the HRP and LRP images. Parameter
ω
represents the geometric information between the high- and low-resolution levels. Thus, an important point in image fusion is the development of a modulation coefficient to mitigate the radiometric distortion and generation of geometric information to inject into the LRM image.
where
LF_{HRP}
is the low frequency of the HRP image and
HF_{HRP}
is the high frequency of the HRP image.
Zeros
is an all-zero matrix of the
LF_{HRP}
image size. Thus, the AW fusion method can be simplified into single WP injection method in the spatial domain as follows:
where the WP has the same size as the resized LRM and HRP images. This fusion method can produce the same type of fused imagery produced by the conventional AW methods. However, the proposed method is simpler and more flexible in that this method fuses in the spatial domain, which illustrated in
Fig. 2
. Those previous studies used the AW method as the addition approach in the wavelet domain (
Amolins ., 2007
).
DT-CWT fusion method with the AW method
However, the proposed fusion method injects the same geometric information into each LRM image because this method does not yet possess the modulation coefficient of the general image fusion framework. The radiometric signature of each HRM image is not preserved, which implies that the fixed injection method can produce unnecessary or redundant information; the redundant information means that the geometric and radiometric information is not presented in the LRM image, but is presented in the fused image, and vice versa. Thus, some further improvements can be achieved by injecting the detailed geometric information using the modulation coefficient. Consequently, to preserve the geometric and radiometric information in the fused data, the proposed fusion method can be rewritten as follows:
where
L
is the number of LRM bands. This method proportionally injects the geometric information into every pixel while considering the relative radiometric signatures of the LRM bands in the manner of the AWLP (AdditiveWavelet Luminance Proportional) (
Otazu ., 2005
) and IAWP (Improved Additive-Wavelet Proportional) fusion methods (
Kim ., 2011b
). To obtain this weighting factor, the ratio between the LRM image and the mean value of all LRM images was calculated. This approach facilitates the injection of geometric details into the LRM image in a manner proportional to their original values. Thus, the fused image can preserve the radiometric angle between the original LRM and fused HRM images. This method is termed the DT-CWTP method. Importantly, there is no need to decompose the LRM image, however, only a simple addition operation was used. Thus, the proposed fusion method is more capable; and it has better computational efficiency than the conventional SW-based fusion methods. In addition, this fusion method can apply the local fusion parameter in the spatial domain rather than the wavelet domain (
Choi ., 2013
). Moreover, the trade-off between radiometric and geometric information could be controlled using a modulation coefficient, as in
LilloSaavedra and Gonzalo (2006)
.
à trous
wavelet-based fusion methods presented in
Otazu ., (2005)
and
Kim ., (2011b)
were compared with our method. Also, the fusion processing time has compared based on the computation cost. The reported times were measured in the MATLAB 2014b environment and in a quad-core 2.5 GHz personal computer platform by using the average elapsed time of 20 trials.
WorldView-2 data
Fused images comparison
The
Table 1
demonstrates the elapsed time of each fusion method. The AWLP method was the fastest method among those compared methods. The IAWP method was two times slower than AWLP method, since the IAWP method must generate the LRP image by filtering with a Gaussian low-pass filter that of frequency response matches the shape of the modulation transfer function. The DT-CWT with the SW method was the slowest method because of the decomposition of the LRM image and the substitution process in the wavelet domain.
Elapsed time comparison
The DT-CWT with the AW method and the proposed DT-CWTP method exhibit similar elapsed times that resemble those of the IAWP method. Above all, the proposed method is nearly five times faster than the DT-CWT with the SW method (
Ioannidou and Karathanassi, 2007
). If the histogram matching were performed with each LRM image rather than intensity image, the elapsed time would be increased in proportion to the total number of bands in all of the fusion methods. On the other hand, the computation complexity or cost is affected by the image size, data read and write speed, pre- and post-processing, and speed of internal memory. Thus, further studies using more accurate mathematical model will be addressed in our future work. Nevertheless, the proposed fusion method is contributes to image fusion community and other image processing fields.
Objectively evaluate these fusion methods, the QNR (Quality with No Reference) index, as proposed by
Alparone . (2008)
, was used. The radiometric distortion is referred to as
D_{λ}
, and the geometric distortion index is referred to as
D_{s}
. The highest value of the QNR is one and obtained when the radiometric and geometric distortion are both zero. The QNR index is defined as
A higher QNR value indicates that most of the radiometric and geometric information of the LRM and HRP images are incorporated during fusion processing. The main advantage of the QNR index is that, in spite of the lack of a reference data, the global quality of a fusion method can be assessed at the full scale of HRP image.
Table 2
shows the performance comparisons of the fused images. The results indicate that the proposed DT-CWTP method provides a less distorted fused image compared with other fusion methods. Thus, as demonstrated by the elapsed time and QNR index, the proposed fusion method has low computational time requirements and is superior in fused image quality.
QNR indices
In addition, we compare the subset WPs of the AWLP, IAWP, and DT-CWT fusion methods in
Fig. 5
. The WP is the sum of high frequencies and the zero-mean image. The WP of AWLP method resembles the primary high frequency in
Choi .(2013)
; that was clearly demonstrated that the primary high-frequency information maximizes the geometric clarity of fused image, while maintaining the radiometric distortion caused by injecting the excessive high-frequency information. The WP of IAWP and DT-CWT is similar but displays a subtle distinction in visual analysis. The WP of the DT-CWT accentuates the dominant edge information of the feature compared with that in the AWLP and IAWP methods.
Comparison of subset WPs

1. Introduction

The satellite sensors typically supply LRM image and HRP (High-Resolution Panchromatic) image separately because of physical and technological constraints (
Thomas ., 2008
;
Wang ., 2005
); which of that implies that the design of the HRM (High-Resolution Multispectral) sensor is limited (
Kim ., 2011a
;
Zhang, 2004
). Those constraints on the signal-to-noise ratio impose with the lower spatial resolution, but the desired spectral resolution is larger. Conversely, if and only if the highest spatial resolution is obtained whenever no spectral diversity is required (
Chen, 2012
), the image fusion or the pan-sharpening aims to improve the geometric information of the original LRM image. However, we cannot obtain an ideal HRM image due to the abovementioned limitation (
Choi ., 2013
;
Wang ., 2005
). Thus, to use an HRM image in various geospatial fields, an image fusion is a requisite with the efficient method in remote sensing.
Among the many image fusion methods that are currently available, the DWT (Discrete Wavelet Transform) methods provide superior fused images that maintain the radiometric information of the LRM image better than CS (Component-Substitution) fusion methods, such as the intensity-hue-saturation transform and principal component analysis (
Wang ., 2005
). Although the DWT-based methods are spectrally consistent, there are two problems; firstly, DWT is a shift-variant, and exhibits artifacts due to the aliasing in fused image (
Ioannidou and Karathanassi, 2007
;
Thomas ., 2008
); secondly, these fusion methods have a high computation cost and complexity compared with CS fusion methods (
Pradhan ., 2006
). In particular, the latest satellite sensors for Earth observation are currently producing a nearly continual stream data. The explosion in the amount of collected information has required the development of computationally efficient techniques for transforming the massive amount of remote sensing data into scientific understanding.
The first problem can be overcome by the DT-CWT, which is nearly shift-invariant and directionally selective in two or more dimensions of making these methods particularly suitable for image fusion (
Ioannidou and Karathanassi, 2007
;
Selesnick ., 2005
). In a practice, the second problem results in additional computational power and a longer wait time for the fused imagery. Thus, to address this issue, this paper proposes the DT-CWTP fusion method to solve the second problem - i.e., the high computation cost and complexity of the fusion method. In other words, the aim of the study is to present a fusion technique enabling an easier implementation of improved wavelet-based image fusion.
There are many studies on the DT-CWT in the remote sensing fields. Those superresolution imaging (
Celik and Tjahjadi, 2010
) and despeckling of synthetic aperture radar data were proposed (
Ranjani and Thiruvengadan, 2010
). In image fusion, the DT-CWT fusion method with SW method (
Ioannidou and Karathanassi, 2007
) and a DT-CWT fusion method with an injection model have been proposed (
Renza ., 2011
). However, the study focused on the SW method rather than AW (Additive-Wavelet) method. In the image fusion community, the AW-based fusion method is preferred because the preservation of the LRM image is critical (
Kim ., 2011b
;
Núnez ., 1999
;
Thomas ., 2008
). Additionally, these studies overlook the convenience of the fusion method in terms of algorithm flexibility.
In this study, we improve upon the DT-CWT fusion method to reduce the computation complexity compared to conventional methods and increase the flexibility of fusion method. To increase the radiometric quality of the fused image, the proposed method considers the WZP for conciseness and adopts the proportional injection model. To verify the efficiency and quality of the proposed method, experimental evaluations are conducted on the WorldView-2 data, and compared with the wavelet-based fusion methods. This paper organized as follows; in section 2, the DT-CWT image fusion and WZP briefly introduced, while the section3 proposed DT-CWTP fusion method; in section 4, those results and discussion are presented, when conclusions are drawn in section 5.
2. DT-CWT and Image Fusion

This section reviews the DT-CWT and related fusion methods with briefly discussions the characteristics of these methods. In addition, we focused on fusion schemes rather than mathematical formulations. It is assumed that those LRM and HRP images are
- 2.1 Perfect reconstruction with WZP

An image can be decomposed into a series of low- and high-frequency sub-bands using the specific wavelet transform. PR (Perfect Reconstruction) means that the synthesized final image is the same as the original input image. Among the many wavelet transforms, the DT-CWT supports the PR condition. This condition is important in signal and image processing because the recovered signal or image should not take the errors within the precision of the specific arithmetic implementation. This paper adapts the PR condition to image fusion.
The WZP was used to produce the super-resolved imagery by filling the unknown high-frequency sub-bands with zeros (
Temizel and Vlachos, 2005b
). In other words, an approximation to the unknown high-resolution image is generated using WZP. Using the available low-resolution image
PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

- 2.2 DT-CWT image fusion and general image fusion framework

The primary objective of DT-CWT fusion method is the reduction of aliasing in DWT-based fusion methods and obtaining in the analytic transform to minimize shift dependence, which is possible to use complex wavelets for a complex-type filter bank. However, the currently proposed DT-CWT fusion methods are based on the SW method (
Ioannidou and Karathanassi, 2007
;
Renza ., 2011
). The primary disadvantage of these methods is the requisite decomposition of the LRM image and the subsequent loss of the high frequency of the LRM image. The first problem increases the computation cost, and the second problem results in the loss of the radiometric and geometric information of the LRM image (
Kim ., 2011b
). The information that is visible in the LRM image can be missing in the HRP image and vice versa (
Thomas ., 2008
). That is, SW-based methods could eliminate both the geometric and radiometric information of the LRM image. Thus, the AW-based fusion methods are preferable. Also, we have to develop an effective fusion method that is based on the AW method using DT-CWT with a fast computation speed. This task requires to maintain the LRM image during the fusion process, and inject the high-frequency, which is not included in the LRM image, by using a simple and effective approach that is the main concept of the proposed method.
Aiazzi . (2009)
demonstrated a general fusion framework that can be defined as follows:
PPT Slide

Lager Image

PPT Slide

Lager Image

3. Proposed DT-CWTP Fusion Method

Firstly, we developed the DT-CWT fusion method as an AW method in the spatial domain by adopting the PR condition and WZP. After that, the proposed DT-CWTP is presented.
The procedure of proposed DT-CWT with AW fusion method can be summarized as follows:
1) Perform histogram matching between the HRP image and intensity image. The intensity image is obtained by averaging the LRM image.
2) Decompose only the histogram-matched HRP image.
3) The low-frequency sub-bands of the decomposed HRP image are filled with zeros.
4) An IDT-CWT (Inverse DT-CWT) is conducted to produce the WP (Wavelet Plane). The WP is the sum of the high frequencies and contains the detailed geometric information of the HRP image.
5) The WP is added to the resized LRM image, and finally, the fused HRM image is generated. Mathematically, the WP can be defined as follows:
PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

PPT Slide

Lager Image

4. Results and Discussion

In this study, we used the WorldView-2 satellite imagery to verify our fusion method. In our experiments, the LRM image size was 512 × 512 pixels, while the HRP image size was 2048 × 2048 pixels. A
Fig. 3
shows the WorldView-2 data, whereas study area covers a large of agricultural fields, urban areas, and river. We compared the proposed fusion method with DT-CWT with the SW method (
Ioannidou and Karathanassi, 2007
). Additionally, the
PPT Slide

Lager Image

PPT Slide

Lager Image

Elapsed time comparison

PPT Slide

Lager Image

PPT Slide

Lager Image

QNR indices

PPT Slide

Lager Image

PPT Slide

Lager Image

5. Conclusions

The critical issue in image fusion is how much radiometric information is preserved while simultaneously increasing the geometric information. Additionally, if the fused data from specific fusion method are nearly identical with other fused data from a more computationally efficient method, the latter method is ideal for a large quantity of remote sensing data. To address these problems, we proposed the DT-CWTP fusion method, which is considered an improvement upon the AW and SW methods of DT-CWT fusion in the sense. The LRM image is not decomposed and injects the geometric information as a proportional method. In the experimental results using WorldView-2 data, the proposed method demonstrated the higher speed and the higher quality. The comparison indicates that the proposed fusion method is nearly five times faster than the DT-CWT with SW fusion method. The most significantly, the proposed fusion method increases flexibility because the wavelet domain fusion is simplified to the spatial domain by adopting the WZP method. This approach facilitates the general and less computationally intensive fusion method using the DT-CWT.
Acknowledgements

This research was supported by an NRF (National Research Foundation) grant funded by the South Korean government (NRF-2014R1A1A1001995).

Aiazzi B.
,
Baronti S.
,
Lotti F.
,
Selva M.
(2009)
A comparison between global and context-adaptive pansharpening of multispectral images
IEEE Geoscience and Remote Sensing Letters
6
(2)
302 -
306
** DOI : 10.1109/LGRS.2008.2012003**

Alparone L.
,
Aiazzi B.
,
Baronti S.
,
Garzelli A.
,
Nencini F.
,
Selva M.
(2008)
Mutltispectral and panchromatic data fusion assessment without reference
Photogrammetric Engineering and Remote Sensing
74
(2)
193 -
200
** DOI : 10.14358/PERS.74.2.193**

Amolins K.
,
Zhang Y.
,
Dare P.
(2007)
Wavelet based image fusion techniques-an introduction, review and comparison
ISPRS Journal of Photogrammetry and Remote Sensing
62
(4)
249 -
263
** DOI : 10.1016/j.isprsjprs.2007.05.009**

Celik T.
,
Tjahjadi T.
(2010)
Image resolution enhancement using dual-tree complex wavelet transform
IEEE Geoscience and Remote Sensing Letters
7
(1)
123 -
126
** DOI : 10.1109/LGRS.2009.2028440**

Chen C.H.
(2012)
Signal and Image Processing for Remote Sensing
CRC Press, Taylor & Francis Group

Choi J.
,
Yeom J.
,
Chang A.
,
Byun Y.
,
Kim Y.
(2013)
Hybrid pansharpening algorithm for high spatial resolution satellite imagery to improve spatial quality
IEEE Geoscience and Remote Sensing Letters
10
(3)
490 -
494
** DOI : 10.1109/LGRS.2012.2210857**

Ioannidou S.
,
Karathanassi V.
(2007)
Investigation of the dual-tree complex and shift-invariant discrete wavelet transforms on quickbird image fusion
IEEE Geoscience and Remote Sensing Letters
4
(1)
166 -
170
** DOI : 10.1109/LGRS.2006.887056**

Kim Y.
,
Eo Y.
,
Kim Y.
,
Kim Y.
(2011a)
Generalized IHS-based satellite imagery fusion using spectral response function
ETRI Journal
33
(4)
497 -
505
** DOI : 10.4218/etrij.11.1610.0042**

Kim Y.
,
Lee C.
,
Han D.
,
Kim Y.
,
Kim Y.
(2011b)
Improved additive-wavelet image fusion
IEEE Geoscience and Remote Sensing Letters
8
(2)
263 -
267
** DOI : 10.1109/LGRS.2010.2067192**

Kingsbury N.
(2001)
Complex wavelets for shift invariant analysis and filtering of signals
Applied and Computational Harmonic Analysis
10
(3)
234 -
253
** DOI : 10.1006/acha.2000.0343**

Lillo-Saavedra M.
,
Gonzalo C.
(2006)
Spectral or spatial quality for fused satellite imagery? A trade-off solution using the wavelet à trous algorithm
International Journal of Remote Sensing
27
(7)
1453 -
1464
** DOI : 10.1080/01431160500462188**

Núnez J.
,
Otazu X.
,
Fors O.
,
Prades A.
,
Pala V.
,
Arbiol R.
(1999)
Multiresolution-based image fusion with additive wavelet decomposition
IEEE Transactions on Geoscience and Remote Sensing
37
(3)
1204 -
1211
** DOI : 10.1109/36.763274**

Otazu X.
,
Fors O.
,
Núnez J.
,
González-Audícana M.
(2005)
Introduction of sensor spectral response into image fusion method. Application to wavelet-based methods
IEEE Transactions on Geoscience and Remote Sensing
43
(10)
2376 -
2385
** DOI : 10.1109/TGRS.2005.856106**

Pradhan P.S.
,
King R.L.
,
Younan N.H.
,
Holcomb D.W.
(2006)
Estimation of the number of decomposition levels for a wavelet-based multiresolution multisensor image fusion
IEEE Transactions on Geoscience and Remote Sensing
44
(12)
3674 -
3686
** DOI : 10.1109/TGRS.2006.881758**

Ranjani J.J.
,
Thiruvengadan S.
(2010)
Dual-tree complex wavelet transform based SAR despeckling using interscale dependence
IEEE Transactions on Geoscience and Remote Sensing
48
(6)
2723 -
2731
** DOI : 10.1109/TGRS.2010.2041241**

Renza D.
,
Martinez E.
,
Arquero A.
(2011)
Quality assessment by region in spot images fused by means dual-tree complex wavelet transform
Advances in Space Research
48
(8)
1377 -
1391
** DOI : 10.1016/j.asr.2011.06.022**

Selesnick I.W.
,
Baranjuk R.G.
,
Kingsbury N.G.
(2005)
The dual-tree complex wavelet transform
IEEE Signal Process. Magazine
22
(6)
123 -
151
** DOI : 10.1109/MSP.2005.1550194**

Temizel A.
,
Vlachos T.
(2005a)
Image resolution upscaling in the wavelet domain using directional cycle spinning
Journal of Electronic Imaging
14
(4)
040501 -
040501-3

Temizel A.
,
Vlachos T.
(2005b)
Wavelet domain image resolution enhancement using cycle-spinning
Electronics Letters
41
(3)
119 -
121
** DOI : 10.1049/el:20057150**

Thomas C.
,
Ranchin T.
,
Wald L.
,
Chanussot J.
(2008)
Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics
IEEE Transactions on Geoscience and Remote Sensing
46
(5)
1301 -
1312
** DOI : 10.1109/TGRS.2007.912448**

Wang Z.
,
Ziou D.
,
Armenakis C.
,
Li D.
,
Li Q.
(2005)
A comparative analysis of image fusion methods
IEEE Transactions on Geoscience and Remote Sensing
43
(6)
1391 -
1402
** DOI : 10.1109/TGRS.2005.846874**

Zhang Y.
(2004)
Understanding image fusion
Photogrammetric Engineering and Remote Sensing
70
(6)
653 -
660

Citing 'Fast and Efficient Satellite Imagery Fusion Using DT-CWT Proportional and Wavelet Zero-Padding
'

@article{ GCRHBD_2015_v33n6_517}
,title={Fast and Efficient Satellite Imagery Fusion Using DT-CWT Proportional and Wavelet Zero-Padding}
,volume={6}
, url={http://dx.doi.org/10.7848/ksgpc.2015.33.6.517}, DOI={10.7848/ksgpc.2015.33.6.517}
, number= {6}
, journal={Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography}
, publisher={Korean Society of Surveying, Geodesy, Photogrammetry and Cartography}
, author={Kim, Yong-Hyun
and
Oh, Jae-Hong
and
Kim, Yong-Il}
, year={2015}
, month={Dec}