*Spectral and Spatial Quality Evaluation of Pansharpened Images*

Using Ward's three property criteria, we tested the spectral synthesis and consistency properties of the pansharpened images using image quality indices. According to Wald [41], the first property stipulates that the pansharpened image, once degraded from its original resolution, should be as identical as possible to the original image. Secondly, the pansharpened image should be as identical as possible to the image that a matching sensor would detect with the highest resolution. Last, the multispectral pansharpened image should be as identical as possible to the multispectral set of images that the matching sensor would detect with the highest resolution. For assessment purposes, these three properties are further condensed into two properties: consistency and synthesis. The Ward protocol for the quality assessment of pansharpened imagery stipulates that consistency can be tested by downsampling the merged image from the higher spatial resolution to its original spatial resolution. The nearest neighbor resampling method was used in the downsampling process to ensure minimum transformation of the pixel values. To validate the synthesis property, the original high spatial resolution panchromatic band and the lower spatial resolution multispectral bands were downsampled to their lower resolutions.

To validate the synthesis property, we first degraded both the multispectral images and the panchromatic band by a factor of 4. This downsampling procedure meant the spatial resolution of the multispectral images changed from 6 m to 24 m while the panchromatic band changed from 1.5 m to 6 m. The degraded multispectral and pansharpened images were then fused and the pansharpened image was then subsequently compared to the original multispectral images for quality assessment. To verify the consistency property, we first pansharpened the native multispectral and panchromatic images to create a fused image that we further downsampled by a factor of 4, thus changing its spatial resolution of the pansharpened image from 1.5 m to 6 m. We subsequently compared the downsampled pansharpened image to the original 6 m multispectral image. The process was applied for all eight pansharpening techniques assessed in this paper.

To quantitatively assess the spectral consistency of the pansharpened results the following statistical measures were used: correlation coefficient (CC), Erreur Relative Global Adimensionnelle de Synthese (ERGAS), difference in variance (DIV), bias, root mean square error (RMSE), relative average spectral error (RASE), and universal image quality index (UIQI). The quality of the synthesis in an important property in pansharpening and we used the ERGAS indices using the original multispectral and panchromatic band as a reference to assess the quality of the synthesis. The ERGAS index, when used in the spatial and spectral dimension, is indicative of the amount of spatial and spectral distortions, respectively. The spatial consistency of the pansharpened results was assessed using a spatial metric that computes the spatial correlation coefficient (SCC) between the high-frequency components of the fusion product and the original PAN. In this case, we used a 3 × 3 Laplacian edge detection convolution filter to filter the bands of the pansharpened images and the original panchromatic band before computing the correlation coefficients between them.

The CC is one of the most widely used statistical measures of the strength and direction of the linear relationship between two images [37]. It is used to determine the amount of preservation of spectral content in two images. The CC between each band of the reference and the pansharpened image indicates the spectral integrity of the pansharpened image. The best fusion will have a higher value close to +1. RMSE measures the similarity between each band of the original and fused image. It measures the changes in the radiance of the pixel values for each band of the input multispectral image and pansharpened image. It is a very good indicator of the spectral quality when considered along homogeneous regions in the image. The best fusion will have a lower value close to zero [42]. RASE characterizes the average performance of a method in the considered spectral bands. The value is expressed in percentage and tends to decrease as the quality increases. UIQI measures the difference in spectral information between each band of the merged and reference image to estimate the global spectral quality of the merged images. It models distortion using three parameters: loss of correlation, luminance distortion, and contrast. The best fusion will have a higher value close to +1. ERGAS is indicative of the synthesizing quality of the pansharpened image. It is a global quality index that is sensitive to mean shifting and dynamic range change. ERGAS measures the amount of spectral distortion in the image. The best fusion will have a lower value, mostly when less than the number of bands [43]. Bias reveals the error and spectral accuracy of the pansharpened image. Ideal values are considered to be close to zero. The difference in variance (DIV) measures the quality of the image fusion by calculating the mean difference in variances between the pansharpened image and the original multispectral image. The quality of the pansharpening is considered ideal if the values are closer to zero.

#### **3. Results and Discussion**

The results of this study are presented and discussed in this section. Spatial consistency, spectral consistency, and spectral synthesis are presented in Tables 1–9.

#### *3.1. Spatial Consistency Quality Assessment*

The spatial consistency results are highlighted in Table 1 below.

**Table 1.** Spatial consistency: correlation coefficient (CC) Laplacian filtering. Abbreviations: Bayesian fusion (BAY); Brovey transform(BRO); Color Normalized Spectral sharpening (CNS); Ehlers fusion technique (EHLERS); Gram–Schmidt (GRS); Local Mean and Variance Matching (IMVM), Modified Intensity Hue Saturation (MIHS), Pansharp algorithm (PANSHARP), Principal component analysis (PCA); Ratio Component Substitution (RCS); WAVELET, Wavelet Resolution merge fusion (WAVELET).


Results are reflective of the correlation between the Laplacian filtered bands of the pan sharpened image and the Laplacian filtered panchromatic band. The domain value range from −1 to +1 and ideal values should be close to 1. The ideal value is 1. The results show the best spatial consistency results were produced by the Baysian pansharpening method with Gram–Schmidt in second place and CNS in third place. The wavelet pansharpening technique produced the worst spatial consistency results.

### *3.2. Spectral Consistency*

The results for the spectral consistency evaluation are outlined in Tables 2–8 below.


**Table 2.** Spectral consistency: correlation coefficient (CC).

The CC results are indicative of spectral similarity between the fused image and original multispectral image. The values range from −1 to +1 and the ideal value is considered to be close to 1. While this metric is quite popular, one of its disadvantages is that it is insensitive to a constant gain and bias between two images and is not able to distinguish subtle fusion artifacts. The results indicate that the IMVM method produced the best results followed by the RCS method. The worst results were produced by the Brovey method.

**Table 3.** Spectral consistency: Erreur Relative Global Adimensionnelle de Synthese (ERGAS).


The ERGAS results are indicative of the spectral distortions in the fused image. This gives an indication of the general quality of the fused image at a global level. Lower values are considered more ideal and the domain values range from zero to infinity. The best results were produced by the IMVM pansharpening method while RCS was second. The Brovey method performed poorly.


**Table 4.** Spectral consistency: universal image quality index (UIQI).

The UIQI results show the spectral and spatial distortions in the fused image. Results of this similarity index point to correlation losses as well as distortions in luminance and contrast. The domain values range from −1 to 1 and values close to 1 are considered ideal. The ideal value for UIQI is 1. The IMVM pansharpening algorithm produced the best results while the RCS method took second place. The worst results were produced by the Brovey method.

**Table 5.** Spectral consistency: relative average spectral error (RASE).


RASE results show the average performance of the fusion algorithm in spectral bands and ideal values should be as small as possible. The results show that the IMVM fusion method produced the best results followed by the RCS method. The PCA method produced the worst results.


**Table 6.** Spectral consistency: root square mean error (RMSE).

The RMSE results are reflective of the average spectral distortion arising from the image fusion and the results are indicative of spectral quality in homogeneous zones in the image. The domain for RMSE value ranges from zero to infinity and lower values close to zero are considered ideal and reflective of high quality. The best results were produced by the IMVM method followed by the RCS method. The worst results were produced by PCA and the Brovey method.

**Table 7.** Spectral consistency: difference in variance (DIV).


The results indicate the fusion quality over the whole image by showing difference in variances relative to the original one. The metric reveals a decrease or increase of information content as a result of the pansharpening process. The results are considered ideal positive when the information content decreases and undesirable when the information content increases. The ideal value should be close to 0. The IMVM pansharpening method produced the best results. The Brovey transform method ranked second and PCA had the worst performance.

**Table 8.** Spectral consistency: bias.


The results are reflective of difference between the original image and fused image and the ideal value should be as small as possible. The IMVM method showed the best results followed by the RCS method. The Brovey transform method showed the worst performance.
