1. Introduction
KOMPSAT-3, a Korean earth observing satellite, provides four multispectral (MS) bands (i.e., blue, green, red, near infrared) and one panchromatic (PAN) band. The image fusion or the pan-sharpening combines MS and PAN images into a single image that increase the spatial resolution while simultaneously preserving the spectral information in the MS images. This leads to high spectral resolution and high spatial resolution for various applications, such as visual inspection to identify the textures and shape of the various objects, feature extraction, and map updating. A number of pan-sharpening techniques have been studied, including intensity-hue-saturation (IHS), high-pass filtering (HPF), principal component analysis (PCA), Brovey and wavelet transforms, and Contourlet transform [
1].
Co-registration is the most critical pre-processing requirement for the fusion of the MS and PAN images, and its accuracy is an important parameter of the fusion product quality [
2]. Because MS and PAN CCD (the charge-coupled device) lines are placed with certain offsets, each CCD line captures the same ground target at different times. Under the influence of satellite with the ephemeris and terrain effects remaining, the MS and PAN images cannot be precisely co-registered to each other by a simple image shift. The inaccurate co-registration also impacts further applications like data fusion, change detection, and spectral-signature-based classification.
Conventional image co-registration has been studied out in two ways. First, the image-matching approach performs sub-pixel matching between MS and PAN images, providing tie points that enable geometric modeling between the images [
3,
4]. In addition, some studies proposed methods consisting of coarse and fine steps [
5,
6]. Secondly, the sensor modeling approach establishes the geometric relationship between MS and PAN to estimate the geometric discrepancy [
7,
8]. The aforementioned approaches have advantages and disadvantages. Image-matching methods are usually based on features, which makes it easier to be implemented than sensor modeling. Thus, they are chosen for post-processing when the ephemeris data or the sensor modeling data are not available. There are also many tools available for image registration processes [
9,
10,
11]. However, the performance is highly dependent on terrain features, showing a limited capability over a monotonous terrain (e.g., highly forested area). The sensor modeling approach tends to show higher accuracy, though the rigorous physical modeling process is more complicated and requires the calibration of both the camera and ephemeris models. Currently, the Korea Aerospace Research Institute (KARI) uses image-matching approaches for pan-sharpening image fusion to show mismatch lower than half a pixel in Root Mean Square Error (RMSE) [
12], and seeks a rigorous method to reduce the mismatch.
In this study, we investigate the KOMPSAT-3 CCD alignment, attitude change effects, and terrain elevation effects for the improved co-registration of both MS and PAN CCD images. The MS and PAN CCD lines in the KOMPSAT-3 camera system capture the same ground targets at different times due to CCD line offsets. Therefore, even with accurately estimated sensor alignment parameters, exterior effects associated with ephemeris and terrain features still remain. The offsets of CCD lines due to different view directions and terrain elevation variations can lead to mismatches between MS and PAN images.
We implemented a grid of conjugate points to analyze the mismatch patterns between MS and PAN. A grid of points on MS were generated and projected into PAN for conjugate points. The coordinate differences between the MS and PAN points come from the differences in the sensor alignment, ephemeris effects, and terrain elevation. These effects can be modeled sequentially. First, the differences of conjugate point coordinates are modelled for constant image offsets and linear coordinates differences. The model can compensate for the coordinate differences, which are mainly from the CCD line offsets between MS and PAN. Second, the coordinate differences from satellite attitude changes are modelled because the attitude changes over time of PAN and MS data acquisition can lead to different PAN and MS image coordinates. Third, the impact of terrain elevation variation on the row coordinates’ differences is modelled to estimate the row coordinates differences through ground-projection-based sensor modeling between PAN and MS image. Finally, the PAN and MS images are resampled for pan-sharpening, taking account of all the aforementioned effects.
The paper is structured as follows. In
Section 2, the KOMPSAT-3 AEISS sensor structure and CCD line offsets are presented, including sensor modeling for image projections and conjugate points computation. In
Section 3, the modeling for our rigorous co-registration of both MS and PAN images and its experimental results are presented with respect to CCD line offsets, ephemeris effects, and terrain elevation variations. The conclusion is presented in
Section 4.
4. Conclusions
We studied the effects of CCD offsets, attitude effects, and terrain elevation variation on the KOMPSAT-3 MS and PAN image co-registration. The KOMPSAT-3 CCD lines acquire the same ground features with certain offsets in the corresponding projection centers. Therefore, we compensated for the offsets based on the rigorous physical sensor modeling. The remaining mismatches were corrected using the first- and second-order equations formed from the coordinate difference analysis. It is noteworthy that the attitudes between each CCD data acquisition change when imaging the same target are modelled with a linear combination of roll, pitch and yaw angle changes for the time gap. Row coordinate differences between MS and PAN images from terrain elevation variation can be modelled using a linear compensation equation. Finally, we compensated for all aforementioned effects on the KOMPSAT-3 MS and PAN image co-registrations with negligible discrepancy, less than 0.1 pixels. A rigorous co-registration approach is more robust and useful with available ephemeris data, whereas image matching-based co-registrations are less reliable over a monotonous terrains such as desert and forest. In further studies, more co-registration methods based on image matching can be analyzed to compare physical sensor model approaches.