1. Introduction
Hyperspectral imaging combines imaging technology and spectroscopy technology to obtain sets of data cubes composed of cross-track, along-track, and spectral dimensions. The hyperspectral images (HSIs) contain spatial, spectral, and radiation information, which greatly improves the ability to monitor the Earth system and human activities [
1]. In particular, the high spectral resolution allows for hundreds of bands to identify ground objects with spectral similarity [
2,
3,
4]. It is important for hyperspectral sensors to capture spectral features accurately, because it will affect hyperspectral image applications [
5,
6].
The hyperspectral imaging system mainly includes a line scanning optical imaging system and a push-broom imaging system. Among them, push-broom imaging is widely used in practice because of its simple structure, high signal-to-noise ratio, and high spectral and spatial resolution [
7]. The full spectrum airborne hyperspectral imager (FAHI), China’s new generation push-broom hyperspectral imager, has been applied to civil and environmental monitoring [
8,
9]. The workflow of a push-broom instrument is shown in
Figure 1. Briefly, as the scanner pushes forward along the track, the radiation from substances in a linear field of view is received by the detector array after collimation, dispersion, and convergence of the optical system. Two common optical artifacts occur in this process, which are keystone and smile [
10,
11]. Keystone is caused by the lateral chromatic aberration of the off-axis dispersive beam exiting from the slit edge. It acts in such a way that a straight line is imaged as a curve for a given pixel in spatial dimensions, known as spatial misregistration. Smile results from the dispersion difference of the dispersive element for the beam emitted from different positions of slit. It leads to the curved line on the image sensor for a given wavelength, known as spectral misregistration. Actually, laboratory measurements are available for keystone and smile optimization, and their values are usually specified in the camera specifications as primary optical indicators [
12]. To obtain spectral signatures useful for scientific research, some studies made requirements for keystone and smile. A spectral uncertainty of less than 0.01 of the full width at half maximum throughput of the spectral response function is considered to be necessary, and a maximum level of allowed keystone to be 5% of a pixel is set [
13,
14].
However, this paper focuses on another image artifact faced by push-broom hyperspectral imagers—linear spatial misregistration (LSM) caused by sensor rotation, which has been neglected in the past. It has been mentioned in previous articles, but not in detail [
15]. In addition, LSM was detected in hyperspectral data obtained by Hyperion and SWIR (short wave Infrared) full spectrum imager (SFSI) instruments [
6,
16].
Figure 2 explains the generation of this artifact, where the gray cube denotes the optical plane, and the red grid denotes the rotational detector array. It refers to the rotational misalignment of the sensor to the optical plane and has nothing to do with the whole optical system. The approximation of a linear function of bands provides a reasonable model for LSM detection and correction in HSIs [
6]. There is a problem that keystone also produces spatial misregistration and theoretically affects the detection performance. Considering the following aspects: (1) keystone drives nonlinear spatial misregistration and has a maximum level below 5% of a pixel, as mentioned above; (2) hyperspectral data contain hundreds of bands; and (3) the setting of threshold in our method hinders keystone to some extent, hence, the impact of keystone for LSM detection is considered negligible in our research.
LSM prevents accurate spectrum acquisition. It weakly distorts the spectrum of pixels in the uniform regions and severely distorts the spectrum of pixels in transition areas, which will be verified in our simulation experiments. The distorted spectral curves bring errors to hyperspectral applications, especially to sub-pixel level applications [
17,
18,
19]. In this case, it is necessary to develop LSM detection and correction preprocessing tools to eliminate LSM and recover spectrum, which compensates for the hardware effects though software. In another way, it reduces the requirement for hardware calibration and provides reference for hardware calibration accuracy. In the absence of prior information, such as the spectral signature of a certain substance, the spatial location of an edge is an available feature for subpixel analysis. In past studies, scene-based methods have been proposed for spatial misregistration detection, which are commonly achieved by locating edges at the subpixel level. The correlation-based method is widely used, including the cross-correlation method (CC) and the phase correlation method (PC) [
20,
21,
22,
23]. Robert calculated the Pearson correlation coefficient over a user defined sliding correlation window at the most prominent edge and determined the inter-band subpixel shift by interpolating the correlation values [
16]. Naoto used the normalized cross-correlation method and the phase correlation method to detect spatial misregistration, and the latter provided better detection performance [
6]. Specifically, the phase correlation method determined the peak location of the phase correlation function as misregistration between two bands. Gradient maximization (GM) is also one of the effective ways for subpixel edge detection [
24,
25], which generally determines the subpixel location by fitting pixels of each row and calculating the maximum gradient. Endice improved it by carrying out a weighted sum to refine each location for spatial misregistration detection [
15]. These methods were initially used in grayscale images, so they only exploit the spatial information and ignore spectral information in HSIs. Sometimes, they suffer from instability.
To improve the LSM detection performance, spectral unmixing is adopted for spectral analysis, which has been widely used in subpixel analysis. [
26,
27,
28]. The method of fitting the sum of abundance (FSAM) and the method of searching for equal abundance (SEAM) are developed to detect LSM, which rely on the linear mixture model (LMM) in spectral unmixing and make good use of the spatial and spectral information in HSIs. FSAM is achieved by fitting the sum of abundance of each row in the selected data, and SEAM aims to search for bands with equal abundance for two adjacent pixels. FSAM and SEAM need rectangular transition areas containing clear edges as test subjects. Each side of the edge is composed of pure pixels (endmembers), and the transition area consists of mixed pixels. To get rid of the dependence on scene, the method based on line detection for band-interleaved-by-line (BIL) images (LDBM) and the method based on the Fourier transform of BIL images (FTBM) are produced. They take images in the form of BIL as test data, considering BIL images intuitively reflect the linear change of radiation with wavelength. LDBM directly performs line detection on BIL images, and FTBM characterizes the lines in the frequency domain of BIL images. According to the LSM detection result, cubic spline interpolation is adopted to correct the HSIs. To make it clear, a full three-step strategy is detailed from aspects of data selection, LSM detection, and image correction.
As one of the airborne push-broom imagers, FAHI suffers the problem of sensor rotation due to mechanical instability and hardware adjustment, which leads to varying degrees of LSM in HSIs obtained from each flight mission. In this paper, the hyperspectral data of FAHI is used in our experiments. On the one hand, the performance of the proposed detection methods is tested and compared through simulation experiments. On the other hand, the artifact existing in FAHI is detected, analyzed, and corrected.
The major contributions of this paper are as follows: This paper concentrates on the LSM phenomenon caused by sensor rotation in push-broom hyperspectral imaging systems, proposes novel methods for HSIs to quantify LSM, and details a full strategy to eliminate LSM. The application of the strategy on FAHI verifies its effectiveness. Moreover, the work provides solutions for other push-broom scanners with similar problems and helps to simplify hardware requirements by compensating for hardware effects in software.
2. Materials and Methods
The proposed three-step strategy is used to deal with the spatial misregistration caused by rotational misalignment of the detector, which contains three parts corresponding to data selection, LSM detection, and image correction, as shown in
Figure 3.
2.1. Data Selection
The removal of LSM should be performed on HSIs without radiometric and geometric corrections since the operation of resampling degrades spectral signatures, which is a common premise. The four proposed detection methods have different requirements for experiment data. FSAM and SEAM depend on the availability of edges in the HSIs, while LDBM and FTBM need to convert HSIs into BIL format. These are described separately in the following paragraphs.
For FSAM and SEAM, subregions containing clear edges crossing the along-track direction are first cropped from HSIs as experiment data. To ensure the accuracy of the subsequent endmember’s extraction, it is necessary to check that each side of the edge is a homogenous area covered by one kind of substance. It is better to choose continuous bands with high signal-to-noise ratio and similar scene contrast, which reduce the computational cost on the one hand and improve the detection accuracy on the other hand. Generally, at least three different subregions are selected in each of the left, center, and right field of view (FOV) to enhance the reliability.
For each subregion, endmembers are then extracted on both sides of the edge to prepare for FSAM and SEAM detection. Considering the complexity of surface distribution and the existence of spectral variability [
29,
30], an endmember extraction algorithm is not suitable for this work. For one thing, it is not sure whether there are endmembers in the cropped subregions, especially when the spatial resolution is low. For another, the mixed pixels in the transition area may be the result of the complex action of various ground objects, and using endmember extraction methods based on LMM will bring large errors. In this work, regions of interest (ROIs) are selected on both sides of the edge and take the average of the pixel’s spectrum within ROIs as representative endmembers.
For LDBM and FTBM, the hyperspectral image in BIL format is required, where the horizontal direction represents spatial dimension, and the vertical direction represents spectral dimension. LDBM and FTBM utilize the feature that BIL images can reflect the linear change of radiation with wavelength for LSM detection. There are three ways of storing hyperspectral data, including BIL, band sequential format (BSQ), and band-interleaved-by-pixel format (BIP), which can be converted to each other.
2.2. LSM Detection
Four methods are developed here for LSM detection. Among them, FSAM and SEAM use vectorized LMM in hyperspectral unmixing and perform abundance analysis to obtain the detected LSM. LDBM and FTBM aim to characterize the slope of the structure in BIL images and convert it to LSM.
2.2.1. Methods Based on Hyperspectral Unmixing
To make good use of spatial and spectral information in HSIs, LMM is improved to vectorized LMM. Based on that, two methods are presented. In this part, vectorized LMM, FSAM, and SEAM are introduced successively.
Vectorized LMM
Hyperspectral unmixing aims to decompose mixed pixels into a combination of endmembers, including the process of endmember extraction and abundance estimation [
31]. The former extracts the spectra of endmembers from the hyperspectral image, and the latter determines the abundance of each endmember in mixed pixels. The mixing model provides the basis for hyperspectral unmixing, which contains LMM and the nonlinear mixing model (NLMM) [
32,
33]. In practice, LMM is most broadly used in terms of its simple structure and good performance. LMM uses a linear combination of endmembers weighted by the corresponding abundance to describe pixels.
For a hyperspectral image
with
endmembers, LMM can be expressed as:
where
is the endmember matrix,
is the abundance matrix, and
represents the additive noise matrix. The abundance matrix is required to satisfy two constraints, the abundance non-negativity constraint (ANC):
and the abundance sum-to-one constraint (ASC):
where
is a vector of ones, and
.
Figure 4 shows a selected subregion and the image of one band, where the red line represents the subpixel edge location. Both sides of the edge consist of pure pixels corresponding to two types of spectral signatures, which are denoted by
and
. According to LMM, pixels crossed by the edge are mixed pixels formed by linear combination of the two endmembers. If there is no LSM between bands, a given pixel
can be described as:
where
and
are abundances corresponding to
and
. However, if there is LSM in the hyperspectral image, the abundances are no longer constant but vary with bands. The given pixel
is formulated as:
where
and
are abundance vectors recording the abundance for each band to
and
, respectively. If
and
are known, the abundance vectors can be calculated as:
According to ASC and ANC in LMM,
where
. Perform hyperspectral unmixing on all pixels, and a three-dimensional abundance map is obtained for
or
.
FSAM Detection
FSAM is achieved by fitting the sum of abundance of each row in the selected data. The edge splits each pixel into two parts, where one part is covered by
, and the other part is covered by
. In the abundance map, each pixel can be considered as a square of
, as the pixel value records the area covered by
or
ranging from 0 to 1.
Figure 5 shows a row of the
th band in the abundance map, containing
pixels. The blue line represents the edge in the
th band, and the red line represents the edge in the
th band with offset of
pixel from the blue line. Then, the difference
in the area covered by
or
in these two bands can be represented by
:
where
refers to the left area of the red line,
is the left area of the blue line, is the right area of the blue line, and
refers to the right area of the red line. Use abundance to replace:
where
is the abundance of the
th pixel in the
th band for
. Thus,
is given as:
To reduce the error caused by endmember estimation and noise, calculate for all bands and obtain . Then, a linear fit to according to is performed, where is used to denote the estimated inter-band misregistration for this row. The of all rows in the subregion is averaged as the final LSM caused by rotational misalignment.
SEAM Detection
For two adjacent pixels in a row, SEAM aims to search for two bands with the same abundance. Specifically, for each row in the selected subregion, a middle band with high signal-to-noise ratio is taken to perform the Sobel operator to enhance the edge. Supposing that the th pixel obtains the maximum value, the abundance vectors to or for the ith pixel and the (+1)th pixel are calculated, which correspond to and .
For
in
,
could be found in
and satisfies:
where
ε is a small constant. It is set as 0.005 in our experiments. To improve the robustness, it is necessary to limit the band range:
where
is an integer. For different band offset, the optimal value of
is different.
Figure 6 depicts this case. The blue line represents the subpixel edge in the
th band of the row, and the red line represents the subpixel edge in the
th band with an offset of 1 to the blue line. The blue area and the red area denote
and
—the abundance of the
ith pixel and the (
+1)th pixel to endmember
or
.
For the th pixel, multiple can be obtained. The average of the multiple as an offset for this row and the average of offsets for all rows are taken as the final LSM for the subregion.
2.2.2. Methods Based on BIL Images
Though FSAM and SEAM take advantage of spatial and spectral information in HSIs, they rely on the availability of edges in the image between homogeneous areas. To get rid of the dependence on scene, two methods based on slope detection are proposed, which are simple to implement on BIL images.
LDBM Detection
The BIL image contains both spatial and spectral information in a linear field of view. The LSM phenomenon is intuitively reflected in BIL images, leading the texture to skew to a certain angle. Thus, an idea for LSM detection is to perform line detection on the BIL image, and the slope of the detected line is taken as the LSM result. The flowchart of LDBM is shown in
Figure 7a.
Canny operator is widely used for edge detection because of its high precision. Due to the influence of spatial resolution and the existence of mixed pixels, the edges in BIL image are blurred. In LDBM, for a BIL image, the canny operator is first adopted to enhance the edges. Then, the Hough transform is used to recognize the line features. Based on the voting mechanism, the lines corresponding to the top voting peaks are selected as effective detection results. Parameter is set as 3, 5, or 7 in our experiments. The coordinates of the head and end points of each line are given by the Hough transform, through which the slope of the line can be calculated. Finally, take average of the slopes of the effective lines as LSM.
FTBM Detection
The rotation component of an image can be clearly displayed in the frequency domain, specifically, presenting strips inclined at a certain angle. So FTBM is determined, which tries to recognize the slope in the Fourier transform of BIL images. The flowchart of FTBM is shown in
Figure 7b.
For a BIL image, fast Fourier transform is first needed to convert it to the frequency domain. Then, the canny operator is applied to extract the edge information. The continuous line structure is characterized by the Hough transform. The lines with the first (3, 5, or 7) voting peaks are selected. Finally, average the slopes of the selected lines as the final LSM.
In LDBM and FTBM, the canny operator, Hough transform, and Fourier transform are realized through functions in MATLAB 2019a, which make the methods simple and fast to achieve. FSAM and SEAM are more complicated to conduct than LDBM and FTBM. But FSAM has the potential to be applied to nonlinear spatial misregistration detection; the other three are mainly used for linear detection.
2.3. Image Correction
After detecting LSM, the image should be corrected according to detected result. As the smaller the and the larger the detection error, a threshold is set to compare with . If is less than the threshold, the effect of LSM is ignored. Contrarily, the hyperspectral image needs to be corrected. Based on the detection results in the simulation experiments, the threshold is set to be 0.005.
The correction method is based on cubic interpolation because of its good trade-off between smoothness and shape preservation [
34]. Specifically, a reference band is first selected, generally in the middle band with high signal-to-noise ratio. Then, the offset of other bands to the reference band is calculated:
where the reference band is the
th band, and
is the offset of the
th band to the
th band. Finally, perform interpolation on each band to correct the image.
3. Results and Discussion
In this Section, experiments on simulation data and real data are conducted to assess the performance of the proposed methods and the three-step strategy. In experiments on simulation data, the accuracy and the adaptability to different spatial and spectral resolutions of FSAM, SEAM, LDBM, and FTBM are mainly tested with PC, CC, and GM as comparison. In experiments on real data, the full strategy is applied to real data of FAHI, aiming to analyze the artifact in FAHI and prove its feasibility and effectiveness.
3.1. Experimental Data
The hyperspectral data obtained by FAHI is used in our experiments [
35,
36]. To evaluate the detection ability for LSM of the proposed methods, we conducted experiments on simulation data. Since whether LSM exists or its value cannot be determined in advance, a sub-image with a detected result less than the threshold was selected as original data. Its size is 500 × 500, and it contains 90 bands covering a spectral range from 1500 to 1800 nm. The spatial resolution is 0.5 m, and the spectral resolution is 3.2 nm. To induce LSM, the band offsets of 0.01, 0.03, and 0.05 of a pixel were set on it using bilinear interpolation functions, respectively, with the first band as the reference band. Then, three simulated datasets were obtained.
Figure 8 shows the selected sub-image and the local comparison with different offsets. The larger the offset, the more obvious the artifact. Finally, three subregions containing clear edges were cropped from simulated datasets for LSM detection, as shown in
Figure 9.
To verify the effectiveness of the full strategy, experiments were carried out on HSIs of FAHI. For each hyperspectral image, three subregions in each of the left, middle, and right FOV were selected to enhance the reliability of the detection results. The results of an image were presented.
3.2. Evaluation Metrics
To evaluate the LSM detection performance of the proposed algorithms, the following two metrics are used:
1. Absolute error (AE):
where
is the real offset, and
is the detected offset. AE directly reflects the detection error.
2. Mean root mean square error (MRMSE):
where
refers to the fitting vector of all bands in the
th row,
refers to the detected vector of all bands,
is the number of bands, and
is the number of rows. MRMSE indicates the stability of the test algorithm. The smaller the values are of AE and MRMSE, the better the detection results are.
To measure the improvement of the image quality after LSM correction, the following three metrics are calculated [
37,
38]:
1. Peak signal to noise ratio (PSNR):
where
is the maximum value of the image, and
is the mean square error of two images. PSNR is the most widely used image quality evaluation indicator based on the error between corresponding pixels.
2. Structural similarity (SSIM):
where
and
are constants,
and
are the mean gray values of two images,
and
are standard deviations of two images, and
is the covariance. SSIM measures the image similarity from brightness, contrast, and structure.
3. Spectral angle mapper (SAM):
where
and
correspond to the spectral vectors of pixels at the same location in two hyperspectral images. SAM is an important indicator for measuring changes in spectral features, since the spectra determine the identification of substances. The larger the values are of PSNR and SSIM and the smaller the value of SAM, the better the image quality is.
3.3. Experiments on Simulation Data
In this section, four sets of experiments are carried out, aiming at comparing the LSM detection performance of all test algorithms, measuring the change of image quality after LSM correction, testing the adaptability to HSIs with different spatial resolution of the proposed methods, and testing the adaptability to HSIs with different spectral resolution of the proposed methods, respectively.
3.3.1. Performance Comparison of Detection Methods
Experiments on simulation data are conducted to assess the performance of the proposed FSAM, SEAM, LDBM, and FTBM with PC, CC, and GM as comparisons. All test methods are performed on the three subregions, as shown in
Figure 9, to detect the LSM. The results of detected
, AE, and MRMSE are shown in
Table 1. Since SEAM, LDBM, and FTBM do not fit all bands, their MRMSE cannot be calculated; only
and AE are available.
From the experimental results: (1) for each subregion, the values of AE obtained by all test algorithms are all within 0.01; (2) for each subregion, compared with SEAM and FTBM, the proposed FSAM and LDBM obtain lower values of AE, with the lowest for FSAM, which means they have higher detection accuracy; (3) during the experiments, it is found that SEAM and FTBM are less stable as their detection performance overly depends on parameter adjustment, such as an angle range, that is very close to the real situation and needs to be set in the Hough transform function in LDBM and FTBM to maintain a good detection performance; (4) for each subregion, in comparison methods, GM provides relatively lower values of AE, indicating it brings smaller detection errors compared to CC and PC; (5) the MRMSE results exhibit a significant difference for each subregion, and this demonstrates the varying degrees of stability of the algorithms; (6) though GM obtains lower AE, its MRMSE results are highest for different levels of the band shift, showing its sensitivity to noise; (7) FSAM obtains the lowest MRMSE, so it is more stable compared to CC, PC, and GM; (8) compared with other methods (including CC, PC, GM, SEAM, LDBM, and FTBM), FSAM shows a smaller detection error and better stability and is more suitable for LSM detection; (9) the consistency of AE and MRMSE for the three subregions also verifies the capability of FSAM to detect LSM; (10) in addition, it seems that subregion1 obtains larger MRMSE, and subregion3 obtains smaller MRMSE. It is related to the complex distribution of vegetation areas such as different vegetation densities.
In conclusion, the proposed FSAM outperforms other methods (including CC, PC, GM, SEAM, LDBM, and FTBM) in terms of detection error and stability, and the detection error of FSAM is within 0.005.
3.3.2. Improvement of Image Quality
The average results of the three subregions obtained by FSAM are used to correct the sub-image, which are 0.0098, 0.0302, and 0.0504. To measure the improvement of the image quality after LSM correction, PSNR, SSIM, and SAM are calculated, and the results are shown in
Figure 10,
Figure 11 and
Figure 12.
From the results, the impact of LSM on image quality is quantified: (1) for a certain offset, the farther away it is from the reference band, the worse the band quality is; (2) for a different offset, the larger the offset is, the worse the image quality is; (3) for offsets of 0.01, 0.03, and 0.05, PSNR decreases to 31.8, 26.6, and 24.4, respectively; SSIM decreases to 0.97, 0.90, and 0.84, respectively; (4) SAM increases significantly, which means the spectra deviate from the ground truth; (5) results of SAM also show that the spectral distortion in the edge is larger than that in the homogeneous region.
After LSM correction, the image quality is improved: (1) for a certain offset, PSNR, SSIM, and SAM are all improved compared with those before LSM correction; (2) for offsets of 0.01, 0.03, and 0.05, PSNR of the last band increased to 34.4, 29.4, and 27.5, respectively; SSIM of the last band increases to 0.98, 0.95, and 0.92, respectively; (3) SAM decreases greatly, thus the spectral curves move closer to the real curves; (4) on the whole, the larger the offset, the more the image quality is improved; (5) though the image is recovered, the image with a large offset is still inferior to the image with a small offset.
In conclusion, LSM degrades hyperspectral quality, and the proposed detection and correction is effective for dealing with the problem, as it improves the image quality and brings recovered spectral features.
3.3.3. Adaptability to HSIs with Different Spatial Resolution
To verify the adaptability of the proposed methods to hyperspectral data with different spatial resolution, experimental data with spatial resolution of 1 m and 2 m are obtained by applying pixel merging to the original data [
39]. Then, the inter-band offset is set to 0.01, 0.03, and 0.05 of a pixel. The detection results of all test methods on subregion 1, subregion 2, and subregion 3 are shown in
Table 2 and
Table 3.
From the experimental results: (1) for each subregion, the values of AE obtained by all test algorithms remain below 0.01, among which, FSAM offers lower values of AE; (2) compared to the results in
Table 1, with the decrease in spatial resolution, AE of CC, PC, GM, FSAM, and SEAM show weak fluctuation, indicating the methods keep their performance for LSM detection; (3) relatively, AE of LDBM and FTBM have clear fluctuation, indicating they are affected by spatial resolution, as the lower the spatial resolution is, the blurrier the edges in BIL images are; (4) for each subregion, GM obtains the highest values of MRMSE, and FSAM and PC obtain lower values of MRMSE, which shows that FSAM and PC are more stable than GM and CC; (5) compared to the results in
Table 1, with the decrease in spatial resolution, MRMSE values of test algorithms decrease significantly, and it is mainly because the images with high spatial resolution can display rich details of ground objects; (6) the MRMSE results also indicate that the algorithms have different stability at different levels of spatial resolution; (7) FSAM keeps lower MRMSE to HSIs at different spatial resolution, which shows the FSAM is more robust to spatial resolution changes compared with CC, PC, and GM.
In conclusion, the spatial resolution mainly affects the detection accuracy of LDBM and FTBM, and the algorithm stability of CC, PC, GM, and FSAM. FSAM shows better adaptability to HSIs with spatial resolution from 0.5 m to 2 m in terms of detection accuracy and stability, and the absolute detection error is kept within 0.005.
3.3.4. Adaptability to HSIs with Different Spectral Resolution
To verify the adaptability of the proposed methods to hyperspectral data with different spectral resolution, experimental data with spectral resolution of 6.4nm and 9.6nm are obtained by applying band merging to the original data [
39]. Then, the inter-band offset is set to 0.01, 0.03, and 0.05 of a pixel. The detection results of all test methods on subregion 1, subregion 2, and subregion 3 are shown in
Table 4 and
Table 5.
From the experimental results: (1) for each subregion, the values of AE obtained by all test algorithms are below 0.01, and FSAM still provides detection error less than 0.005; (2) compared to the results in
Table 1, with the decrease in spectral resolution, AE of the test algorithms shows an increasing tendency in fluctuation overall, demonstrating the detection accuracy of most algorithms is reduced; (3) relatively, the AE values obtained by FSAM, PC, and GM show little change, which indicates that FSAM keeps better ability to detect LSM at the spectral resolution of 6.4nm and 9.6nm; (4) AE of LDBM, FTBM, and CC increase significantly, which represents they are easily affected by spectral resolution; (5) for each subregion, GM obtains higher values of MRMSE, and FSAM still keeps lower values of MRMSE, showing the good stability of FSAM; (6) compared to the results in
Table 1, with the decrease in spectral resolution, MRMSE shows slight fluctuation, meaning the stability of CC, PC, GM, and FSAM remains steady with the change of spectral resolution.
It is concluded that the spectral resolution affects the detection accuracy of most test methods. The higher the spectral resolution is, the more number of bands there are, and the higher the detection accuracy. Considering detection error and algorithm stability, FSAM provides better adaptability to HSIs with spectral resolution from 3.2 nm and 9.6 nm compared with other methods, and the absolute detection error remains within 0.005.
3.4. Experiments on Real Data
In this section, the results of a hyperspectral image obtained by FAHI are present. Three subregions are selected in each of the left, middle, and right FOV of the image, and the proposed FSAM is utilized to detect LSM on the subregions. The results of detected
are shown in
Table 6.
Results detected for all subregions are all around 0.03 with slight fluctuation (
Table 6), which demonstrates the sensor rotation existing in FAHI instrument. The average of the results is used to correct the image, and qualitative analysis is implemented here. The buildings, plants, and seaside areas before and after correction are compared in
Figure 13. In the original sub-images, there is noticeable ghosting phenomenon caused by spatial misregistration, especially in the transition areas involving two or more substances. In the corrected sub-images, this phenomenon is greatly reduced, and the transition regions become sharp and clear. The image in BIL format also reflects the change before and after correction, where the horizontal direction represents spatial dimension, and the vertical direction represents spectral dimension. The BIL images of the selected two lines are shown in
Figure 14. As can be seen, there are stripes inclined at a certain angle to the spectral dimension in the original images, and this angle is significantly reduced after LSM correction. The application on FAHI proves the effectiveness and the feasibility of the full strategy.
4. Discussion
Overall, the results of the selected subregions show consistency, which enhances the reliability of detection. For the proposed methods, FSAM and SEAM are more complex to conduct, since they depend on the availability of edges in the homogenous area. LDBM and FTBM are easier to perform, as they are based on BIL images. From the scope of application, FSAM calculates the offset for each band and can be used for nonlinear detection, while the other three are only used for LSM detection. In terms of detection accuracy, FSAM and LDBM perform better than SEAM and FTBM. SEAM searches for bands with equal abundance, thus it has a high requirement on spectral unmixing. The line structure in the frequency domain after the Fourier transform is nearly covered by other complex frequency components, resulting in the large error in FTBM. In terms of stability, it is found that the performance of SEAM, LDBM, and FTBM is easily affected by parameter setting. In SEAM, the parameter is set differently according to the simulated inter-band shift. A rotation angle range that is close to the real value needs to be set in LDBM and FTBM, and a range less than 2° achieves better accuracy. For comparison methods, GM offers small absolute error but poor stability, while CC and PC balance on both. Though CC, PC, and GM obtain the offset for each band, it is inferred that CC and PC can achieve high precision on nonlinear spatial misregistration detection.
In addition, LDBM and FTBM are more sensitive to the decrease in spatial resolution, owing to the blurred edge information in BIL images at low spatial resolution. The other methods keep their accuracy and obtain better fitting stability, which is mainly because the decreased spatial resolution leads to the loss of detailed information. All methods demand high spectral resolution to realize good performance. The higher the spectral resolution, the more number of bands there are, and the higher the detection accuracy is.
Considering accuracy and stability, FSAM performs better in LSM detection compared to other methods. In the field of linear detection, FSAM, LDBM, and GM can be selected for adoption. In nonlinear detection, FSAM, CC, and PC show their advantages. Moreover, for HSIs with spatial resolution and spectral resolution set in our experiments, the detection error of all methods is less than 0.01 of a pixel, especially if that of FSAM is less than 0.005 of a pixel, which simplifies the requirement for lab calibration and provides reference value for the calibration accuracy.
5. Conclusions
In push-broom hyperspectral imagers, the sensor rotation to optical axis leads to LSM in HSIs. This paper develops methods (including FSAM, SEAM, LDBM, and FTBM) to detect LSM and details a full strategy to eliminate LSM. It facilitates the analysis of the sensor rotation degree in push-broom scanners and, more importantly, reduces the hardware demand by compensating for hardware effects in software. Specifically, the experimental data is first selected. For FSAM and SEAM, subregions with distinct edges in each of the left, center, and right FOV are selected, and representative spectrum are extracted as endmembers for each subregion. For LDBM and FTBM, the hyperspectral image needs to be converted to a BIL image as input. Then, the detection method is applied to evaluate LSM. FSAM and SEAM conduct abundance analysis for the abundance map after spectral unmixing, while LDBM and FTBM aim to recognize the slope of the line structure. Finally, cubic interpolation is performed on each band to correct the image according to the detected result.
The proposed methods are tested by experiments on simulation data with CC, PC, and GM as comparison. For HSIs with spatial resolution of 0.5 m, 1 m, and 2 m, and spectral resolution of 3.2 nm, 6.4 nm, and 9.6 nm, all methods ensure that the absolute detection error is less than 0.01. Though FSAM is more complicated to implement, it outperforms other methods in terms of accuracy and stability with absolute error less than 0.005 and shows good adaptability to the change of spatial resolution and spectral resolution. The sensor rotation existing in FAHI is detected through the proposed strategy in experiments on real data, and the image is effectively calibrated, which provides reference for other push-broom hyperspectral instruments with similar problems.
Though several methods have been proposed to detect LSM, it still leaves much to be desired. The algorithm accuracy determines the application value. Under the premise of accuracy improvement of FSAM, nonlinear spatial misregistration detection of band-by-band will contribute to the joint analysis of keystone and sensor rotation.