A Superpixel-by-Superpixel Clustering Framework for Hyperspectral Change Detection

Abstract

Hyperspectral image change detection (HSI-CD) is an interesting task in the Earth’s remote sensing community. However, current HSI-CD methods are feeble at detecting subtle changes from bitemporal HSIs, because the decision boundary is partially stretched by strong changes so that subtle changes are ignored. In this paper, we propose a superpixel-by-superpixel clustering framework (SSCF), which avoids the confusion of different changes and thus reduces the impact on decision boundaries. Wherein the simple linear iterative clustering (SLIC) is employed to spatially segment the different images (DI) of the bitemporal HSIs into superpixels. Meanwhile, the Gaussian mixture model (GMM) extracts uncertain pixels from the DI as a rough threshold for clustering. The final CD results are obtained by passing the determined superpixels and uncertain pixels through K-means. The experimental results of two spaceborne bitemporal HSIs datasets demonstrate competitive efficiency and accuracy in the proposed SSCF.

1. Introduction

1.1. Background of Hyperspectral Change Detection

Change detection (CD) is a sensing task that analyzes the bitemporal or multitemporal images for the identification of the changed scene over time. Recently, lots of satellite missions carrying hyperspectral sensors have been launched consecutively, which means hyperspectral images (HSIs) have become an important data source for Earth observation. The abundant spectral information can boost target detection [1,2,3], anomaly detection [4,5,6], and classification [7,8,9]. Binary hyperspectral image CD (HSI-CD) is a special task in which the final change map (change map) reflects the change or not at the pixel level, i.e., zeros denote unchanged regions, and ones indicate changed regions. Such tasks can be applied to disaster assessment [10,11], agriculture and forestry monitoring [12,13], urban expansion research [14,15], etc. The HSI-CD task consists of three steps: data preprocessing, change identification, and change map output and evaluation. Among them, change identification is the most important and challenging step. The changes can be divided into strong and subtle changes according to the change intensity [16]. Strong changes are associated with bitemporal HSIs that have significantly different spectral features. In contrast, subtle changes just have small differences in spectral features between the bitemporal HSIs. For example, during the transition of land cover from bare land to crop, different water contents or different growth rates of the crops indicate different changes, where the lower water content or slower growth rate corresponds to subtle changes. Furthermore, subtle changes may be induced by mixed pixels that are usually present in the edge areas of the HSIs, because the spatial resolution of HSIs is limited. The changes of partial endmembers in the mixed pixels belong to subtle changes. However, the detection of subtle changes is challenging. Since HSIs can provide intensive sampling of spectral features over a wide spectral range, it is possible to accurately monitor changes at fine spectral scales. That is, HSIs have the advantage of being able to characterize subtle changes.

To exploit changes, kinds of methods have been proposed, including supervised and unsupervised methods. The former ones are limited by the availability of ground truths. Contrarily, the later ones do not require any a priori information and have aroused wide attention. Furthermore, the accurate detection of subtle changes without a priori data and with a lower false alarm rate is an interesting and meaningful task for HSI-CD. Therefore, we focus on unsupervised methods. Unsupervised binary CD methods that are applicable for HSI-CD can be generally classified into four types: (a) image algorithm-based methods [17], (b) image transform-based methods [18], (c) HSI-CD specified methods [19], and (d) deep learning-based methods [20].

1.1.1. Image Algorithm-Based Methods

The image algorithm-based methods assume that changes lead to significant differences in gray pixel levels and thus directly perform algebraic operations on bitemporal HSIs to determine pixel changes. The simple and commonly used arithmetical operations are image subtraction [21], image regression [22], and image rationing [23]. One typical algorithm is the change vector analysis (CVA) [24], which uses spectral vector subtraction to analyze the differences in the spectral bands. Recently, some modified CVA algorithms have also been proposed [25,26]. Structural similarity (SSIM) is also introduced into the image similarity measurement based on structural information degradation [27] and then used for the HSI-CD in [28]. These methods directly detect pixel pairs independently, which makes them sensitive to noise and misalignment errors.

1.1.2. Image Transform-Based Methods

Image transformation-based methods transform images into a specific feature space to emphasize changed pixels and suppress unchanged ones. Principal component analysis (PCA) [29] is a common algorithm for dimensionality reduction. Nielsen et al. [30] proposed a multivariate alteration detection (MAD) method based on typical correlation analysis, which used linear transformations of bitemporal HSIs to maximize changes. MAD has been successfully applied to vegetation monitoring in HSIs [31]. Iterative reweighted MAD (IR-MAD) [32] is an expanded version of MAD in iterative form. In addition, the slow feature analysis (SFA) method extracts slowly changing features from a time series [33,34]. It can be used for HSI-CD by suppressing unchanged features and highlighting the changed features [35]. Iterative SFA (ISFA) [36] assigns high weights to invariant pixels during iteration so that they can play a greater role in feature extraction.

1.1.3. HSI-CD Specified Methods

Recently, many CD methods have been proposed specifically for HSI-CD. Chen et al. [37] proposed an HSI-CD model based on spectrally and spatially regularized low-rank and sparse decomposition. It improved the already established low-rank and sparse decomposition to implement HSI-CD. Wu et al. [38] proposed an HSI anomalous CD method based on joint-sparse representation. In this method, the background dictionary is constructed by randomly selecting background pixels from the image. Then, it uses the constructed background dictionary to capture changes. In addition, spectral unmixing is also widely used in the implementation of HSI-CD [39,40,41,42,43]. These methods use spectral unmixing to determine whether the pixel changes directly or indirectly.

The tensor decomposition reconstruction detector (TDRD) HSI-CD method [44] implements a Tucker decomposition and reconstruction strategy for bitemporal HSIs to form new HSIs with increased separability. A novel patch tensor-based CD method (PTCD) [45] considers the non-overlapping local similarity property to make full use of the spatial structure information of bitemporal HSIs. In [46], MaxtreeCD is first proposed to exploit multiple morphological attributes to fully explore the spatial information, then a spectral angle weighted-based local absolute distance (SALA) is designed to determine the spectral change. It is found that MaxtreeCD can detect the complete changes and have good detection performance.

1.1.4. Deep Learning-Based Methods

Deep learning has swept across the field of remote sensing image interpretation due to the significant advantages in deep feature representation and nonlinear problem modeling [47,48,49,50]. For unsupervised HSI-CD, the pseudo-labels generated with unsupervised model-driven methods are usually used for training an artificial neural net (ANN). Li et al. [51] proposed a noise modeling-based unsupervised HSI-CD framework, in which the noise model is used to purify pseudo-labels for the end-to-end training process. Song et al. [52] proposed an HSI-CD architecture based on a recurrent 3D fully convolutional network, in which the pseudo-labels were generated by principal component analysis (PCA) and spectral correlation angle (SCA). Wang et al. [53] proposed a general end-to-end 2-D CNN (GETNET) HSI-CD framework, in which mixed-affinity matrices were formed, and features were extracted for classification. The pseudo-labels of the GETNET were produced by CVA. Du et al. [54] proposed a DSFA framework that extracted unchanged paired pixels from the CVA as training samples. The two trained ANNs were used to transform the bitemporal images separately. The invariant pairwise pixels were suppressed, and the changed pairwise pixels were highlighted using SFA constraint. Li et al. [28] proposed an improved pseudo-label generation mechanism that utilized CVA and SSIM to jointly guide the pseudo-label generation, which can be called the self-generated credible labels method (SGCL). In this case, the simple ANN with a single convolution layer can obtain accurate CD results. However, the generalization of the method needs to be improved because it cannot achieve desirable results on complex datasets with various changes, and the quality of the pseudo-label depends on both CVA and SSIM. Sun et al. [55] designed a new population confidence-based sample selection method to extract better quality and diverse pseudo-labels. However, the method is time-consuming. The image difference (ID) algorithm and spectral unmixing (SU) manner were also used to generate pseudo-training data [56], and the performance of this method is heavily dependent on the quality of spectral unmixing. In addition, some methods do not rely on pseudo-labels to achieve unsupervised HSI-CD [57,58,59] but exploit the characteristics of HSIs and the power of neural networks.

1.2. Problem Statements

The detection of subtle changes is challenging. Subtle changes arise from changes in similar species or mixed image elements. In general, subtle changes refer to changes with small differences in spectral features, and such changes are easy to be ignored. Moreover, the intensity of spurious changes caused by spectral susceptibility to distortion is similarly small. Therefore, it is difficult to correctly detect subtle changes while suppressing false changes. In [60], the results of some HSI-CD methods based on real HSIs datasets were compared. It is found that the methods without a priori knowledge are hard to detect subtle changes.

The performance of unsupervised data-driven CD methods such as deep learning is limited by the accuracy of pseudo-label. Moreover, deep learning CD methods are difficult to apply practically due to the high computational resource requirements and high time costs. In contrast, for model-driven CD methods, the decision boundary is susceptible to strong changes. As a result, subtle changes are ignored. Specifically, the model-driven CD methods do not distinguish between strong changes and subtle changes during detection, so the threshold for capturing subtle changes is subject to the presence of strong changes. In this case, subtle changes with a change intensity lower than the threshold are ignored. Therefore, an effective model-driven CD method is expected to detect the subtle changes specific to HSI.

1.3. Contributions of the Paper

In this paper, HSI-CD is specifically considered for a clustering task. To detect the subtle changes accurately, we propose a concise and effective hyperspectral change detection framework named SSCF. In SSCF, a single superpixel is the data to be detected, and the uncertain pixel from GMM can be considered as a rough threshold, and then a comparison between the two is implemented using K-means. The main contributions are summarized as follows.

• We propose ingenious strategies to achieve the detection of subtle changes. SLIC in SSCF can spatially segment different changes according to the intensity of the changes, which greatly increases the possibility of subtle changes being detected. GMM in SSCF separates uncertain pixels from the whole image as a rough threshold to highlight changes and accurately capture subtle changes. Consequently, the experimental results show that SSCF is able to detect subtle changes without increasing the false alarm rate.
• Compared with the existing traditional methods, SSCF can achieve more accurate detection. Compared with deep learning methods, SSCF achieves higher accuracy and shorter detection time.

The rest of the paper is organized as follows. Section 2 presents the proposed methodology. Experiments and discussions are given in Section 3. Finally, Section 4 concludes the paper.

2. Methodology

The structure is shown in Figure 1. The input is the bitemporal HSIs, T₁ and T₂, and the output is the change map. SSCF consists of four key parts: (1) the difference image (DI) is obtained according to T₁ and T₂, (2) all superpixels are acquired using SLIC, (3) uncertain pixels are obtained using GMM as a rough threshold, and (4) K-means clustering is used to obtain CD results for individual superpixels. The number of clustering is equal to the number of superpixels, and all results from all superpixels are assembled into a complete change map.

In this section, we describe, in detail, the basic principles of SLIC and GMM, as well as the process of SSCF.

2.1. SLIC

Superpixel segmentation can divide an image into non-overlapping superpixels, and each superpixel consists of a set of pixels with a similar color or other low-level cues. It is a useful preprocessing step in various HSIs processing and computer vision applications. Among the various superpixel algorithms, the simple linear iterative clustering (SLIC) algorithm [61] outperforms others in terms of computational complexity, storage efficiency, boundary consistency, undivided error, and boundary error. Specifically, SLIC provides a tradeoff between superpixels compactness control and boundary consistency.

SLIC is a superpixel segmentation algorithm inspired by K-means clustering. It is first introduced for RGB images, which converts the image from RGB color space to CIELAB. Then SLIC divides the CIELAB image into non-overlapping regions, which are controllable in size and compactness.

The first step is initialization. We initialize B clustering centers in a regular grid. To make the size of each superpixel consistent, the grid spacing should be $S=\sqrt{N/B}$, where N is the number of pixels and B is the number of superpixels. Then the gradient values of all pixels in the neighborhood 3 × 3 are calculated, and the clustering center is moved to the place with the smallest gradient in that neighborhood. This operation can avoid the clustering center falling on the contour boundary with a large gradient, which may affect the subsequent clustering effect.

The second step is assignment. This step assigns clustering centers to pixels. The desired superpixel size is set as S × S, and the search area is limited to 2S × 2S around the cluster center. By evaluating the specific distances of all pixels from the cluster center in the search area, the pixels are then specified to the nearest cluster center. The limited search area instead of all of the area is the key to speeding up the efficiency of the SLIC segmentation method. This is the significant difference between SLIC and K-means, where the search area of K-means clustering is not limited, i.e., each pixel must be compared with all cluster centers. Let the pixel of the image be expressed as $I={\left[LAB,c,r\right]}^{\mathrm{T}}$, where $LAB=\left[l,a,b\right]$ represents the CIELAB color space and $\left[c,r\right]$ denotes the spatial position. Then, two distance s between pixel i and cluster center (pixel j) are defined as:

$$d_c=\sqrt{{\left(l^{\left(j\right)}-l^{(i)}\right)}^2+{\left(a^{\left(j\right)}-a^{\left(i\right)}\right)}^2+{\left(b^{\left(j\right)}-b^{\left(i\right)}\right)}^2},$$

(1)

$$d_s=\sqrt{{\left(c^{\left(j\right)}-c^{\left(i\right)}\right)}^2+{\left(r^{\left(j\right)}-r^{\left(i\right)}\right)}^2,}$$

(2)

where d_c and d_s are color distance and spatial distance, respectively. The two distances are normalized and combined into the total distance D’:

$$D^{\prime }=\sqrt{{\left(\frac{d_c}{w}\right)}^2+{\left(\frac{d_s}{S}\right)}^2},$$

(3)

where S becomes the maximum spatial distance, and w is a constant indicating the maximum color distance. Then, D′ is simplified to the specific distance:

$$D=\sqrt{d_c^2+{\left(\frac{d_s}{S}\right)}^2{w}^2}.$$

(4)

In this case, w is allowed to measure the relative importance between color similarity and spatial proximity. If compactness w is larger, the spatial proximity is more important, and the resulting superpixels are more compact (i.e., they have a lower area to perimeter ratio). If compactness w is smaller, the resulting superpixels adhere more tightly to the image boundaries but do not have a regular size and shape. When using the CIELAB color space, compactness w can be in the range 1–40. A detailed interpretation of D′ can be found in [61].

The third step is update. The second step has grouped all pixels into their corresponding superpixels based on distances. However, once each pixel is assigned to the nearest cluster center, it is necessary to update the cluster center. Therefore, the third step needs to calculate the new cluster centers. The new cluster center is the average of all pixels’ vectors specified to the previous cluster center. L2norm is used to compute the residual E between the new cluster center and the previous cluster center. The assignment and update steps can be repeated iteratively until the error converges. Finally, the post-processing step enhances the connectivity by reassigning the disconnected pixels to nearby superpixels.

2.2. GMM

The Gaussian mixture model (GMM) is an effective statistical model that can be used for clustering data into several finite components. It assumes that each component belongs to a Gaussian distribution, and the whole components become a Gaussian mixture distribution. GMM is first proposed for background subtraction [62]. Thus far, it has been used in the field of image processing.

GMM assumed that an image is a mixture of K Gaussian distributions. Let each pixel on the image be expressed as $x={\left\{x_m\right\}}_{m=1}^M$ over M bands. Then the assembly of all N pixels on the image is $X={\left\{x^{(i)}\right\}}_{i=1}^N$. In this case, the probability density function of $x^{(i)}$ is modeled as

$$p\left(x^{(i)}|{\mathsf{\theta }}_k\right)={\displaystyle \sum _{k=1}^K{\alpha }_k\varphi \left(x^{(i)}|{\mathsf{\theta }}_k\right)}={\displaystyle \sum _{k=1}^K\frac{{\alpha }_k}{{\left(2\pi \right)}^{\frac{M}{2}}{\left|{\Sigma }_k\right|}^{\frac{1}{2}}}}\exp \left\{-\frac{{\left(x^{(i)}-{\mu }_k^{(i)}\right)}^{\mathrm{T}}{\Sigma }_k^{-1}\left(x^{(i)}-{\mu }_k^{(i)}\right)}{2}\right\},$$

(5)

where ${\alpha }_k$ is the mixture parameter that satisfies ${\alpha }_k\ge 0$ and ${\displaystyle \sum _{k=1}^K{\alpha }_k=1}$, and ${\mathsf{\theta }}_k=\left({\mu }_k,{\Sigma }_k\right)$ consists of the mean ${\mu }_k$, and covariance matrix ${\Sigma }_k$ of the kth Gaussian distribution $\varphi \left(x^{(i)}|{\mathsf{\theta }}_k\right)$. The parameters $\Theta ={\left\{{\alpha }_k,{\mathsf{\theta }}_k\right\}}_{k=1}^K$ determine the joint probability of the assemble $X$ using the likelihood function

$$L\left(X|\mathsf{\Theta }\right)={\displaystyle \prod _{i=1}^N{\displaystyle \sum _{k=1}^K{\alpha }_k\varphi \left(x^{(i)}|{\mathsf{\theta }}_k\right)}}.$$

(6)

Correspondingly, the log-likelihood expression is

$$\ln L\left(X|\mathsf{\Theta }\right)={\displaystyle \sum _{i=1}^N\ln \left({\displaystyle \sum _{k=1}^K{\alpha }_k\varphi \left(x^{(i)}|{\mathsf{\theta }}_k\right)}\right)}.$$

(7)

Then, $\Theta$ is estimated with maximum likelihood estimate (MLE) as

$${\Theta }^{*}=\arg \underset{\Theta }{\max }\left\{\ln L\left(X|\Theta \right)\right\}.$$

(8)

The expectation-maximization (EM) algorithm is effective for the MLE of mixture models. It can fit the GMM to the observed data by maximizing the log-likelihood function with respect to the parameters of the model (mean, covariance, and prior probability). The EM algorithm is based on the interpretation of $X$ is incomplete data. Herein, the affiliation of $x^{(i)}$ to the specific GMM components is unknown. We let this incompletion be a set of latent vectors $Z={\left\{z^{(i)}\right\}}_{i=1}^N$ with $z^{(i)}={\left[z_k^{(i)}\right]}_{k=1}^K$. If $x^{(i)}$ is generated from the kth component, $z_k^{(i)}=1$, otherwise $z_k^{(i)}=0$. Then the log-likelihood of the complete data $Y=\{X,Z\}$ is defined as

$$\ln L(X,Z|\Theta )={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{k=1}^K{z}_k^{(i)}\ln \left({\alpha }_k\varphi \left(x^{(i)}|{\mathsf{\theta }}_k\right)\right)}}.$$

(9)

Finally, the EM algorithm iterates both the E-step and the M-step until the likelihood function converges to a certain optimal value.

E-step:

Due to the existence of the latent variable $Z$, a conditional expectation is defined

$$Q\left(\Theta ,{\Theta }^{(t)}\right)=E\left[\left.\ln L\left(\left.X,Z\right|\Theta \right)\right|X,{\Theta }^{(t)}\right],$$

(10)

where ${\Theta }^{(t)}$ is the parameter estimated by the current iteration t. Using the Bayes theorem, it is easy to draw the following formula

$$Q\left(\Theta ,{\Theta }^{(t)}\right)={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{k=1}^K{\gamma }_k^{(i)}\left[\ln \left({\alpha }_k^{(t)}\varphi \left(x^{(i)}\left|{\mathsf{\theta }}_k^{(t)}\right.\right)\right)\right]}},$$

(11)

where

$${\gamma }_k^{(i)}=E\left[\left.z_k^{(i)}=1\right|x^{(i)},{\Theta }^{(t)}\right]=\frac{{\alpha }_k^{(t)}\varphi \left(\left.x^{(i)}\right|{\mathsf{\theta }}_k^{(t)}\right)}{{\displaystyle \sum _{k=1}^K{\alpha }_k^{(t)}\varphi \left(\left.x^{(i)}\right|{\mathsf{\theta }}_k^{(t)}\right)}}.$$

(12)

The probabilistic assignment ${\gamma }_k^{(i)}$ can achieve the clustering of each pixel.

M-step:

Maximizing the expectation of the log-likelihood function, i.e., updating ${\Theta }^{(t)}$ by maximizing the Q-function

$${\Theta }^{*}={\Theta }^{\left(t+1\right)}=\arg \underset{\Theta }{\max }Q\left(\Theta ,{\Theta }^{(t)}\right).$$

(13)

2.3. Proposed SSCF Method

The difference image (DI) between the bitemporal HSIs, T₁ and T₂, is defined as

$$DI=abs\left(T_1-T_2\right),$$

(14)

where each pixel in the DI corresponds to an M-dimensional spectral change vector (SCV). The DI characterizes the change intensity of each pixel in all bands. Therefore, The DI replaces the bitemporal HSI for subsequent detection.

Since the DI is a 3D data cube equal in size to bitemporal HSIs, SLIC theory for RGB images cannot be used directly for the DI. To make SLIC applicable to 3D data, the spectral distance needs to be measured explicitly, and the rest remains the same. Let each pixel on the DI be expressed as ${[x,c^{\prime },r^{\prime }]}^{\mathrm{T}}$, where $x={\left\{x_m\right\}}_{m=1}^M$ denotes SCV over M bands, and $\left[c^{\prime },r^{\prime }\right]$ indicates the pixel’s spatial position. Then, the spectral distance between pixel i and cluster center (pixel j) is defined as:

$$d_c{}^{\prime }={‖x^{(i)}-x^{(j)}‖}_2.$$

(15)

It is found that the Euclidean distance metric is the most suitable one in SLIC to tackle high-dimensional vectors [61]. The DI is segmented by SLIC into B superpixels, which can be indicated as $SP={\left\{s{p}_i\right\}}_{i=1}^B$.

Figure 2 shows the segmentation result using SLIC on the fifth band DI of the USA dataset. As seen, the pixels in the boxed circular area have a similar color in each of the three images. That means the circular area is a superpixel and the changes in the pixels in this superpixel are similar. Therefore, SLIC can successfully separate different variations in the spatial dimension.

The GMM assumed that the DI is a mixture of K = 3 Gaussian distributions. Let each pixel on the DI be expressed as $x={\left\{x_m\right\}}_{m=1}^M$ over M bands. Then the assembly of all N pixels on the DI is $X={\left\{x^{(i)}\right\}}_{i=1}^N$. In this case, GMM divides the $X$ into K = 3 components: unchanged pixels, uncertain pixels, and changed pixels, as shown in Figure 3. Where the uncertain pixels are denoted as $UN={\left\{u{n}_j\right\}}_{j=1}^J$.

After obtaining the superpixels, $SP={\left\{s{p}_i\right\}}_{i=1}^B$, and uncertain pixels, $UN={\left\{u{n}_j\right\}}_{j=1}^J$, K-means clustering is used to determine the superpixels. K-means reassigns all pixels of sp and un to two classes, changed and unchanged. The determination process can be expressed as

$$\mathrm{Determination}:\left\{\begin{array}{c}sp>un, sp \mathrm{are} \mathrm{changed} \mathrm{pixels}\\ sp\approx un, \mathrm{More} \mathrm{iterations} \mathrm{are} \mathrm{required}\\ sp<un, sp \mathrm{are} \mathrm{unchanged} \mathrm{pixels}\end{array}\right..$$

(16)

When sp is overall greater or less than un, sp is simply determined as changed pixels or unchanged pixels, respectively, and the two clustering centers are from sp and un, respectively. When sp is slightly different from un, the determination of sp is challenged, and the source of the clustering center needs to be constantly updated.

It should be noted that the ground truth for multiple change classes with different colors in Figure 2c is to verify that the superpixel segmentation can split the different changes. However, the number of components in GMM is set as three classes to extract the uncertainty pixels as a rough threshold during clustering. That is, the uncertain pixels extracted by GMM are used to cluster superpixels segmented by SLIC.

A flow chart of the proposed SSCF algorithm is shown in Figure 4. The important parameters involved are the number of superpixels and the compactness w in SLIC. The relevant hyperspectral tasks empirically set the number of superpixels and the compactness to [50, 100, 150, … 500] and [0, 0.1, … 1]. These parameter settings are also adaptable to any bitemporal HSIs. The selection of the number of superpixels parameters depends on the complexity of the ground feature. The choice of the compactness parameter depends on the contrast of the image and the shape of the ground feature. The optimal choice of the parameters differs for different HSI datasets. In any case, neither parameter has a decisive influence on the results.

3. Experiment

3.1. Datasets

Two main aspects should be considered in the selection of the datasets. The first is the availability of ground-truth data. Only high-quality and reliable ground-truth data are available to accurately assess the model’s performance. The second is the diversity among the datasets. Both simple and complex datasets are required to fully validate the generalization capability of the model. In terms of these points, two real-world HSIs datasets are used to evaluate the performance of SSCF [16,60], both of which have been widely used for the evaluation of existing HSI-CD efforts. Both datasets were acquired by the Hyperion sensor mounted onboard the EO-1 satellite and open access via the website (http://rslab.ut.ac.ir, accessed on 1 February 2022) [63]. They have a swath width of 7.7 km with a spatial resolution of 30 m and a spectral range of 0.4–2.5 µm with a spectral resolution of 10 nm [64,65].

As shown in Figure 5, each dataset consists of three elements: bitemporal HSIs and binarized ground-truth change map. The white pixels in the ground-truth change map indicate the changed classes, and the black pixels indicate the unchanged classes. The two datasets are described in detail as follows.

The first dataset (China) is farmland near the city of Yancheng Jiangsu province in China, which was acquired on 3 May 2006 and 23 April 2007 [66]. This scene is mainly a combination of soil, rivers, trees, buildings, roads, and agricultural fields. Almost all the changes in this area are related to changes in the crop and soil in the fields. Each HSI has 155 bands with a spatial size of 450 × 140 pixels. While the changed regions contain 18,277 pixels, the unchanged regions contain 42,723 pixels.

The second dataset (USA) belongs to an irrigated agricultural field of Hermiston city in Umatilla County, Oregon, OR, the USA, which was acquired on 1 May 2004 and 8 May 2007 [60]. The land cover types are soil, irrigated fields, rivers, buildings, types of cultivated land, and grassland. Almost all the changes in this area are related to changes in the crop, soil, or water content (caused by different amounts of irrigated water) in the fields. Each HSI has 154 bands with a spatial size of 307 × 241 pixels. The changed regions contain 16,676 pixels, whereas the unchanged regions contain 57,311 pixels.

Both datasets have their own specific changes. Figure 6 shows the heatmap of DI in the 14th band and multiple SCVs from specified pixels. The heatmap visually reflects the change intensity of each pixel, and the SCV reflects the changes. The SCVs belong to three ground-truth changed pixels (C1, C2, C3) in the China dataset, six ground-truth changed pixels (C1, C2, …, C6) in the USA dataset, and two ground-truth unchanged pixels (UCs).

As seen, the heatmap of the USA dataset contains more colors, which means the changes are more diverse. Both datasets have subtle changes, such as C2 in the China dataset and C4 and C5 in the USA dataset. Since the SCVs of these pixels are very close to UC, it is challenging to detect these points. Therefore, the separation of different changes becomes necessary and meaningful.

3.2. Parameter Setup

The performance of our proposed model SSCF is compared with seven existing HSI-CD methods (CVA [24], SSIM [27], ISFA [37], IRMAD [32], MaxtreeCD [46], Unet [67], SGCL [28]). The code of the MaxtreeCD method is implemented with Matlab, and other methods are implemented with Python.

SLIC segments the DI into specified 300 superpixels spatially with the compactness of 1. GMM divides the DI image into K = 3 components: unchanged pixels, uncertain pixels, and changed pixels. Figure 7 shows the pixels’ number and percentage of each component. As can be seen, the uncertain pixels occupy a large percentage, which are more than the test pixels in each superpixel (detection unit). To reduce the computational effort, only a fraction of the uncertain pixels are randomly selected as a rough threshold. The fraction selection experiments are implemented in Section 4. It is found that 10% of the uncertain pixels are good candidates and will be used in the comparison experiments. These 10% uncertain pixels are selected completely randomly without any dependence.

3.3. Evaluations Measures

Evaluation measures are critical to analyzing the performance of CD methods. To demonstrate our proposed SSCF comprehensively, three evaluation metrics are applied, namely, overall accuracy (OA), F1-score, and Kappa coefficient. We first calculate four indexes: (a) the true-positives (TP), (b) the true-negatives (TN), (c) the false-positives (FP), and (d) the false-negatives (FN). Then, their practical values are counted by comparing the change map and ground truth, as shown in

Table 1. Four indices.

	Change Map
		1	0
Ground-truth	1	TP	FN
	0	FP	TN

. Then precision and recall are calculated with

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\left(\mathrm{TP}+\mathrm{FP}\right)},$$

(17)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\left(\mathrm{TP}+\mathrm{FN}\right)}.$$

(18)

Finally, the F1-score, OA, and Kappa coefficient are obtained, respectively, as

$$\mathrm{OA}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}},$$

(19)

$$\mathrm{F}1-\mathrm{score}=\frac{2\times \left(\mathrm{Precision}\times \mathrm{Recall}\right)}{\left(\mathrm{Precision}+\mathrm{Recall}\right)},$$

(20)

$$\mathrm{Kappa}=\frac{\mathrm{OA}-\mathrm{OP}}{1-\mathrm{OP}},$$

(21)

$$\mathrm{OP}=\frac{(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P})(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N})}{{(\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N})}^2}+\frac{(\mathrm{F}\mathrm{N}+\mathrm{T}\mathrm{N})(\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N})}{{(\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N})}^2}.$$

(22)

They can reveal the overall performance. The larger the value, the better the performance. Especially, F1 score combines both precision and recall metrics. The Kappa coefficient indicates the consistency between the change map and ground truth.

3.4. Results

The experimental results are presented in three forms: quantitative evaluation metrics, binary change maps, and zoomed-in analysis of the subtle change regions.

Table 2. OA, F1, and Kappa of all methods.

Methods	China Dataset	USA Dataset
	OA/F1/Kappa	OA/F1/Kappa
CVA	0.9689/0.9470/0.9250	0.9245/0.8022/0.7574
SSIM	0.9673/0.9424/0.9196	0.9040/0.7372/0.6824
ISFA	0.9596/0.9323/0.9035	0.8344/0.6920/0.5827
IRMAD	0.9652/0.9417/0.9169	0.9180/0.7830/0.7347
MaxtreeCD	0.9683/0.9453/0.9214	0.9255/0.8437/0.7940
Unet	0.9655/0.9420/0.9175	0.8920/0.7553/0.6861
SGCL	0.9746/0.9562/0.9383	0.9130/0.7655/0.7151
SSCF	0.9770/0.9597/0.9435	0.9383/0.8577/0.8184

shows the evaluation metrics of the eight methods. Obviously, our proposed SSCF on the two datasets outperforms the other seven methods on each metric, which fully validates the feasibility of SSCF. It is found that the performance of the seven compared methods is similar in the China dataset and diverse in the USA dataset. All evaluation metrics on the USA dataset are lower than that on the China dataset. Mainly because the detection difficulty is proportional to the complexity of the dataset. The USA dataset contains more subtle changes, which makes the detection more challenging. Therefore, detection methods that are insensitive to subtle changes will not achieve the desired results in the USA dataset. In contrast, SSCF has the advantage of detecting subtle changes. It should be noted that the parameter setting is the same for both datasets (the number of superpixels is 300, the percentage of random uncertain pixels is 10%, and the compactness is 1). However, this setting is not the best, for example, the China dataset has a better result when the compactness is 0.6. A detailed comparison and discussion are presented in Section 4.

To further demonstrate the superiority of SSCF in the accurate extraction of subtle changes, the edges in region R0 shown in Figure 8a should be analyzed carefully.

Figure 9 shows the change maps of all methods on the China dataset. As seen, while all methods can capture large areas, the detection details of each method are different. The detection difficulty of this dataset exists on the edges, corresponding to the subtle changes caused by the mixture of crops, bare land, and field ridges. SSCF maintains a high degree of agreement with the ground-truth map over these regions.

Figure 10 shows the details of R₀ with four colors, indicating four detection results. While the regions with the green and white colors are correctly detected as changed and unchanged, respectively, the areas with the pink and blue colors represent missed detections and false alarms, respectively. Obviously, SSCF has fewer missed detections and false alarms. Especially in the lower-left region, there are fewer pink and blue points relative to other methods.

Figure 11 shows the change maps from all methods on the USA dataset. Intuitively, the change map of SSCF is closest to the ground truth. Both the large change regions and most discrete change points are detected. The false alarm rate of the ISFA method is high. The missed detection rate of the CVA, SSIM, IRMAD, and SGCL methods is high. The change map of MaxtreeCD is far away from the ground truth. Although the Unet can detect the aggregated change regions, fragmented change points are incomplete.

Figure 12 shows the details of the three selected areas (R₁, R₂, and R₃ in Figure 8b). SSCF has the least missed detections and false alarms. Although MaxtreeCD also has fewer missed detections in these areas, the false alarms on the whole change map are higher.

The computation time of SSCF is compared with two deep learning methods. The computing device is equipped with Intel(R) Xeon(R) Gold 6136 CPU (2.99 GHz) and NVIDIA GeForce RTX 3090 GPU. The Unet method and the SGCL method are written in Python via the code library of PyTorch. The SSCF method does not use parallel computing. Here, we list the computing time for each dataset in

Table 3. Time cost (seconds) of each dataset.

	China Dataset	USA Dataset
Unet	556.35	692.74
SGCL	125.21	141.70
SSCF	61.50	73.23

. This figure shows that SSCF outperforms these two deep learning methods in terms of efficiency.

4. Discussion

The sensitivity of parameters needs to be analyzed. For SSCF, the selection of two parameters, the number of superpixels and the proportion of uncertain pixels, on the detection results are revealed in graphs.

Additional experiments are performed to clarify the effects of the number of superpixels parameters or the proportions of uncertain pixels on the detection results. Figure 13 shows that the influence of the number of superpixels is weak, especially for the China dataset. Furthermore, the results of SSCF at any superpixel number are more accurate than those of the comparison method. Therefore, the number of superpixels is experimentally set to 300 for better performance. It should be noted that the number of superpixels, as a parameter to be set in advance, is independent of the ground truth. For unknown data (no ground-truth available), it is recommended to choose the values in a reasonable range of 50–500, and 300 is a relatively good choice.

To evaluate the effects of randomly selecting uncertain pixels with different proportions, 10 trials with random sampling are conducted, and the average of the indexes is taken as the final result. As shown in Figure 14, the effects with different ratios are similar and steady. The lower ratio of 5% leads to a slight decrease in accuracy. To save computational resources, the ratio of 10% is preferred.

Figure 15 shows the effect of the compactness w in SLIC on the detection performance of SSCF. Figure 16 shows the schematic diagram of the superpixel segmentation results for the two datasets based on different compactness w. The higher compactness w values lead to more complex edges. The optimal compactness of different datasets varies depending on the data characteristics. Compared with the China dataset, the USA dataset is more affected by the parameter compactness w because the spatial information is more important. As a result, a parameter of 0.6 to 1 is a good choice for both datasets. In our work, the compactness is set to 1.

In summary, SSCF outperforms seven comparison methods with a simple and easy-to-implement framework. In particular, SSCF has a clear advantage in the simultaneous and correct detection of the subtle changes and strong changes. In addition, the robustness of the method to noise needs to be improved. The ability of SSCF to highlight subtle changes is prone to spurious changes when there is much noise in the bitemporal HSIs. Therefore, SSCF requires high preprocessing of the data.

5. Conclusions

The decision boundary of the existing HSI-CD methods is adversely affected by strong changes, which leads to the submergence of subtle changes. In this paper, SSCF is proposed to detect weak changes. SSCF treats CD as a clustering task and clusters superpixels and a randomly selected portion of uncertain pixels (rough threshold) to obtain the detection results of superpixels. In this case, the threshold is a statistical distribution independent of strong changes, so the strategy of SSCF can significantly reduce the decision boundary being adversely affected. Meanwhile, SSCF outperforms deep learning methods in terms of efficiency and outperforms other methods in terms of accuracy. It requires fewer computation resources to implement the HSI-CD task. The overall accuracy and Kappa coefficient of SSCF exceed 93% and 0.81, respectively, on the two real bitemporal HSIs datasets. The total detection time is less than 100 s. These improved detection accuracy and speed for our SSCF are important for the detection of weakly change scenes and the implementation of emergency tasks. In the future, we will combine deep learning methods with other methods to carry out HSI-CD multi-classification tasks.