Unsupervised SAR Image Change Type Recognition Using Regionally Restricted PCA-Kmean and Lightweight MobileNet

Abstract

Change detection using synthetic aperture radar (SAR) multi-temporal images only detects the change area and generates no information such as change type, which limits its development. This study proposed a new unsupervised application of SAR images that can recognize the change type of the area. First, a regionally restricted principal component analysis k-mean (RRPCA-Kmean) clustering algorithm, combining principal component analysis, k-mean clustering, and mathematical morphology composition, was designed to obtain pre-classification results in combination with change type vectors. Second, a lightweight MobileNet was designed based on the results of the first stage to perform the reclassification of the pre-classification results and obtain the change recognition results of the changed regions. The experimental results using SAR datasets with different resolutions show that the method can guarantee change recognition results with good change detection correctness.

1. Introduction

Globally, natural phenomena such as earthquakes and heavy rainfall events, which sometimes occur simultaneously, can lead to building collapse and flooding [1,2,3], causing significant damage and serious economic and social impacts on the natural environment and human infrastructure. Synthetic aperture radar (SAR) sensors are used in remote sensing geodynamic monitoring owing to their all-day, all-weather operation. Image change detection using SAR has become increasingly important for disaster assessment in urban areas [4], deforestation [5], and flood and glacier monitoring to analyze events that change a geographical area following a disaster [6]. However, the existing change detection only detects the change area and cannot recognize the change type, such as the change from what-to-what ground object type. If the change type can be directly recognized, the application of biphasic SAR images can be greatly expanded.

Several efforts have been made in the application of SAR images [7,8,9,10]. Similar to change detection, the presence of inherent noise in SAR images makes change identification difficult. A common approach is to first derive the difference image (DI) of a multi-temporal SAR image and then analyze the DI to obtain a change map [11], and our proposed change recognition draws on this idea.

For DI analysis, the hierarchical fuzzy C-means clustering (HFCM) algorithm was used in previous studies [12,13,14] to obtain pre-classification results. In these studies, the pre-classification stage of the HFCM algorithm had more misclassified pixels in the pre-classification results, which led to incorrect training samples being provided to the deep learning network classification model and ultimately resulted in misclassification. A principal component analysis (PCA) and k-mean clustering algorithm (PCA-Kmean) [15] was proposed to obtain detection results that better retain the change region, albeit with more false alarms. Recently, deep learning (DL) has become an effective nonlinear modeling tool for the reclassification stage, and various neural network classification models are widely used for change detection, as they can extract high-dimensional abstract features from images to achieve better automatic classification. The standard convolutional neural network (CNN) [16] is a common classification model; however, its simple structure usually leads to poor classification results. Gong et al. [17] proposed a deep neural network (DNN) for SAR image change detection, but it has the disadvantages of large parameters, slow training, and limited performance. The PCANet [18] was found to be more time consuming due to the long feature extraction time. However, for the same accuracy conditions, smaller models have fewer parameters, are faster to train, and are easy to deploy on mobile devices. The SAR image change detection method of pre-classification, which involves clustering and then post-classification by a neural network model, is expected to be a lightweight and well-classified neural network classification model. Recently, several lightweight DL networks such as SqueezeNet [19], ShuffleNet [20], and MobileNet V2 [21] were proposed, with all of them achieving good classification efficiency; however, their network depth is redundant for training small-sized image blocks, and the training time is long.

Inspired by the two-stage idea of clustering pre-classification of SAR change detection and deep learning network classification, we designed a new application of unsupervised change recognition using Mobile PCA_kmean (RRPCA-Kmean) and lightweight networks. It can greatly expand the application of bi-temporary SAR images and is no longer limited to the detection of changing regions by the existing change detection.

The novelty of this study is based on the following three points:

(1) An RRPCA-Kmean clustering algorithm was designed to provide highly reliable pre-classification results. These results can be used as a pseudo-label for training samples that emphasize central pixels and ignore edge noise.

(2) A Lightweight MobileNet (LMNet) classification model was designed to provide a fast and efficient classification network for change recognition.

(3) A two-stage unsupervised change recognition framework was designed. The method simultaneously implements change region detection and change recognition.

2. Methodology

The methods proposed in this study can be categorized into two stages (pre-classification and reclassification), which are described in Figure 1. First, RRPCA-Kmean was used to obtain pre-classification results from the DI; subsequently, the pre-classification results were used as pseudo-labels to generate training samples from the dual-temporal SAR images. Finally, an LMNet was designed to train the samples and reclassify the pre-classification results.

2.1. RRPCA-Kmean Clustering Algorithm

Two SAR images were taken from the same location at different times: $I_1=\{I_1(m,n),1\le m\le M,1\le n\le N\}$ and $I_2=\{I_2(m,n),1\le m\le M,1\le n\le N\}$, both of size $M\times N$. Change recognition is required to recognize the change type from $I_1$ to $I_2$.

The first step was to generate the initial DI from two original SAR images. Considering its ability to suppress speckles, log-ratio is a common operator in many change detection studies [22]. The DI is defined as follows:

$$I_{DI}(m,n)=\left|\log \left(\frac{I_1(m,n)+1^{}}{I_2(m,n)+1}\right)\right|$$

(1)

Subsequently, we combined mathematical morphology, PCA, and k-mean clustering to design a RRPCA-Kmean clustering algorithm. First, pre-classification results were obtained using PCA and k-means clustering, as they consider a large number of mis-classifiable pixels as intermediate classes, which greatly reduces the generation of incorrect labels. To reduce the classification error problem of a large number of intermediate classes in reclassification, we also introduced mathematical morphology erosion to restrict the pre-classification results. A 50 × 50 all-1 matrix was used as the structural element for local erosion, and the minimum value was found to replace the original gray value within this 50 × 50 neighborhood. In the pre-classification result, there were only three gray values: 0.5, 1, and 0. The unchanged region with gray value 0 occupied most of the matrix; consequently, the gray values of the changed region (with gray value 1) and the intermediate class region (with gray value 0.5) were reduced to 0 to achieve morphological region restriction. The process of the algorithm is summarized in Algorithm 1.

Algorithm 1 RRPCA-Kmean Algorithm

Input: DI Step 1: Extract the PCA feature vector. Step 2: Run the k-mean clustering algorithm to generate three classes ${\Omega }_{\mathrm{c}}^1$,${\Omega }_{\mathrm{i}}^1$ and ${\Omega }_{\mathrm{u}}^1$. where ${\Omega }_{\mathrm{c}}^1$ is a pseudo-changed class, ${\Omega }_{\mathrm{i}}^1$ is a pseudo-intermediate class, and ${\Omega }_{\mathrm{u}}^1$ is a pseudo-unchanged class. Step 3: Calculate the ratio of the mean value to the number of pixels for each class and arrange them from smallest to largest to obtain three classes $\{{\Omega }_{\mathrm{c}}^2,{\Omega }_{\mathrm{i}}^2,{\Omega }_{\mathrm{u}}^2\}$, the initial pre-classification results. Step 4: Perform mathematical morphological erosion of the pre-classified result map using a 50 × 50 all-1 matrix. Step 5: Take out the pre-classified result map within the corrupted range, the RRPCA-Kmean result map. Output: RRPCA-Kmean result map containing $\{{\Omega }_{\mathrm{c}}^{},{\Omega }_{\mathrm{i}}^{},{\Omega }_{\mathrm{u}}^{}\}$.

2.2. Generation of Training Samples

The next step was to extract training samples for change recognition. We further filtered the changed class pixels in the regionally restricted pre-classification result. The changed class pixels with grayscale values greater than 0 in the time 2 image and time 1 image subtraction results denote the land to water (LW) changed labels, and the change class pixels with grayscale values less than 0 denote the water to land (WL) changed labels. It is worth noting that mathematical morphology is introduced in this paper to improve the accuracy of labels with different variation types and reduce the recognition error. Unchanged type labels are represented as unchanged class in the pre-classification result. The training samples are generated as shown in Figure 2.

First, image patches of positions of interest (pixels belonging to ${\Omega }_{\mathrm{c}}^{}$ and ${\Omega }_{\mathrm{u}}^{}$) were generated. These image patches contained enough change information around the positions. $P_{mn}^{I_1}$ represents a patch with the center at position $(m,n)$ in image $I_1$, and the size of $P_{mn}^{I_1}$ is $(k/2)\times (k/2)$.$P_{mn}^{I_2}$ represents the corresponding patch in image $I_2$. To dilute the noise, a bilinear interpolation was applied to obtain blocks of size $(k/2)\times k$. The two blocks were then combined and multiplied with a mask (blue represents gray value 1 and white represents 0 in Figure 2) to obtain the training samples. This mask processing step suppressed edge noise and emphasized the central pixel.

2.3. Lightweight MobileNet Classification Model

The MobileNet v2 [21] has many layers; it contains depth-separable convolution and pointwise convolution modules as well as an inverse residual module, which were introduced in our proposed network. Among them, the depth-separable convolution and pointwise convolution modules mainly reduce the parameters and computation cycles. The inverse residual module mainly avoids the gradient vanishing problem and reduces information loss. Both are effective for our small image blocks, since they minimize the number of parameters and operations and shorten the training time. We designed a lightweight MobileNet reclassification model for reducing information loss and realizing efficient classification. The proposed model mainly consisted of five modules: Modules A, B, C, D, and E. Module A and Module B are depth-separable convolutions with the addition of a 1 × 1 convolution; Module C is an inverse residual module; Module D is a squeeze excitation (SE) module; and Module E is an efficient final stage module. Depth-separable convolution is good for reducing the computational effort, but it exhibits reduced accuracy. The addition of a 1 × 1 pointwise convolution in Module A and Module B helps to alleviate this problem. The inverse residual module in Module C helps to alleviate the problem of information loss caused by dimensional transformation. Module D is a lightweight attention module for reducing computational effort. To alleviate the problem of high resource consumption in the output of the network, Module E uses a 1 × 1 convolution for expansion, and Relu6 activation function is used immediately after the pooling layer to improve the network speed. The Relu6 activation function maintains the robustness of the network well, as shown in Equation (2).

$$Relu6=\min (\max (0,x),6)$$

(2)

The network finally uses a 1 × 1 convolution for linear output to prevent information loss due to dimensional transformations. The use of the drop layer helps to further reduce computation, accelerate convergence, and alleviate overfitting. The processing of this efficient final stage module increases the speed of computation while preserving accuracy. The network structure is shown in Figure 3, and the specific network body architecture is shown in

Table 1. Lightweight MobileNet body architecture.

Type	Filter Shape	Input Size
Conv	3 × 3 × 32	10 × 10 × 1
Module A	3 × 3 × 32	5 × 5 × 32
Module B	3 × 3 × 96	5 × 5 × 96
Module C	1 × 1 × 24 × 144	3 × 3 × 24
Module D	3 × 3	3 × 3 × 144
Module E	1 × 1 × 24 × 32	3 × 3 × 24

.

3. Results

3.1. Datasets

Three actual SAR image data sets in Figure 4 were employed to prove the superiority of the proposed approach.

Table 2. Details of a real SAR data set.

Dataset	A	B	C
Sensor	Radarsat-2	Radarsat-2	GaoFen-3
Location	Yellow River, China	Yellow River, China	Yellow River, China
Band	C	C	C
Polarization	VV	VV	VV
Date	2008.06	2008.06	2021.07.20
	2009.06	2009.06	2021.07.24
Size	257 × 289	233 × 356	300 × 300
Resolution	8 m	8 m	5 m
Changes	Farming	Flood	Farming

lists a detailed description of each dataset, including sensor type, location, imaging date, image size, resolution, and reason for variation.

To evaluate the effectiveness of our proposed LMNet, the related models were considered for comparison, including the standard CNN [16], SqueezeNet [19], ShuffleNet [20], and MobileNet v2 [21] networks. The training samples for each network were selected as in the RRPCA-Kmean. The SqueezeNet, ShuffleNet, and MobileNet networks were trained for classification using migration learning with modified input size and final output of the softmax and fully connected layers. All experiments were performed using MATLAB 2020b. The initial learning rate was set to 0.001, and the “adam” optimizer was used.

3.2. Evaluation Metric

We used four credible evaluation indexes to measure the performance of the proposed LMNet in change recognition, including the kappa coefficient (k) of change detection, LW recognition accuracy $P_1$, water to land (WL) recognition accuracy $P_2$, overall accuracy (OA), and average accuracy (AA).

$$P_1=\frac{T{P}_1}{T{P}_1+F{P}_1+F{N}_1}$$

(3)

$$P_2=\frac{T{P}_2}{T{P}_2+F{P}_2+F{N}_2}$$

(4)

$$OA=\frac{T{P}_1+T{P}_2}{T{P}_1+T{P}_2+F{P}_1+F{N}_1+F{P}_2+F{N}_2}$$

(5)

$$AA=(P_1+P_2)/2$$

(6)

For each prediction, separate statistics were used for TP (predicted answer correct), FP (wrongly predicted other class as this class), and FN (predicted this class label as other class label). The subscripts in the above equation indicate that they were calculated separately for different types of changes.

3.3. Analysis of Results

The final recognition result was a subjective evaluation of the recognition of the change type by the color change type map. To evaluate the results objectively, k was evaluated for change detection of the binary image of the reference image. By comparing the change recognition map with the change type reference map, LW, WL, OA, and AA were calculated. Figure 5a–e shows the recognition results of different methods, and Figure 5f indicates the corresponding reference chart.

Table 3. Change recognition results of different methods on three datasets.

Method	Results on the A dataset
	k (%)	LW Area (%)	WL Area (%)	OA (%)	AA (%)
CNN	82.80	78.90	37.58	74.97	58.24
SqueezeNet	72.95	65.10	41.75	62.68	53.42
ShuffleNet	78.83	73.98	40.00	70.08	56.99
MobileNet v2	84.02	79.79	48.73	76.72	64.26
LMNet	87.64	84.00	49.61	81.59	66.80
Method	Results on the B dataset
	k (%)	LW Area (%)	WL Area (%)	OA (%)	AA (%)
CNN	61.31	61.59	28.22	45.93	44.90
SqueezeNet	66.69	58.18	38.89	51.67	48.54
ShuffleNet	72.89	64.13	47.29	58.77	55.71
MobileNet v2	76.52	66.41	55.45	63.22	60.93
LMNet	81.10	71.32	63.93	69.23	67.62
Method	Results on the C dataset
	k (%)	LW Area (%)	WL Area (%)	OA (%)	AA (%)
CNN	69.39	61.36	45.59	56.01	53.47
SqueezeNet	66.18	70.66	31.60	52.95	51.13
ShuffleNet	74.00	73.44	44.32	62.15	58.88
MobileNet v2	74.94	68.24	49.93	62.53	59.09
LMNet	78.96	71.32	59.86	67.79	65.59

and Figure 6 present the results of each method quantitatively.

Speckle noise is a critical and unavoidable factor affecting the change detection results in SAR image processing. In addition, complex backgrounds, such as the edges of land and water, can also introduce some interference.

As for the standard CNN, it can be seen from Figure 5a that some pixels in dataset A were missed, while extra pixels in dataset B should not have been detected. Table 3 also shows that the standard CNN had lower evaluation metrics in all the B and C datasets. These indicate that the simple shallow neural networks were not sufficiently generalized and are prone to increased false positives or misses.

For SqueezeNet, it can be seen from Figure 5b that there were many false alarms in all three datasets, which were not effective. As can be seen from Table 3, the metric OA (51.67%) was better than CNN (45.93) in dataset B but was lower than CNN on the other two datasets A and C. This indicates that such a lightweight network is not suitable for high-resolution detection of the types of changes that occur at river edges.

For ShuffleNet, it can be seen from Figure 5c that more false detections occurred in datasets A and C due to noise interference. However, the recognition of change types at river edges in dataset B, such as CNN and SqueezeNet, had fewer false pixels than CNN and SqueezeNet, indicating that ShuffleNet is more suitable for change recognition at river edges. As shown in Table 3, OA (70.08%) was lower than CNN (74.97%) on dataset A. Therefore, this lightweight network needs further improvement to enhance its adaptability.

A comparison of Figure 5a–d shows that MobileNet V2 had the lowest number of recognition errors, and the details were better retained. MobileNet V2 recognizes each change type better than the first three methods. The evaluation metrics of MobileNet V2 in Table 3 were also optimal, indicating that this deep lightweight network is better adapted to recognize various datasets. Although the MobileNet V2 network is time-consuming, it can be further improved to suit different needs.

As for our proposed LMNet, a comparison of Figure 5a–e shows that our result was the best with the least number of incorrectly recognized pixels. LMNet achieved the highest basic k values, OA, and AA in all datasets in Table 3. In addition, it can be seen from

Table 4. The training times and parameters of compared methods.

Methods	CNN	SqueezeNet	ShuffleNet	MobileNet v2	LMNet
Times	1.3 min	7.3 min	21.4 min	32.2 min	5.4 min
Parameters	39.7 k	9.8 M	863 k	3 M	158 k

that LMNet required less time and had the least number of parameters, indicating that our LMNet works best.

In addition, to clearly understand the effect of each method in Table 3, we present Figure 6, which clearly shows that the indicators of our method are optimal compared to those of other methods.

3.4. Analysis of the Patch Size

The training samples were captured by image patches of size k. We evaluated the performance of the proposed LMNet with k = 8, 10, 12, 14, 16, and 18. Figure 7 shows the relationship between k and OA. The value of OA first increased and then decreased as k gradually increased. The OA curve shows that the training sample size was very important for the change recognition task. However, large patch sizes increase the computational burden and may introduce some information noise that affects the performance of change recognition. Therefore, we took k = 10 for the first two datasets and k = 14 for the last dataset in the experiments. The sizes differed, as the first two datasets had few and concentrated change regions, while the last dataset had small and many change regions with high resolutions. The larger dataset contained more information and was more suitable for the recognition of small change regions.

4. Conclusions

In this study, a RRPCA-Kmean and LMNet two-stage unsupervised change recognition method was proposed for further application to change detection in SAR images. The RRPCA-Kmean designed in this study can be applied to various change detection and recognition methods that generate pre-classification results by clustering. The proposed training sample design method emphasizes the central change pixels and suppresses edge noise. The proposed LMNet has a good balance between recognition time and recognition effect, while exhibiting a good application value. Our method achieved good results on SAR images with different resolutions, and experiments demonstrated the future application of the algorithm in change detection. In our future work, we will aim to recognize more types of changes.