A Multi-Scale Graph Based on Spatio-Temporal-Radiometric Interaction for SAR Image Change Detection

Abstract

Change detection (CD) in remote sensing imagery has found broad applications in ecosystem service assessment, disaster evaluation, urban planning, land utilization, etc. In this paper, we propose a novel graph model-based method for synthetic aperture radar (SAR) image CD. To mitigate the influence of speckle noise on SAR image CD, we opt for comparing the structures of multi-temporal images instead of the conventional approach of directly comparing pixel values, which is more robust to the speckle noise. Specifically, we first segment the multi-temporal images into square patches at multiple scales and construct multi-scale K-nearest neighbor (KNN) graphs for each image, and then develop an effective graph fusion strategy, facilitating the exploitation of multi-scale information within SAR images, which offers an enhanced representation of the complex relationships among features in the images. Second, we accomplish the interaction of spatio-temporal-radiometric information between graph models through graph mapping, which can efficiently uncover the connections between multi-temporal images, leading to a more precise extraction of changes between the images. Finally, we use the Markov random field (MRF) based segmentation method to obtain the binary change map. Through extensive experimentation on real datasets, we demonstrate the remarkable superiority of our methodologies by comparing with some current state-of-the-art methods.

1. Introduction

1.1. Background

Change detection (CD) refers to the analysis of remote sensing images acquired at different times of the same scene, to identify changes occurring on the Earth’s surface [1]. It finds extensive applications in both civilian and military domains, such as disaster relief, agricultural surveys, urban planning, and military monitoring [2,3,4]. Among these, synthetic aperture radar (SAR) is an active imaging system known for its all-weather and all-day imaging capabilities, as well as its insensitivity to atmospheric and lighting conditions. Therefore, SAR image CD technology has been receiving increased attention [5,6,7].

In general, CD methods can be categorized, based on the requirement for labeled data, into supervised, semi-supervised, and unsupervised methods [8,9,10]. While supervised and semi-supervised methods often yield more accurate detection outcomes, these approaches necessitate the utilization of labeled samples for training, which incurs substantial labor costs and demands significant domain expertise. Consequently, such methods are relatively constrained in practical applications. Traditional unsupervised SAR CD methods can be divided into three steps: pre-processing, computation of difference images (DI), and extraction of change results [11]. During the pre-processing phase, operations such as radiometric correction and image registration are typically employed to enhance radiometric and spatial comparability among multi-temporal SAR images. The second step involves generating DI to initially distinguish changed and unchanged regions. Ultimately, in the third step, the DI is subject to analysis and segmentation into categories of changed and unchanged.

1.2. Related Work

The mechanism of SAR coherent imaging leads to the intrinsic property of speckle noise in SAR images. This speckle noise manifests as random pixel intensity variations within homogeneous regions on SAR images, visually appearing as grainy speckles. Due to the impact of speckle noise, generating a high-quality DI in the SAR CD becomes a challenging task [12,13,14].

Ratio operators [15], logarithmic-ratio operators [16], mean-ratio operators [17], and neighborhood-ratio operators [18] have the ability to transform multiplicative speckle noise into additive noise, while also enhancing low-intensity pixels to some extent. They exhibit a certain robustness to speckle noise, making them commonly used in constructing DI. Furthermore, some researchers have proposed methods for fusing DI, such as using wavelet fusion techniques to fuse logarithmic and mean ratio images [19], Gaussian logarithmic ratio images, and logarithmic ratio images [20], using Shearlet fusion techniques to fuse saliency images and Gaussian logarithmic ratio images [21] and saliency-guided logarithmic ratio images [22], and so on. Although methods based on image ratios can alleviate the impact of noise to some extent, they cannot fully exploit the information from multi-temporal images, leading to the presence of residual noise that can still interfere with the final detection results. In [23], the authors have tested various despeckling methods for their impact on change detection performance. Although despeckling SAR images prior to comparison can alleviate, to some extent, the impact of speckle noise on DI, this approach also carries a certain risk; namely, the details lost during the despeckling process cannot be recovered in subsequent processes. To address this challenge, a model that combines the statistical properties of logarithmically transformed SAR images and non-local low-rank modeling has been proposed [5]. This model does not require the application of despeckling methods separately to multi-temporal SAR images, nor does it directly apply ratio/logarithmic-ratio/mean-ratio operators to SAR images. Therefore, this approach avoids the loss of information during the despeckling and image contrast enhancement processes, thus obtaining improved DI.

It is important to note that completely eliminating speckle noise from SAR images is unattainable without compromising the structural details of the images. Moreover, speckle noise in high-resolution SAR images exacerbates intensity variations, which can impact the accuracy of change detection. Consequently, generating difference images typically encounters the challenge of striking a balance between the robustness against speckle noise and the preservation of image details without compromising the effectiveness of coherent speckle noise reduction.

Recently, to address this contradiction, some graph-based CD methods have been proposed. In [24], a pointwise graph model is first constructed for one image to capture the intensity and geometry information, and then the changes are evaluated by directly comparing the two images within this pointwise graph. Wu et al. have constructed an object-based graph for each image [25], which first segments the images into superpixels as the vertices of the graph and uses the log-ratio distance to compute the edge weights of the graph, subsequently transforming the issue of change measurement into the comparison of edge weights between two graphs. Wang et al. have not only established a spatio-temporal graph model [26] for extracting the spatio-temporal neighborhood information of images but also extended the concept of hypergraphs into CD [27]. They have employed hyperedges by forming sets of pixels with similar characteristics within local neighborhoods, effectively transforming the problem of SAR CD into one of hypergraph matching. Sun et al. have introduced a series of CD methods based on graph signal processing [28]. The central concept of these methods involves utilizing K-nearest neighbor (KNN) graphs that represent the intrinsic similarity relationships within images to characterize their topological structures. Subsequently, they employ graph mapping [12,29,30] or image regression [31,32,33] techniques to compare structural differences between images and detected changes. Expanding on this, Chen et al. have proposed CD methods that combine local and non-local graph structures [34,35], Zheng et al. have proposed global and local graph-based DI enhancement methods [13] and a change smoothness method [36] for CD. Furthermore, numerous methods based on graph neural networks (GNN) have also been introduced for SAR CD tasks, such as the dynamic graph-level neural network [37], the multi-scale graph convolutional network [38], the variational graph auto-encoder network [39] and the graph-based knowledge supplement network [40].

From these methods, it can be observed that graph-based CD methods offer several advantages. Firstly, graphs can depict the intricate relationships between objects within images, representing the image’s structure, which are relatively stable and less affected by speckle noise. Secondly, the construction of graph models is versatile, that is, selecting different vertices (such as pointwise, patch, superpixels) and various types of connections (like fully connected, K-nearest neighbors, spatial neighbors, etc.) can yield different graph models (such as local, non-local, global-graph), capturing distinct information (such as spatio-temporal, geometric, topological, and intensity information). Thirdly, there is a diverse range of change measurement methods, enabling CD through different graph comparison techniques, such as direct comparison, graph mapping, graph regression, and more.

1.3. Motivation and Contribution

The above graph-based methods still have some limitations in utilizing graph information, primarily in two respects: firstly, they overlook the inherent multi-scale information in remote sensing images. It is well known that remote sensing images depict complex scenes with significant variations in object sizes. For instance, within the same scene, there can be smaller objects like vehicles and buildings alongside larger ones such as farmlands, forests, rivers, and lakes. Consequently, single-scale graph models struggle to capture the intricate structures present in remote sensing images. Secondly, these methods neglect the temporal-spatial-radiometric information interaction in constructing graph models across different time frames of images. While previous approaches have aimed to capture the spatiotemporal relationships using graph models, they often employ a single graph model to represent the temporal dimension’s correlations and do not consider integrating multiple graph models to capture the multifold relationships across other dimensions.

To overcome the aforementioned limitations and leverage the advantages of graph models in CD tasks, this paper proposes a multi-scale graph based on spatio-temporal-radiometric information interaction (STRMG) for SAR images. Initially, it partitions multitemporal images into image patches and constructs multi-scale K-nearest neighbor graphs for each image using the image patches as vertices. These graphs then capture spatial information, radiometric information, and multi-scale details within the images. Then, a graph fusion strategy is devised to fuse graphs constructed at different scales, enhancing the accuracy and richness of the image’s structural information. Subsequently, STRMG maps a graph constructed on one temporal image to another, enabling spatio-temporal-radiometric interaction between graph models. Finally, by comparing the original graph and the mapped graph models, a change metric is constructed, and a Markov random field model (MRF) model is employed to segment the DI and extract the final change map (CM).

The main contributions of this paper are summarized as follows:

• This paper introduces a multi-scale graph model and devises a well-designed graph fusion strategy, enabling the comprehensive utilization of multi-scale information present in remote sensing images. This approach better represents the intricate relationships among features within the images.
• This study achieves spatio-temporal-radiometric information interaction between graph models by employing graph mapping, which can effectively explore the associations between multi-temporal images and result in more accurate extraction of changes between images.
• Experimental comparisons with several state-of-the-art methods on three datasets demonstrate the competitive performance of the proposed STRMG method, underscoring its strong capabilities in change detection.

2. Methodology

The SAR images from pre-event ($t_1$) and post-event ($t_2$), denoted as $\tilde{\mathbf{X}},\tilde{\mathbf{Y}}\in {\mathbb{R}}^{M\times N}$, respectively, are registered. Their corresponding pixel values are represented as ${\tilde{x}}_{m,n},{\tilde{y}}_{m,n}$. The objective of change detection is to determine a binary change map $\mathbf{B}\in {\mathbb{R}}^{M\times N}$, which indicates whether each pixel has undergone a change. The proposed method in this paper consists of three main steps: (1) pre-processing: dividing the images into image patches; (2) generating the DI: constructing KNN graphs for SAR images, implementing spatio-temporal-radiometric interaction through graph mapping, and quantifying the change level; (3) solving for the CM: formulating an MRF model to segment the DI and extract the changed region. The algorithmic framework is depicted in Figure 1.

2.1. Pre-Processing

First, the multi-temporal SAR images are partitioned into non-overlapping square image patches at S different scales $s=1,2,\cdots ,S$. Taking the SAR image $\tilde{\mathbf{X}}$ at time $t_1$ as an example, it is divided into image blocks of size $sp\times sp$, where $p\ge 2$ is a positive integer. These image patches are vectorized and stacked to form the image patch group matrix (PGM) at different scales, denoted as ${\mathbf{X}}^{\left(s\right)}\in {\mathbb{R}}^{s^2{p}^2\times N_s}$, where $N_s=⌈M/sp⌉\times ⌈N/sp⌉$, and $⌈·⌉$ denotes the ceiling operation. Similarly, for the SAR image $\tilde{\mathbf{Y}}$ at time $t_2$, the same partitioning procedure is applied to obtain the PGM ${\mathbf{Y}}^{\left(s\right)}\in {\mathbb{R}}^{s^2{p}^2\times N_s}$. As a result, for the same image patches ${\mathbf{X}}_i^{\left(s\right)}$ and ${\mathbf{Y}}_i^{\left(s\right)}$, $i=1,2,\cdots ,N_s$ in $\mathbf{X}$ and $\mathbf{Y}$, respectively, they still represent the same geographic region.

2.2. Multi-Scale Graph Construction

Based on the self-similarity property of the images, for every small image patch, there are always other image patches within the image that are very similar to it. We refer to this similarity relationship among the image patches within the image as the image structure. After obtaining the PGMs at different scales, we construct S multi-scale KNN graphs to represent the image structure. Taking the time $t_1$ image $\tilde{\mathbf{X}}$ as an example, at each scale s, we treat the image patches ${\mathbf{X}}_i^{\left(s\right)}$ as vertices in the graph, and connect each vertex to its $k=⌈{\sqrt{N}}_s⌉$ most similar neighbors. This results in the construction of S different-scale graph models, denoted as $G_{t1}^{\left(s\right)}=\left\{V_{t1}^{\left(s\right)},E_{t1}^{\left(s\right)},{\mathbf{W}}_{t1}^{\left(s\right)}\right\}$ with

$$\begin{array}{cc}\hfill & V_{t1}^{\left(s\right)}=\left\{{\mathbf{X}}_i^{\left(s\right)}|i=1,\cdots ,N_s\right\}\hfill \\ \hfill & E_{t1}^{\left(s\right)}=\left\{\left({\mathbf{X}}_i^{\left(s\right)},{\mathbf{X}}_j^{\left(s\right)}\right)|j\in {\mathcal{N}}_i^{x\left(s\right)};i=1,\cdots ,N_s\right\}\hfill \\ \hfill & W_{t1}^{\left(s\right)}\left(i,j\right)=\left\{\begin{array}{cc}exp\left(-\lambda d_{i,j}^{x\left(s\right)}\right);& \mathrm{if}\left({\mathbf{X}}_i^{\left(s\right)},{\mathbf{X}}_j^{\left(s\right)}\right)\in E_{t1}^{\left(s\right)}\\ 0;& \mathrm{otherwise},\end{array}\right.\hfill \end{array}$$

(1)

where $\lambda >0$ denotes the parameter that controls the exponential kernel bandwidth, which is set to be $\lambda =0.5$ in the proposed method, $d_{i,j}^{x\left(s\right)}=dist\left({\mathbf{X}}_i^{\left(s\right)},{\mathbf{X}}_j^{\left(s\right)}\right)$ denotes the distance between different patches of ${\mathbf{X}}_i^{\left(s\right)}$ and ${\mathbf{X}}_j^{\left(s\right)}$, and ${\mathcal{N}}_i^{x\left(s\right)}$ denotes the KNN index set of ${\mathbf{X}}_i^{\left(s\right)}$ in the vertex set $V_{t1}^{\left(s\right)}$, and ${\mathbf{W}}_{t1}^{\left(s\right)}$ denotes the weighting matrix.

Regarding the distance metric $dist\left({\mathbf{X}}_i^{\left(s\right)},{\mathbf{X}}_j^{\left(s\right)}\right)$, due to the influence of multiplicative speckle noise following a gamma distribution in SAR images, we employ the following function derived by using the generalized likelihood ratio in [41]:

$$\begin{array}{c}\hfill d_{i,j}^{x\left(s\right)}=dist\left({\mathbf{X}}_i^{\left(s\right)},{\mathbf{X}}_j^{\left(s\right)}\right)=\frac{1}{{\left(sp\right)}^2}\sum _{q=1}^{{\left(sp\right)}^2}log\left(\frac{x_{i,q}^{\left(s\right)}+x_{j,q}^{\left(s\right)}}{2\sqrt{x_{i,q}^{\left(s\right)}{x}_{j,q}^{\left(s\right)}}}\right),\end{array}$$

(2)

where $x_{i,q}^{\left(s\right)}$ and $x_{j,q}^{\left(s\right)}$ are the q-th pixels in the patches of ${\mathbf{X}}_i^{\left(s\right)}$ and ${\mathbf{X}}_j^{\left(s\right)}$, respectively. For the construction of the KNN index set ${\mathcal{N}}_i^{x\left(s\right)}$, it can be obtained by sorting the distance vector $\left\{d_{i,j}^{x\left(s\right)}|j=1,2,\cdots ,N_s\right\}$ and extracting the indices of the smallest k values among them.

For the post-event image $\tilde{\mathbf{Y}}$ obtained at time $t_2$, we construct S different-scale graph models $G_{t2}^{\left(s\right)}=\left\{V_{t2}^{\left(s\right)},E_{t2}^{\left(s\right)},{\mathbf{W}}_{t2}^{\left(s\right)}\right\}$ in a similar manner as follows:

$$\begin{array}{cc}\hfill & V_{t2}^{\left(s\right)}=\left\{{\mathbf{Y}}_i^{\left(s\right)}|i=1,\cdots ,N_s\right\}\hfill \\ \hfill & E_{t2}^{\left(s\right)}=\left\{\left({\mathbf{Y}}_i^{\left(s\right)},{\mathbf{Y}}_j^{\left(s\right)}\right)|j\in {\mathcal{N}}_i^{y\left(s\right)};i=1,\cdots ,N_s\right\}\hfill \\ \hfill & W_{t2}^{\left(s\right)}\left(i,j\right)=\left\{\begin{array}{cc}exp\left(-\lambda d_{i,j}^{y\left(s\right)}\right);& \mathrm{if}\left({\mathbf{Y}}_i^{\left(s\right)},{\mathbf{Y}}_j^{\left(s\right)}\right)\in E_{t2}^{\left(s\right)}\\ 0;& \mathrm{otherwise},\end{array}\right.\hfill \end{array}$$

(3)

where $d_{i,j}^{y\left(s\right)}=dist\left({\mathbf{Y}}_i^{\left(s\right)},{\mathbf{Y}}_j^{\left(s\right)}\right)$ denotes the distance between different patches of ${\mathbf{Y}}_i^{\left(s\right)}$ and ${\mathbf{Y}}_j^{\left(s\right)}$, and ${\mathcal{N}}_i^{y\left(s\right)}$ denotes the KNN index set of ${\mathbf{Y}}_i^{\left(s\right)}$ in the vertex set $V_{t2}^{\left(s\right)}$ that are computed by sorting the distance vector of $\left\{d_{i,j}^{y\left(s\right)}|j=1,2,\cdots ,N_s\right\}$.

2.3. Fusing the Multi-Scale Graphs

Once the multi-scale graphs of $G_{t1}^{\left(s\right)}$ and $G_{t2}^{\left(s\right)}$ are constructed, we employ a fusion strategy to fuse the multi-scale graphs, aiming to comprehensively represent multi-scale information within the images. This enables the graph model to more accurately capture the relationships among objects of various sizes in the SAR image.

Taking the pre-event image $\tilde{\mathbf{X}}$ as an example, for each patch ${\mathbf{X}}_i^{\left(s\right)}$, $i=1,2,\cdots ,N_s$, $s\ge 2$, at the coarsest scale parameter, it can be derived by merging several patches ${\mathbf{X}}_j^{\left(1\right)}$ at the finest scale parameter. This implies the presence of a parent–child aggregation relationship among these patches under different scales. Since patches at the finest scale parameter retain more detailed information and exhibit higher internal homogeneity, we propose to design a fusion matrix to integrate $G_{t1}^{\left(s\right)},s\ge 2$ and $G_{t1}^{\left(1\right)}$. We intend to utilize the area ratio and intensity similarity between child-patches and parent-patches to define S fusion matrices, denoted as ${\mathbf{F}}_{t1}^{\left(s\right)}\in {\mathbb{R}}^{N_1\times N_s}$, with elements defined as follows:

$$\begin{array}{c}\hfill F_{t1}^{\left(s\right)}\left(i,j\right)=\left\{\begin{array}{cc}\frac{\#\left({\mathbf{X}}_i^{\left(1\right)}\right)}{\#\left({\mathbf{X}}_j^{\left(s\right)}\right)}exp\left(-\lambda ·\overline{dist}\left({\mathbf{X}}_i^{\left(1\right)},{\mathbf{X}}_j^{\left(s\right)}\right)\right);& \mathrm{if}{\mathbf{X}}_i^{\left(1\right)}\subseteq {\mathbf{X}}_j^{\left(s\right)}\\ 0& \mathrm{otherwise},\end{array}\right.\end{array}$$

(4)

where $\#\left({\mathbf{X}}_j^{\left(s\right)}\right)$ represents the number of pixels contained in the patch ${\mathbf{X}}_j^{\left(s\right)}$, then we have $\frac{\#\left({\mathbf{X}}_i^{\left(1\right)}\right)}{\#\left({\mathbf{X}}_j^{\left(s\right)}\right)}=1/s^2$. And $\overline{dist}\left({\mathbf{X}}_i^{\left(1\right)},{\mathbf{X}}_j^{\left(s\right)}\right)$ represents the mean radiometric intensity distance between two patches, defined as $\overline{dist}\left({\mathbf{X}}_i^{\left(1\right)},{\mathbf{X}}_j^{\left(s\right)}\right)=log\frac{mean\left({\mathbf{X}}_i^{\left(1\right)}\right)+mean\left({\mathbf{X}}_j^{\left(s\right)}\right)}{2\sqrt{mean\left({\mathbf{X}}_i^{\left(1\right)}\right)mean\left({\mathbf{X}}_j^{\left(s\right)}\right)}}$, with $mean\left({\mathbf{X}}_j^{\left(s\right)}\right)=\frac{1}{s^2{p}^2}{\displaystyle \sum _{q=1}^{s^2{p}^2}}{x}_{j,q}^{\left(s\right)}$ denoting the average intensity value. If the intensity values of the parent-patch and child-patch are close, and their area ratio is substantial, the corresponding element value in the fusion matrix ${\mathbf{F}}^{\left(s\right)}$ would be higher. This is because in such cases, the parent-patch and child-patch would contain the same or closely similar types of objects. Consequently, we can obtain the fused graph $G_{t1}^f=\left\{V_{t1}^f,E_{t1}^f,{\mathbf{W}}_{t1}^f\right\}$ after fusing multi-scale information with

$$\begin{array}{cc}\hfill & V_{t1}^f=\left\{{\mathbf{X}}_i^{\left(1\right)}|i=1,\cdots ,N_1\right\}\hfill \\ \hfill & E_{t1}^f=\left\{\left({\mathbf{X}}_i^{\left(1\right)},{\mathbf{X}}_j^{\left(1\right)}\right)\mid W_{t1}^f\left(i,j\right)\ne 0\right\}\hfill \\ \hfill & {\mathbf{W}}_{t1}^f=\sum _{s=1}^S{\mathbf{F}}_{t1}^{\left(s\right)}{\mathbf{W}}_{t1}^{\left(s\right)}{\left({\mathbf{F}}_{t1}^{\left(s\right)}\right)}^T.\hfill \end{array}$$

(5)

In this way, the multi-scale graphs with different number of vertices are fused.

For the post-event image $\tilde{\mathbf{Y}}$, obtained at time $t_2$, we fuse the S different-scale graphs $G_{t2}^{\left(s\right)}$ in a similar manner to construct the graph $G_{t2}^f=\left\{V_{t2}^f,E_{t2}^f,{\mathbf{W}}_{t2}^f\right\}$ as follows:

$$\begin{array}{cc}\hfill & V_{t2}^f=\left\{{\mathbf{Y}}_i^{\left(1\right)}|i=1,\cdots ,N_1\right\}\hfill \\ \hfill & E_{t2}^f=\left\{\left({\mathbf{Y}}_i^{\left(1\right)},{\mathbf{Y}}_j^{\left(1\right)}\right)\mid W_{t2}^f\left(i,j\right)\ne 0\right\}\hfill \\ \hfill & {\mathbf{W}}_{t2}^f=\sum _{s=1}^S{\mathbf{F}}_{t2}^{\left(s\right)}{\mathbf{W}}_{t2}^{\left(s\right)}{\left({\mathbf{F}}_{t2}^{\left(s\right)}\right)}^T,\hfill \end{array}$$

(6)

where ${\mathbf{F}}_{t2}^{\left(s\right)}\in {\mathbb{R}}^{N_1\times N_s}$ denotes the s-th fusion matrix, similar to the construction of ${\mathbf{F}}_{t1}^{\left(s\right)}$ in (4).

The fusion strategy mentioned above is based on the following considerations: The finest scale graph can retain more detailed information and better avoid the issue of inaccurate feature representation caused by a patch containing multiple land cover types. However, relying solely on a patch at the finest scale is insufficient to accurately represent objects with various shapes and sizes. Reasonably fusing information from coarser scales helps extract multi-scale features from the image, resulting in richer image structural information.

2.4. Spatio-Temporal-Radiometric Interaction

After obtaining the fused multi-scale graphs of $G_{t1}^f$ and $G_{t2}^f$, they respectively depict the structures of the preceding and succeeding temporal images, encompassing the spatial and radiometric information of each image. Subsequently, we employ a graph mapping approach to construct new graphs of $G_{t1}^m$ and $G_{t2}^m$ to facilitate the temporal interaction between these two graph models. First, we map the graph $G_{t1}^f$ (5) constructed on pre-event image $\tilde{\mathbf{X}}$ into the post-event image $\tilde{\mathbf{Y}}$ to obtain the mapped graph $G_{t1}^m=\left\{V_{t1}^m,E_{t1}^m,{\mathbf{W}}_{t1}^m\right\}$ as

$$\begin{array}{cc}\hfill & V_{t1}^m=\left\{{\mathbf{Y}}_i^{\left(1\right)}|i=1,\cdots ,N_1\right\}\hfill \\ \hfill & E_{t1}^m=\left\{\left({\mathbf{Y}}_i^{\left(1\right)},{\mathbf{Y}}_j^{\left(1\right)}\right)\mid W_{t1}^m\left(i,j\right)\ne 0\right\}\hfill \\ \hfill & {\mathbf{W}}_{t1}^m=\sum _{s=1}^S{\mathbf{F}}_{t2}^{\left(s\right)}{\mathbf{Q}}_{t2}^{\left(s\right)}{\left({\mathbf{F}}_{t2}^{\left(s\right)}\right)}^T,\hfill \end{array}$$

(7)

with the matrix ${\mathbf{Q}}_{t2}^{\left(s\right)}$ defined as:

$$\begin{array}{c}\hfill Q_{t2}^{\left(s\right)}\left(i,j\right)=\left\{\begin{array}{cc}exp\left(-\lambda d_{i,j}^{y\left(s\right)}\right);& \mathrm{if}{W}_{t1}^{\left(s\right)}\left(i,j\right)\ne 0\\ 0;& \mathrm{otherwise}.\end{array}\right.\end{array}$$

(8)

By comparing $G_{t1}^f$ (5) and $G_{t1}^m$ (7), we have that because the positions of the nonzero elements in matrices ${\mathbf{W}}_{t1}^{\left(s\right)}$ and ${\mathbf{Q}}_{t2}^{\left(s\right)}$ are the same, and the positions of the nonzero elements in ${\mathbf{F}}_{t1}^{\left(s\right)}$ and ${\mathbf{F}}_{t2}^{\left(s\right)}$ are also the same, then the positions of the nonzero elements in matrices ${\mathbf{W}}_{t1}^f$ and ${\mathbf{W}}_{t1}^m$ are the same; that is, the edge connectivity in graphs $G_{t1}^f$ and $G_{t1}^m$ are the same. However, the computation of their edge weights differs, namely ${\mathbf{W}}_{t1}^f$ and ${\mathbf{W}}_{t1}^m$ are computed based on images $\tilde{\mathbf{X}}$ and $\tilde{\mathbf{Y}}$, respectively. Contrasting $G_{t2}^f$ (6) and $G_{t1}^m$ (7) reveals that, while their edge weights are both based on image $\tilde{\mathbf{Y}}$, their edge connectivity is distinct. Hence, it can be observed that $G_{t1}^m$ encompasses both structural information from image $\tilde{\mathbf{X}}$ (as seen in the edge connectivity of $E_{t1}^m$) and radiometric intensity information from image $\tilde{\mathbf{Y}}$ (as seen in the computation of edge weights ${\mathbf{W}}_{t1}^m$).

Second, we map the graph $G_{t2}^f$ (6) constructed on post-event image $\tilde{\mathbf{Y}}$ into the pre-event image $\tilde{\mathbf{X}}$ to obtain the mapped graph $G_{t2}^m=\left\{V_{t2}^m,E_{t2}^m,{\mathbf{W}}_{t2}^m\right\}$ as:

$$\begin{array}{cc}\hfill & V_{t2}^m=\left\{{\mathbf{X}}_i^{\left(1\right)}|i=1,\cdots ,N_1\right\}\hfill \\ \hfill & E_{t2}^m=\left\{\left({\mathbf{X}}_i^{\left(1\right)},{\mathbf{X}}_j^{\left(1\right)}\right)\mid W_{t2}^m\left(i,j\right)\ne 0\right\}\hfill \\ \hfill & {\mathbf{W}}_{t2}^m=\sum _{s=1}^S{\mathbf{F}}_{t1}^{\left(s\right)}{\mathbf{Q}}_{t1}^{\left(s\right)}{\left({\mathbf{F}}_{t1}^{\left(s\right)}\right)}^T,\hfill \end{array}$$

(9)

with the matrix ${\mathbf{Q}}_{t1}^{\left(s\right)}$ defined as:

$$\begin{array}{c}\hfill Q_{t1}^{\left(s\right)}\left(i,j\right)=\left\{\begin{array}{cc}exp\left(-\lambda d_{i,j}^{x\left(s\right)}\right);& \mathrm{if}{W}_{t2}^{\left(s\right)}\left(i,j\right)\ne 0\\ 0;& \mathrm{otherwise}.\end{array}\right.\end{array}$$

(10)

Similarly, we have that $G_{t2}^m$ encompasses both structural information from image $\tilde{\mathbf{Y}}$ (as seen in the edge connectivity of $E_{t2}^m$) and radiometric intensity information from image $\tilde{\mathbf{X}}$ (as seen in the computation of edge weights ${\mathbf{W}}_{t2}^m$).

2.5. DI Calculation

Based on the structure consistency between multi-temporal images [12,28], for unchanged images, despite differences in imaging conditions between the preceding and subsequent moments (such as sensor variations, humidity, etc.)—which may result in inconsistent radiometric intensity values and consequently pixel value discrepancies for the same objects—the structural characteristics of the two images remain consistent. In other words, if image patches ${\mathbf{X}}_i^{\left(1\right)}$ and ${\mathbf{X}}_j^{\left(1\right)}$ represent identical types of objects (i.e., ${\mathbf{X}}_i^{\left(1\right)}$ and ${\mathbf{X}}_j^{\left(1\right)}$ are highly similar), then, for unchanged patches ${\mathbf{Y}}_i^{\left(1\right)}$ and ${\mathbf{Y}}_j^{\left(1\right)}$ in the other image, they will likewise represent identical types of objects (in this case, ${\mathbf{Y}}_i^{\left(1\right)}$ and ${\mathbf{Y}}_j^{\left(1\right)}$ are also highly similar). Conversely, when the area represented by image patches undergoes changes, this similarity relationship—which signifies structural consistency—will also change, rendering it suitable for detecting regions of change.

To quantify the change level in image patches, a careful comparison of the structural information represented by different graph models reveals that they depict the structural differences between the multi-temporal images. Specifically, to measure the change level ${\alpha }_i$ of the i-th patch ${\mathbf{X}}_i^{\left(1\right)}$, we compare the similarity relationships between ${\mathbf{X}}_i^{\left(1\right)}$ and the unchanged ${\mathbf{X}}_j^{\left(1\right)}$ in the graph $G_{t1}^f$ and the similarity relationships between ${\mathbf{X}}_i^{\left(1\right)}$ and the unchanged ${\mathbf{X}}_j^{\left(1\right)}$ in the graph $G_{t2}^m$, that is,

$$\begin{array}{c}\hfill {\alpha }_i=\left|\frac{{\sum }_{j=1}^{N_1}{W}_{t1}^f\left(i,j\right)\left(1-p_j\right)}{{\sum }_{j=1}^{N_1}\delta \left(W_{t1}^f\left(i,j\right)\ne 0\right)\left(1-p_j\right)+ϵ}-\frac{{\sum }_{j=1}^{N_1}{W}_{t2}^m\left(i,j\right)\left(1-p_j\right)}{{\sum }_{j=1}^{N_1}\delta \left(W_{t2}^m\left(i,j\right)\ne 0\right)\left(1-p_j\right)+ϵ}\right|,\end{array}$$

(11)

where $ϵ={10}^{-8}$ is used to make the equation well-defined, $\delta (·)$ equals 1 if the specified condition inside holds, and 0 otherwise, and $p_j$ represents the change probability of the j-th patch, which is used to reduce the influence of the changed patches in the change measurement.

Similarly, we can compute the structural differences of the post-event image on graphs $G_{t2}^f$ and $G_{t1}^m$ to calculate the change level ${\beta }_i$ of the i-th patch ${\mathbf{Y}}_i^{\left(1\right)}$ as:

$$\begin{array}{c}\hfill {\beta }_i=\left|\frac{{\sum }_{j=1}^{N_1}{W}_{t2}^f\left(i,j\right)\left(1-p_j\right)}{{\sum }_{j=1}^{N_1}\delta \left(W_{t2}^f\left(i,j\right)\ne 0\right)\left(1-p_j\right)+ϵ}-\frac{{\sum }_{j=1}^{N_1}{W}_{t1}^m\left(i,j\right)\left(1-p_j\right)}{{\sum }_{j=1}^{N_1}\delta \left(W_{t1}^m\left(i,j\right)\ne 0\right)\left(1-p_j\right)+ϵ}\right|.\end{array}$$

(12)

In (11) and (12), $p_j$ is used to represent the change probability. However, since we cannot pre-determine the change probability for each image patch, an iterative process is employed to compute the change probability vector $\mathbf{p}$ with four steps.

Step 1. Compute the initial change probability ${\mathbf{p}}^o$ by using the Log-ratio operator, and compute the initial ${\alpha }^o$ and ${\beta }^o$ with ${\mathbf{p}}^o$ by using (11) and (12), respectively.

Step 2. Compute the fused change level ${\eta }^o=\frac{{\alpha }^o+{\beta }^o}{2}$, and update the change probability $\mathbf{p}=\frac{{\eta }^o-min\left\{{\eta }^o\right\}}{max\left\{{\eta }^o\right\}-min\left\{{\eta }^o\right\}}$.

Step 3. Substitute $\mathbf{p}$ into (11) and (12) to update the $\alpha$ and $\beta$, respectively.

Step 4. Compute the fused change level $\eta =\frac{\alpha +\beta }{2}$ and the final change probability of ${\mathbf{p}}^{*}$ as ${\mathbf{p}}^{*}=\frac{\eta -min\left\{\eta \right\}}{max\left\{\eta \right\}-min\left\{\eta \right\}}$.

The purpose of this iterative process to obtain more accurate structural differences between images and change probability vector ${\mathbf{p}}^{*}$, which in turn results in better DI computed by using ${\mathbf{p}}^{*}$. As can be seen from Step 1 to Step 4, we used (11) and (12) twice to iteratively correct the change probability vector $\mathbf{p}$. Therefore, the main computational cost of this iterative process is the two calculations of (11) and (12), which is a very low computational complexity of $\mathcal{O}(N_1\times N_1)$.

Then, we obtain the change probability for each node (patch) and compute the DI by measuring the structural differences between different graph models.

2.6. CM Computation by MRF Segmentation

After obtaining the DI, the change detection problem can be regarded as an image binary classification task, which can be addressed using the following approaches: (1) threshold segmentation methods, such as Otsu’s method [42]; (2) clustering methods, like k-means clustering [43] and fuzzy c-means clustering [44]; (3) random field methods, such as Markov random field (MRF) [45]. Thresholding and clustering methods often neglect the neighborhood information of pixels in the DI, leading to the emergence of considerable “salt-and-pepper noise” in the segmented CM. MRF can incorporate both spatial and change information from the DI, thereby improving the outcomes of change detection. In this paper, we directly adopted the MRF segmentation method proposed in [45], but with a modification where we replaced the R-adjacency spatial neighbors of superpixels in [45] with the 8-connected neighbors of image patches. Once the MRF model is solved by using the min-cut/max-flow algorithm [46], we can obtain the change label of each patch, and subsequently generate the CM. The framework of the proposed multi-scale graph based method with spatio-temporal-radiometric information interaction for CD is summarized in Algorithm 1.

Algorithm 1: STRMG based CD.

Input: Images of $\tilde{\mathbf{X}}$ and $\tilde{\mathbf{Y}}$.             Parameters of p and S. Pre-processing:    Segment $\tilde{\mathbf{X}}$ and $\tilde{\mathbf{Y}}$ into patches with different scales.    Stack the patches to obtain the multi-scale PGM of ${\mathbf{X}}^{\left(s\right)}$ and ${\mathbf{Y}}^{\left(s\right)}$. Computing the DI:    Construct the multi-scale graphs of $G_{t1}^{\left(s\right)}$ and $G_{t2}^{\left(s\right)}$.    Compute the fusion matrices of ${\mathbf{F}}_{t1}^{\left(s\right)}$ and ${\mathbf{F}}_{t2}^{\left(s\right)}$.    Fuse the multi-scale graphs to obtain graphs of $G_{t1}^f$ and $G_{t2}^f$.    Construct the mapped graphs of $G_{t1}^m$ and $G_{t2}^m$.    Compute the change probability vector of ${\mathbf{p}}^{*}$. Computing the CM:    Compute final CM by using MRF segmentation method.

3. Experiment Results and Discussion

In this section, we validate the performance of the proposed algorithm through comparative experiments with various SAR change detection methods on three real datasets.

3.1. Experimental Settings

We conducted tests on three pairs of SAR images. As illustrated in Figure 2, Dataset A and Dataset B were acquired using the Radarsat-2 SAR sensor over the Yellow River Estuary in China. The pre-event images were obtained in June 2008, while the post-event images were captured in June 2009. The spatial resolution of the images was 8 m per pixel. Dataset A shows the changes in the coastline, where only a small portion of the pixels of the image have changed in this dataset. Dataset B shows a section of inland water, which was chosen because the area of change is concentrated on the boundary line of the river, which is relatively difficult to detect. Dataset C was collected using the COSMO-SkyMed SAR sensor in Guizhou, China. The pre-event images were captured in June 2016, and the post-event images were acquired in April 2017; they encompass various features such as mountains, rivers, and trees. The spatial resolution of the images was 1 m per pixel. The ground change maps are labeled by experts based on these SAR images, optical images from the same time period, and in combination with practical examinations. The details of these datasets are presented in

Table 1. Description of the datasets.

Dataset	Date	Sensor	Location	Image Size	Resolution	Polarizations	Waveband
Dataset A	June 2008–June 2009	Radarsat-2	Yellow River Estuary, China	$280\times 450$	8 m	HH	C-band
Dataset B	June 2008–June 2009	Radarsat-2	Yellow River Estuary, China	$444\times 291$	8 m	HH	C-band
Dataset C	June 2016–April 2017	COSMO-SkyMed	Guizhou, China	$400\times 400$	1 m	HH	X-band

.

We selected the following ten SAR change detection methods for comparison: the classical operators of difference (Diff), log-ratio (LogR) [16], mean-ratio (MeanR) [17], neighborhood-ratio operator (NbhdR) [18], the sparsity-driven method for CD (SDCD) [14], and the recently proposed graph-based methods of improved nonlocal patch-based graph model (INLPG) [12], iterative robust graph with MRF co-segmentation method (IRG-McS) [29], the graph signal processing-based CD method (GSPCD) [28], as well as the deep learning-based methods of convolutional-wavelet neural network- (CWNN) [8] and deep convolutional generative adversarial network-based robust unsupervised CD method (DcGANCD) [10]. For these comparative methods (excluding the classical operators), we directly utilized their open-source code with default parameters. For the proposed STRMG, we set the patch size as $p=2$ and the scale parameter as $S=3$.

To assess the performance of different methods, we employed two categories of evaluation metrics. Firstly, we directly assessed the DI’s quality using: (1) the empirical receiver operating characteristics (ROC) curve along with the areas beneath the ROC curve (AUR); (2) the precision–recall (PR) curve and the corresponding areas under the PR curve (AUP). Secondly, we evaluated the CM’s quality using the false alarm rate (FA) calculated by $\mathrm{FA}=\mathrm{FP}/(\mathrm{FP}+\mathrm{TN})$, the miss rate (MR) calculated by $\mathrm{MR}=\mathrm{FN}/(\mathrm{TP}+\mathrm{FN})$, the overall accuracy (OA) calculated by $\mathrm{OA}=(\mathrm{TP}+\mathrm{TN})/(\mathrm{TN}+\mathrm{FN}+\mathrm{TP}+\mathrm{FP})$, and the Kappa coefficient (KC) calculated by $\mathrm{KC}=(\mathrm{OA}-\mathrm{PRE})/(1-\mathrm{PRE})$ with

$$\begin{array}{c}\hfill \mathrm{PRE}=\frac{(\mathrm{TP}+\mathrm{FN})(\mathrm{TP}+\mathrm{FP})+(\mathrm{TN}+\mathrm{FP})(\mathrm{TN}+\mathrm{FN})}{{(\mathrm{TN}+\mathrm{FN}+\mathrm{TP}+\mathrm{FP})}^2},\end{array}$$

(13)

and the F1-score was calculated by $\mathrm{F}1=2\mathrm{TP}/(\mathrm{FN}+\mathrm{FP}+2\mathrm{TP})$.

3.2. Experimental Results

3.2.1. Results on the Dataset A

In Figure 3, we show the DIs generated by the proposed STRMG on Dataset A against other currently popular methods. Compared to other traditional methods, the proposed STRMG is able to achieve better DI, which is more robust to the noise. For example, the DIs generated by the traditional algebraic operators of Diff, LogR, MeanR and NbhdR are heavily influenced by the speckle noise, as they are all based on direct comparison of pixels or image patches, making them difficult to effectively distinguish between changed and unchanged areas. In contrast, the image structure-based methods perform better due to the relative stability of image structure, such as the GSPCD. Further comparison between the proposed STRMG and other structure-based methods (i.e., INLPG, IRG-McS and GSPCD) shows that STRMG, which incorporates multi-scale information and spatio-temporal-radiometric information, can effectively improve DI performance, which is also confirmed by the ROC and PR curves plotted in Figure 4.

For a more intuitive understanding of the differences between our proposed method and other approaches, we also show the CMs on Dataset A in Figure 5, where we distinguish FN and FP by using distinct colors, cyan and red, to facilitate visual examination and comparison. From Figure 5, we can see that there is a significant number of false alarms in CMs generated by LogR, MeanR, NbhdR, and CWNN due to the speckle noise in SAR images. In contrast, INLPG, DcGANCD, and STRMG generate fewer false alarms. Further comparison of these three in Figure 5 also shows that STRMG has the least missed detections. In

Table 2. Quantitative measures of DIs and CMs on Dataset A.

Methods	AUR ↑	AUP ↑	FA ↓	MR ↓	OA ↑	KC ↑	F1 ↑
Diff	0.845	0.086	0.083	0.461	0.912	0.099	0.116
LogR	0.851	0.086	0.241	0.208	0.759	0.046	0.066
MeanR	0.973	0.813	0.352	0.029	0.652	0.036	0.056
NbhdR	0.983	0.758	0.386	0.008	0.619	0.033	0.053
SDCD	0.943	0.342	0.044	0.216	0.955	0.257	0.270
INLPG	0.992	0.879	0.001	0.246	0.996	0.813	0.815
DcGANCD	0.975	0.808	0.001	0.281	0.996	0.784	0.787
CWNN	0.971	0.668	0.106	0.038	0.895	0.147	0.163
IRG-McS	0.954	0.229	0.020	0.197	0.978	0.429	0.438
GSPCD	0.931	0.711	0.005	0.271	0.993	0.674	0.678
STRMG	0.992	0.912	0.001	0.095	0.998	0.886	0.887

, we present the quantitative comparison results of the DIs and CMs generated by these methods on Dataset A. It can be seen that the proposed STRMG achieves the best scores across all evaluation criteria. Compared to the advanced deep learning-based CWNN and DcGANCD, our method has at least a 10.0% improvement in F1-scores. When compared to the recently proposed graph model-based INLPG, IRG-McS, and GSPCD, our method shows an improvement of 7.2–44.9% in F1-scores.

3.2.2. Results on the Dataset B

Figure 6 shows the DIs generated by all the comparison methods on the Dataset B, from which it can be seen that most of the methods are able to highlight the changed areas to some extent. A careful comparison shows that the Diff, LogR, and SDCD are weak at distinguishing between changed and unchanged areas, which is mainly affected by inherent speckle noise. The DI results from Dataset A and Dataset B also confirm the poor detection performance achieved by traditional algebraic operators for SAR image CD where the speckle noise is more severe, whereas graph model-based methods such as INLPG, IRG-McS, GSPCD, and STRMG are able to take full advantage of the structural relationships within the image that are relatively robust to noise, thus achieving better detection results. Figure 7 displays the corresponding ROC and PR curves, and we can find that the proposed STRMG can obtain the highest AUR and AUP, as reported in

Table 3. Quantitative measures of DIs and CMs on Dataset B.

Methods	AUR ↑	AUP ↑	FA ↓	MR ↓	OA ↑	KC ↑	F1 ↑
Diff	0.788	0.210	0.257	0.323	0.741	0.094	0.147
LogR	0.916	0.520	0.185	0.130	0.817	0.192	0.238
MeanR	0.974	0.802	0.314	0.023	0.696	0.122	0.175
NbhdR	0.979	0.805	0.181	0.029	0.824	0.222	0.266
SDCD	0.907	0.439	0.183	0.140	0.818	0.192	0.238
INLPG	0.992	0.825	0.007	0.290	0.984	0.732	0.741
DcGANCD	0.983	0.822	0.005	0.218	0.988	0.810	0.816
CWNN	0.978	0.823	0.012	0.114	0.985	0.783	0.791
IRG-McS	0.977	0.515	0.013	0.160	0.983	0.751	0.760
GSPCD	0.970	0.790	0.007	0.211	0.986	0.787	0.794
STRMG	0.995	0.920	0.007	0.085	0.991	0.860	0.865

.

In Figure 8, we show the CMs generated by different methods on Dataset B, from which we can find that DcGANCD, CWNN, GSPCD, and STRMG perform better than other comparison methods, especially with fewer false alarms in their CMs. In addition, further comparison also reveals that the missed detections in STRMG are also relatively rare with a small MR of 0.085, as listed in Table 3; thus, STRMG is able to obtain better detection results than the other compared methods, for example, its KC and F1 are higher than the second-ranked DcGANCD by 5.0% and 4.9%, respectively.

3.2.3. Results on the Dataset C

Figure 9 presents the DIs of various methods on Dataset C. It can be observed that some methods do not perform as well on this dataset as they did on the first two datasets, such as the INLPG, CWNN, and IRG-McS. This is due to two reasons: one is that the noise on the SAR images in Dataset C is much more severe, which leads to difficulties in accurately portraying the structures of images, and tends to produce a lot of bright patches in the difference maps, which can be mistakenly detected as changed regions; on the other hand, there are very few changed regions in this dataset, i.e., the changed and unchanged categories are extremely unbalanced, which can easily lead to the changed regions being submerged in the unchanged regions, resulting in missed detection. In comparison, the STRMG proposed in this paper is able to achieve better DI, which can also demonstrated by the ROC and PR curves shown in Figure 10.

We present the CMs of each method on Dataset C in Figure 11. From Figure 11a–f, it can be seen that some methods create a large number of false alarms in the change detection results, leading to very high FA values of 10.4–55.1% as listed in

Table 4. Quantitative measures of DIs and CMs on Dataset C.

Methods	AUR ↑	AUP ↑	FA ↓	MR ↓	OA ↑	KC ↑	F1 ↑
Diff	0.915	0.303	0.353	0.080	0.648	0.020	0.032
LogR	0.785	0.088	0.104	0.789	0.891	0.012	0.024
MeanR	0.970	0.292	0.314	0.015	0.688	0.027	0.039
NbhdR	0.902	0.050	0.348	0.056	0.654	0.021	0.034
SDCD	0.982	0.493	0.118	0.026	0.883	0.085	0.096
INLPG	0.960	0.614	0.551	0.009	0.452	0.010	0.023
DcGANCD	0.991	0.806	0.002	0.286	0.996	0.714	0.716
CWNN	0.912	0.048	0.107	0.386	0.891	0.056	0.067
IRG-McS	0.930	0.247	0.005	0.439	0.993	0.488	0.492
GSPCD	0.965	0.247	0.003	0.307	0.995	0.635	0.638
STRMG	0.994	0.900	0.001	0.125	0.998	0.859	0.860

. By comparing Figure 11g,i–k, we can observe that, although DcGANCD, IRG-McS, and GSPCD yield relatively clean detection results with fewer false alarms, they also have difficulty in detecting smaller changed regions. In contrast, the proposed STRMG produces fewer detection errors and more accurate edge compared to other methods. In particular, the F1-score of STRMG is higher than the second-ranked DcGANCD by 14.4%. This demonstrates the effectiveness of the multi-scale graph model and the spatio-temporal-radiometric information interaction in the method.

3.3. Ablation Study and Discussion

3.3.1. Ablation Study

To validate the effectiveness and rationality of the components in STRMG, we conducted ablation experiments on all the evaluated datasets. We construct a baseline by directly comparing the single-scale graphs of $G_{t1}^{\left(1\right)}$ and $G_{t2}^{\left(1\right)}$ to detect the changes without utilizing the multi-scale information and spatio-temporal-radiometric interaction between the graph models. We use the aforementioned metrics to evaluate the impact of the multi-scale graph (MsG) and spatio-temporal-radiometric interaction (STR) on change detection performance.

Table 5. Ablation study of STRMG measured by the average scores.

Methods	AUR ↑	AUP ↑	FA ↓	MR ↓	OA ↑	KC ↑	F1 ↑
Base	0.982	0.736	0.006	0.151	0.992	0.775	0.778
Base+MsG	0.987	0.851	0.004	0.198	0.994	0.809	0.812
Base+STR	0.990	0.878	0.003	0.118	0.995	0.851	0.854
STRMG	0.994	0.911	0.003	0.102	0.995	0.868	0.871

reports the average quantitative measures of the baseline model (denoted as “Base”), and the baseline model with multi-scale graph (denoted as “Base+MsG”), the baseline model with spatio-temporal-radiometric interaction (denoted as “Base+STR”), and the proposed STRMG.

By comparing the “Base” and “STRMG”, we can find that, when multi-scale graphs are not used to mine the multi-scale information in SAR images and the spatio-temporal-radiometric interaction is not used in the graph mapping, the change detection performance will greatly decrease. As listed in Table 5, when the multi-scale graphs are employed in the structure representation, the average AUP is increased by 11.5% by comparing the “Base” and “Base+MsG”. When the spatio-temporal-radiometric interaction is utilized in the structure comparison, more accurate detection results are obtained, leading to a 7.6% increase of KC by comparing the “Base” and “Base+STR” and a 5.9% increase of KC by comparing the “Base+MsG” and “STRMG”.

3.3.2. Parameter Analysis

Next, we discuss the impact of the scale parameter S and patch size p on the proposed STRMG. In Figure 12, we show the KC of CMs obtained by STRMG on all the evaluated datasets with different scale parameters, i.e., $S=2,3,4$. As shown in Figure 12, higher-order multi-scale graphs give better detection results than multi-scale graphs using only two layers ($S=2$), especially on Dataset A. The KC values obtained at $S=3$ are 3.5%, 1.2%, and 2.3% higher than those obtained at $S=2$ on the three Datasets, A, B, and C, respectively. In addition, the performance tends to slightly decrease when $S=4$. This could be attributed to the larger scale, where individual image patches encompass various types of land features, resulting in less accurate representation of image structures by the larger-scale graph models. Thus, in this paper, we set the scale parameter as $S=3$.

In Figure 13, we vary the patch size p from 2 to 6 with step one. It can be found that the patch size has an important impact on the STRMG. When the patch size p is too large, such as $p=6$, it will cause a dramatic degradation of STRMG performance. This is mainly due to the fact that too-large image patches result in over-smoothing and the fact that the interior of a single image patch may contain different types of objects (internal homogeneity is destroyed), making it difficult for the constructed graph model to accurately portray the image structure. Obviously, setting $p=2$ is a good choice as shown in the Figure 13.

3.3.3. Test of Different Noise Levels

To evaluate the performance of the proposed STRMG under different noise conditions, we use a simulated SAR dataset as shown in the top row of Figure 14, where Figure 14a,b show two simulated multi-temporal images, and Figure 14c shows the ground truth image. By adding different levels of multiplicative gamma-distributed speckle noise ($L=10,5,2$) to the multi-temporal images, we can obtain different DIs of STRMG at different noise levels, as shown in Figure 14c. From these results, we can find that the proposed STRMG is very robust to the speckle noise. This is due to the fact that STRMG utilizes the image structure represented by the similarity between image patches, thus mitigating the effect of noise. Moreover, since the proposed method is insensitive to speckle noise, we believe it can be valuable in other similar application scenarios; for example, in the change detection of synthetic aperture sonar (SAS) images [47,48,49,50], as the basic theory of SAS and SAR is identical to some degree.

4. Conclusions

In this study, we have proposed a novel multi-scale graph-based method with spatio-temporal-radiometric information interaction, named STRMG, for SAR image change detection. To mitigate the impact of inherent speckle noise in SAR images on change detection, we have chosen to compare the structural relationships between two images, which is more robust to noise. This transforms the change detection problem into an assessment of image structural differences. To more accurately depict the intricate structural relationships within the images, we have constructed multi-scale graph models and designed an effective graph fusion strategy, which can fully exploit the spatial information, radiometric information, and multi-scale details within the images. In order to adequately compare graph models to accurately detect changes, we have employed the graph mapping approach to realize the spatio-temporal-radiometric information interaction between graph models, which can effectively explore the associations and differences between multi-temporal images, leading to more accurate change detection results. Experiments on different datasets have demonstrated the effectiveness of the proposed method. In addition, compared with the traditional KNN graph, graph neural networks (GNN) have stronger structural characterization ability, which can more efficiently capture the complex relationships between land features within the SAR image, especially for the change detection of high-resolution SAR images. Therefore, future work might consider applying the idea of multi-scale graphs and spatio-temporal-radiometric interaction to GNN-based SAR image change detection, expecting a better detection performance.