Statistics Learning Network Based on the Quadratic Form for SAR Image Classification

Abstract

The convolutional neural network (CNN) has shown great potential in many fields; however, transferring this potential to synthetic aperture radar (SAR) image interpretation is still a challenging task. The coherent imaging mechanism causes the SAR signal to present strong fluctuations, and this randomness property calls for many degrees of freedom (DoFs) for the SAR image description. In this paper, a statistics learning network (SLN) based on the quadratic form is presented. The statistical features are expected to be fitted in the SLN for SAR image representation. (i) Relying on the quadratic form in linear algebra theory, a quadratic primitive is developed to comprehensively learn the elementary statistical features. This primitive is an extension to the convolutional primitive that involves both nonlinear and linear transformations and provides more flexibility in feature extraction. (ii) With the aid of this quadratic primitive, the SLN is proposed for the classification task. In the SLN, different types of statistics of SAR images are automatically extracted for representation. Experimental results on three datasets show that the SLN outperforms a standard CNN and traditional texture-based methods and has potential for SAR image classification.

1. Introduction

Synthetic aperture radar (SAR) has significant research value and very broad application prospects because it can obtain ground information all day and during all weather [1], etc. SAR is widely used in land resources, disaster management, agroforestry, oceans and other fields [2,3,4]. Germany, Canada, Russia, China, and other countries have successfully developed SAR systems, such as TerraSAR-X [5], RADARSAT-2 [6], Sentinel-1 [7], and Gaofen-3 [8], etc. With the increasing abundance of SAR image data and the gradually increasing resolution of SAR images [9], the problem of automatically interpreting SAR images has received extensive attention, especially for classification purposes [10].

1.1. Related Work

Over recent years, many methods have been proposed for SAR image classification. These methods can be broadly categorized as statistical models, textural analysis, and deep networks.

Perhaps the most widely used approach is based on statistical analysis. The idea behind this approach is rooted in the statistical nature of SAR echoes. The coherent imaging mechanism causes SAR to exhibit strong fluctuations in echoes, which are known as speckle [11]. This distinctive characteristic allows a statistical model to be used as an effective tool for SAR image analysis. Numerous statistical models have been presented describing SAR images, e.g., Rayleigh [12], Gamma [13], $\mathcal{K}$ [14], Log-normal [15], Weibull [16], Fisher [17], etc. These statistical models provide good approximations to the distributions of various types of SAR data; for example, the $\mathcal{K}$ distribution is appropriate for describing heterogeneous regions [18] and the log-normal distribution shows more potential than Weibull, Gamma and $\mathcal{K}$ in modeling high-resolution TerraSAR-X images [19]. However, accurately approximating the distributions of various SAR data by relying on a single statistical model is still challenging. To cope with this issue, mixture models, such as finite mixture models (FMM) [20] and Mellin transform-based approaches [21], have been used to improve the generalization capability. Moreover, the new generation of SAR systems with high-resolution images increases the challenge of describing the images with a statistical model. In high-resolution SAR, the number of elementary scatterers inside each resolution cell is reduced, which invalidates the central limit theorem and causes the emergence of heterogeneous regions. Consequently, how to precisely model high-resolution SAR data is still an open issue [22]. The idea of statistical analysis is strongly embraced in this paper, which extracts different types of statistics of SAR data for description.

The textural analysis category is another mainstream approach in SAR image classification. Fundamental works have been presented such as the gray-level co-occurrence matrix (GLCM) [23], Gaussian Markov random fields (GMRF) [24], Gabor filter [25], local binary pattern (LBP) [26], and others. In the context of texture analysis, a series of primitives or so-called textons are first explicitly extracted. These primitives capture the elementary patterns embedded in an image. Then, the histogram of these primitives is formed into a feature vector to represent the image. Finally, this feature vector is fed into a classifier such as a support vector machine (SVM) [27] to perform the classification task. For SAR images, different regions, such as vegetation and high-density residential and low-density residential areas, manifest with discriminative textures. Therefore, texture-based methods can be effectively used to capture the underlying patterns. However, the texture feature is a type of low-level or middle-level representation; therefore, the robustness of texture-based methods require improvement, especially for high-resolution SAR images that contain structural and geometrical information. In this paper, the presented method is a multilayer model that extracts a more abstract textural description, i.e., a high-level feature, for SAR image representation.

Deep networks [28] is the third category for SAR image classification, and significant efforts have been made to gradually shift to this paradigm [29,30,31,32,33]. In addition, the convolutional neural network (CNN) [34] can be interpreted as a multilayer mapping function that maps from the input data to the task-related output. Several studies have explored applying deep learning schemes to SAR image classification: (i) One typical work uses the probability distribution. Concretely, probabilistic graphical models such as the restricted Boltzmann machine (RBM) and the deep belief network (DBN) [35] are used to capture the underlying dependencies between the input and output. For example, Liu et al. presented a Wishart-Bernoulli RBM (WRBM) for polarimetric SAR (PolSAR) image classification [36], and both Wishart and Bernoulli distributions have been used to model the conditional probabilities of visible and hidden units. This idea was also adopted in [37]; in fact, the DBN has been tested on urban regions without considering the prior distribution of PolSAR data [38]. Zhao et al. proposed a generalized Gamma deep belief network (g$\Gamma$-DBN) for SAR image statistical modeling and land-cover classification [39]. (ii) Other works have mainly focused on learning effective features for SAR images, where the discriminant function is generally not imposed by a probability distribution. In [40,41], handcrafted features (e.g., HOG, Gabor, GLCM, etc.) are integrated into an autoencoder for feature learning. Zhao et al. presented a discriminant deep belief network (DisDBN) for discriminant as well as high-level feature learning [42]. However, it is often difficult to collect massive amounts of training data. In this case, how to leverage the specific domain knowledge of the SAR mechanism to augment deep model learning is of great importance [43]. (iii) In addition, to make full use of the phase information contained in SAR data, several works have extended real-valued deep models to the complex-valued domain. For example, Ronny presented a complex-valued multilayer perceptron (CV-MLP) for SAR image classification [44], and Zhang et al. proposed a complex-valued CNN (CV-CNN) [45]. Indeed, phases in multiple channels are mutually coherent and thus carry useful information [16]. In this paper, the proposed statistics learning network (SLN) is a typical deep network, and the quadratic primitive used in the SLN is an extension to the convolutional primitive, involving both nonlinear and linear transforms.

1.2. Motivations

The coherent imaging mechanism causes SAR signals to present strong fluctuations. Exploiting this distinctive nature while investigating the potential of CNNs for SAR image interpretation is the motivation of this paper. To establish this goal, the existing challenges are as follows:

• The intrinsic randomness of SAR signal allows statistical models to be an effective tool for SAR image analysis. The parameters of a statistical model capture valuable information for describing the SAR image—for example, the mean $\mu$ and variance ${\sigma }^2$ of a normal distribution $\mathcal{N}\left(\mu ,{\sigma }^2\right)$. However, fitting these distribution parameters using CNNs is still a challenge, especially when insufficient training data is available. Consider the variance ${\sigma }^2$ of $\mathcal{N}\left(\mu ,{\sigma }^2\right)$ as an example. This variable can be estimated by

  $$\begin{array}{cc}\hfill {\widehat{\sigma }}^2& =\frac{1}{n}\sum _{i=1}^n{\left(x_i-\frac{1}{n}\sum _{j=1}^n{x}_j\right)}^2\hfill \\ & =\frac{1}{n}\sum _{i=1}^n\left[x_i^2-\frac{2}{n}\sum _{j=1}^n{x}_i{x}_j+\frac{1}{n^2}{\left(\sum _{j=1}^n{x}_j\right)}^2\right]\hfill \\ & =\left(\frac{1}{n}-\frac{1}{n^2}\right)\sum _{i=1}^n{x}_i^2-\frac{2}{n^2}\sum _{j=1}^{n-1}\sum _{k=j+1}^n{x}_j{x}_k\hfill \end{array},$$

  (1)

  where n denotes the number of pixels, and $x_i$ is the i-th pixel value. It should be noted that both the quadratic terms $x_i^2$ and the cross terms $x_i{x}_j$ are needed in Equation (1). However, in a typical CNN, both the quadratic and cross terms are discarded.
• The CNN has demonstrated a powerful ability to learn features, but that ability relies primarily on the availability of big data. However, collecting massive amounts of SAR data is difficult in practice. Moreover, the coherent imaging mechanism causes the SAR signals to present strong fluctuations. This distinctive nature calls for many degrees of freedom (DoFs) for SAR image description, which increases the difficulty of applying a CNN to SAR image interpretation.

1.3. Contributions

This paper concentrates on the statistical characteristics of SAR signals and presents a statistics learning network (SLN) based on the quadratic form for SAR image classification. The SLN seeks to automatically extract statistical features for that describe the SAR image. The main contributions of this study are summarized as follows.

• A quadratic primitive is designed to comprehensively learn elementary statistical features. The novel aspects of this primitive lie in an adaptation and extension of the standard convolutional primitive to cope with the SAR image. Concretely, the weighted combination of high-order components, including the quadratic and cross terms, are implemented in this primitive. As shown in Section 3.1, the motivation behind this primitive is derived from the finding that the quadratic and cross terms are often needed for elementary statistical parameter fitting.
• With the aid of the quadratic primitive, the SLN is presented for SAR image classification as depicted in Figure 3. The SLN is a type of deep model that seeks to automatically fit statistical features for SAR image representation.

The remainder of this paper is organized as follows. In Section 2, the statistics of the SAR image and the framework of deep learning are briefly reviewed. The quadratic primitive and SLN are detailed in Section 3. Section 4 presents the experiment and discussion. Finally, conclusions are given in Section 5.

2. Preliminaries

This section briefly reviews the statistical properties of a SAR image and the framework of deep learning.

2.1. SAR Image Statistics

SAR is an active remote sensing technology that radiates electromagnetic waves and receives the backscattered echoes. The signal received by SAR can be interpreted as a coherent sum of many elemental scatterers’ echoes within a resolution cell; thus, it can be expressed as follows:

$$A{e}^{j\varphi }=\sum _i^N{A}_i{e}^{j{\varphi }_i},$$

(2)

where N is the number of scatterers, and $A_i$ and ${\varphi }_i$ denote the amplitude and phase of the i-th backscattered echoes, respectively. Due to the randomness of $A_i$ and ${\varphi }_i$, the received signal $A{e}^{j\varphi }$ presents strong pixel-to-pixel fluctuations between adjacent resolution cells, known as speckle [12].

In the fully developed case, the amplitude A follows a Rayleigh distribution, and the intensity $I=A^2$ has an exponential probability density function [12]. However, these statistical models hold only in modeling single-look SAR data backscattered from homogeneous regions. In many practical cases, many statistical models have been presented to fit the distributions of SAR data. These statistical models can be roughly divided into two categories, i.e., empirical distribution model and a priori hypothesis models, which are summarized in

Table 1. Typical statistical models for SAR image analysis.

Type	Model	Application Cases
Empirical Distribution	Log-Normal [15]	homogeneous, amplitude/intensity
	Weibull [16]	homogeneous, single look,
		amplitude/intensity
	Fisher [17]	homogeneous/heterogeneous,
		single look/multi-look, amplitude/intensity
Priori Hypothesis	Rayleigh [12]	homogeneous, single look, amplitude
	Gamma [13]	homogeneous, multi-look, intensity
	$\mathcal{K}$ [14]	heterogeneous, single look/multi-look,
		amplitude/intensity

. For the empirical distribution model, e.g., the log-normal [15], Weibull [16], Fisher [17], etc. distributions are obtained primarily by analyzing SAR data. In contrast, the a priori hypothesis model, is derived from different a priori assumptions and mainly involves distributions such as Rayleigh [12], Gamma [13], $\mathcal{K}$ [14].

Despite the large numbers of statistical models that have been proposed for SAR image analysis, these models are valid only in certain circumstances. For example, the Gamma distribution is effective only when modeling multi-look intensity SAR data within homogeneous regions; however, high-resolution SAR images present both structural and geometrical information, resulting in heterogeneous regions. New statistical models have been presented for high-resolution SAR image, such as the generalized Gamma mixture model (G$\Gamma$MM) [20]. However, when the resolution is improved, SAR signals follow heavy-tailed distributions, and modeling these distributions is a challenging task.

2.2. Deep Learning Framework

Deep learning is characterized by its multilayer architecture, in which linear transformation, probability theory and other types of machine learning algorithms are combined. Various types of deep models have appeared in the literature, e.g., CNN [34], DBN [35], and recurrent neural network (RNN) [46], etc. In particular, CNNs have shown great success in image processing, and many popular architectures have been presented, such as AlexNet [47], VGGNet [48], ResNet [49], GoogLeNet [50], and DenseNet [51]. In a typical CNN, an input undergoes a series of transformations, including convolution, activation, etc. Figure 1 shows an example illustrating the convolutional and tanh (tanh is a type of activation function expressed as $\tan \mathrm{h}\left(x\right)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$.) layers. As shown in Figure 1b, the input is rotated, translated, and scaled by the convolutional layer. However, the relative spatial positions of the two curves remain unchanged, and they still cannot be separated by a straight line. The tanh layer performs nonlinear transformation, and the output of this layer can be linearly separated, as illustrated in Figure 1c.

A deep model is composed of a recursive application of several elementary layers, e.g., convolution, activation, and pooling layers in CNN. Stacking multiple levels of these elementary layers increases the depth of the deep model, allowing it to approximate complex functionality similar to human visual perception in the brain. A deep model is typically composed of the following modules:

• Information extraction module: This module involves the convolutional primitive [34], autoencoder primitive [52], RBM [53], etc. The convolution primitive is used for linear transformation, the autoencoder is used for data compression, and the RBM is used for probability distribution modeling.
• Activation and pooling module: This module is mainly devoted to nonlinear transformation and dimensionality reduction, which are implemented by activation functions and pooling operations, respectively. Popular activation functions include the sigmoid, tanh and rectified linear unit (ReLU), etc. For the pooling operation, max and average pooling are widely used at present [54].
• Tricks module. Different types of optimization strategies are included in this module to speed up the training procedure and prevent the overfitting. Generally, preprocessing, data augmentation, dropout [55], shakeout [56], and batch-normalization [57] are included in this module.

Deep learning has achieved excellent results in optical image understanding, for example, ResNet-152 [49] shows 3.57% top-5 error on the ImageNet task [58] while SENet [59] shows only 2.30%, which are better than human error for this task at 5%. However, it is still a challenge to effectively apply deep learning for SAR image interpretation. The coherent imaging mechanism causes SAR images to present high fluctuations. This distinctive nature of SAR data is essentially different from optical images. Without considering this difference, the straightforward use of deep models for SAR image understanding cannot reach comparable performance levels to those obtained with optical images. Thus, how to exploit the power of deep learning as well as the mechanism of the SAR image is the starting point of this paper, and will be detailed in the following section.

3. Methodology

In this section, first, the quadratic primitive is presented; then, a comparison between the quadratic and convolutional primitives is provided; and finally, the proposed SLN is detailed.

3.1. Quadratic Primitive

The intrinsic randomness of SAR signals allows a statistical model to be an effective tool for SAR image analysis. A statistical model is generally defined by its parameters, and estimating these parameters is a significant topic in both probability theory and statistics. The method of moments (MoM) is one commonly used estimation approach. The MoM estimates for some typical statistical models are summarized in

Table 2. Parameter estimates for typical statistical models by MoM.

Type	Model	Parameters	MoM Estimation
Empirical Distributions	Log-Normal [15]	$\mu ,\sigma$	$\begin{array}{c}\mu =2ln\left(m_1\right)-\frac{ln\left(m_2\right)}{2}\hfill \\ \sigma =\sqrt{ln\left(m_2\right)-2ln\left(m_1\right)}\hfill \end{array}$
	Weibull [16]	$b,c$	$\begin{array}{c}\frac{\Gamma \left(1+{}^2/{}_c\right)}{\Gamma {\left(1+{}^1/{}_c\right)}^2}=\frac{m_2}{{m_1}^2}\hfill \\ b=\frac{m_1}{\Gamma \left(1+{}^1/{}_c\right)}\hfill \end{array}$
	Fisher [17]	$M,\mu$	$\begin{array}{c}M=\frac{L{m}_2}{L\left(m_2-m_1^2\right)-m_1^2}+1\hfill \\ \mu =\left(1-\frac{1}{M}\right)m_1\hfill \end{array}$
Priori Hypothesis	Rayleigh [12]	b	$b=\sqrt{2/\pi }{m}_1$
	Gamma [13]	$\mu ,L$	$\begin{array}{c}\mu =m_1\hfill \\ L=\frac{m_1^2}{m_2-m_1^2}\hfill \end{array}$
	$\mathcal{K}$ [14]	$\mu ,v$	$\begin{array}{c}\mu =m_1\hfill \\ v=\frac{(L+1)m_1^2}{L\left(m_2-m_1^2\right)-m_1^2}\hfill \end{array}$

$\Gamma (·)$ is the Gamma function, namely, $\Gamma \left(x\right)={\int }_0^{+\infty }{t}^{x-1}{e}^{-t}dt$, L denotes the number of looks.

, where $m_1$ and $m_2$ denote the first- and second-order moments, respectively, and $m_1^2$ is the square of $m_1$. These values are formulated as follows:

$$m_1=\frac{1}{n}{\sum }_i{x}_i,$$

(3)

$$m_2=\frac{1}{n}{\sum }_i{x}_i^2,$$

(4)

$$m_1^2={\left(\frac{1}{n}{\sum }_i{x}_i\right)}^2,$$

(5)

where n is the number of pixels, and $x_i$ denotes the i-th pixel value.

Table 2 indicates that $m_1,m_1^2$ and $m_2$ are the elementary components needed to fit the parameters of statistical models. The first-order moment, $m_1$, is the linear weighted sum of $x_i$ and can be realized by a convolution layer in which all kernel weights are set to $\frac{1}{n}$. In contrast, $m_1^2$ and $m_2$ cannot be fitted by a typical convolution layer because the quadratic terms $x_i^2$ and cross terms $x_i{x}_j$ are needed. Nevertheless, they can be expressed in terms of the quadratic form in linear algebra theory. Generally, a quadratic form can be formulated as

$$\begin{array}{cc}\hfill f\left(x_1,x_2,...,x_n\right)& =a_{11}{x}_1^2+a_{12}{x}_1{x}_2+...+a_{1n}{x}_1{x}_n+a_{21}{x}_2{x}_1+a_{22}{x}_2^2\hfill \\ & +...+a_{2n}{x}_2{x}_n+...+a_{n1}{x}_n{x}_1+a_{n2}{x}_n{x}_2+...+a_{nn}{x}_n^2\hfill \\ & =\sum _{i=1}^n{a}_{ii}{x}_i^2+2\sum _{j=1}^{n-1}\sum _{k=j+1}^n{a}_{jk}{x}_j{x}_k,\hfill \end{array}$$

(6)

where $f(x_1,x_2,\dots ,x_n)$ is a quadratic polynomial, namely, the quadratic form; $a_{ii}$ denotes the coefficients of the quadratic terms $x_i^2$, while $a_{jk}$ represents the coefficients of the cross terms $x_j{x}_k$. The quadratic form can be used to fit $m_2$ and $m_1^2$. Consider $m_2$, for example, where only the quadratic terms $x_i^2$ are retained, and the associated coefficients are set to $\frac{1}{n}$, which is a special case of the quadratic form.

The quadratic form in Equation (6) can be reformulated as the following matrix expression:

$$\begin{array}{cc}\hfill f\left(x_1,x_2,\cdots ,x_n\right)& =\left[x_1,x_2,\cdots ,x_n\right]\left[\begin{array}{c}{a}_{11}{x}_1+a_{12}{x}_2+...+a_{1n}{x}_n\\ a_{21}{x}_1+a_{22}{x}_2+...+a_{2n}{x}_n\\ \cdots \\ a_{n1}{x}_1+a_{n2}{x}_2+\cdots +a_{nn}{x}_n\end{array}\right]\hfill \\ & =\left[x_1,x_2,\cdots ,x_n\right]\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1n}\\ a_{21}& a_{22}& \cdots & a_{2n}\\ ⋮& ⋮& ⋮& ⋮\\ a_{n1}& a_{n2}& \cdots & a_{nn}\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_n\end{array}\right]\hfill \\ & ={\mathit{x}}^T\mathit{Wx},\hfill \end{array}$$

(7)

where $\mathit{W}$ is a weight matrix containing the coefficients for the quadratic and cross terms, $\mathit{x}={\left(x_1,x_2,\dots ,x_n\right)}^T$, and ${(·)}^T$ denotes the transpose operation. Using this strategy, the parameter estimation can be concretely represented in terms of the matrix expression. For example, the variance estimate ${\widehat{\sigma }}^2$ can be formulated as follows:

$$\begin{array}{cc}\hfill {\widehat{\sigma }}^2& =\frac{1}{n}\sum _{i=1}^n{\left(x_i-\frac{1}{n}\sum _{j=1}^n{x}_j\right)}^2\hfill \\ & =\left(\frac{1}{n}-\frac{1}{n^2}\right)\sum _{i=1}^n{x}_i^2-\frac{2}{n^2}\sum _{j=1}^{n-1}\sum _{k=j+1}^n{x}_j{x}_k.\hfill \end{array}$$

(8)

As shown in Equation (8), ${\widehat{\sigma }}^2$ can be estimated by the weighted combination of the quadratic and cross terms, and the associated matrix expression for ${\widehat{\sigma }}^2$ is

$$\begin{array}{cc}\hfill {\widehat{\sigma }}^2& =\left[x_1,x_2,\cdots ,x_n\right]\left[\begin{array}{cccc}\frac{1}{n}-\frac{1}{n^2}& -\frac{1}{n^2}& \cdots & -\frac{1}{n^2}\\ -\frac{1}{n^2}& \frac{1}{n}-\frac{1}{n^2}& \cdots & -\frac{1}{n^2}\\ ⋮& ⋮& \ddots & ⋮\\ -\frac{1}{n^2}& -\frac{1}{n^2}& \cdots & \frac{1}{n}-\frac{1}{n^2}\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_n\end{array}\right].\hfill \end{array}$$

(9)

To avoid discarding the first terms $x_i$, they are combined with the quadratic form by an additive neuron and thereby form the quadratic primitive, which is defined as

$$\begin{array}{cc}\hfill f\left(x_1,x_2,\cdots ,x_n\right)& =a_1{x}_1+a_2{x}_2+\cdots +a_n{x}_n\hfill \\ & +\sum _{i=1}^n{a}_{ii}{x}_i^2+2\sum _{j=1}^{n-1}\sum _{k=j+1}^n{a}_{jk}{x}_j{x}_k\hfill \\ & =\sum _{i=1}^n{a}_i{x}_i+\sum _{j=1}^n\sum _{k=1}^n{a}_{jk}{x}_j{x}_k\hfill \\ & ={\mathit{w}}_1^T\mathit{x}+{\mathit{w}}_2^{T}Vec\left(\mathit{x}{\mathit{x}}^T\right),\hfill \end{array}$$

(10)

where ${\mathit{w}}_1={\left(a_1,a_2,\cdots ,a_n\right)}^T$, ${\mathit{w}}_2={\left(a_{11},a_{12},\cdots ,a_{nn}\right)}^T$, $\mathit{x}={\left(x_1,x_2,\cdots ,x_n\right)}^T$, and $Vec(·)$ denote the vectorization operation. In Equation (10), the quadratic primitive is an extension of the convolutional primitive. In the quadratic primitive, the quadratic and cross terms and the first terms are exploited for parameter fitting and are expected to be able to extract the effective features for describing SAR image. Inspired by the convolution primitive, the local connection and weight-sharing strategies are also adopted in the quadratic primitive.

3.2. Comparison between Quadratic and Convolutional Primitives

Inspired by Colah [60], the two curves, shown in Figure 2a, which cannot be linearly separated, are used as an example to compare the behavior of quadratic and convolutional primitives. In Figure 2b, the curves are rotated, scaled, and shifted by the convolutional primitive. However, the curves in Figure 2b are linearly transformed and still cannot be separated by a straight line. For the quadratic primitive, as shown in Figure 2c, the curves undergo nonlinear as well as linear transformations. Apparently, the relative spatial position of the curves in Figure 2c is changed, and the transformed curves can be linearly separated.

3.3. Statistics Learning Network

Figure 3 illustrates the framework of the statistics learning network (SLN) based on the quadratic primitive. An input is first fed into the quadratic module for elementary statistical parameter extraction. Then, further fitting is implemented in the CNN module, where the structural information is considered. Finally, the output is generated by the fully connected and SoftMax layers.

3.3.1. Forward Pass

As shown in Figure 3, the first step of the SLN is learning the elementary statistics, which consists of two branches, i.e., the quadratic form and the convolutional operation. (i) The quadratic form is the weighted combination of the quadratic and cross terms; the associated output is $z_2$ in Figure 3. (ii) The convolutional operation is the weighted sum of the first terms, and the output is $z_1$. Finally, $z_1$ and $z_2$ are summed to form z in Figure 3.

The second step of the SLN is implemented by the CNN module. The output of the quadratic module is denoted as $\mathit{z}$ which is input into the CNN module for further feature learning. The CNN module is composed of a recursive application of convolution, ReLU activation, and pooling layers. The convolutional layer preserves the spatial information in the input; the ReLU performs nonlinear feature mapping, as shown in Figure 3, and avoids problems with vanishing gradient and exploding gradient and simplifies the calculation process; and the pooling layer reduces the dimensionality of features with translation invariance. Formally, for a CNN with L layers, the forward pass can be expressed as follows:

$${\mathit{y}}_l=f\left(\mathit{z},{\left\{{\mathit{W}}_l,{\mathit{b}}_l\right\}}_{l=1}^L\right),$$

(11)

where ${\mathit{y}}_l$ represents the output of the l-th layer, $f(·)$ is an activation function, and ${\mathit{W}}_l$ and ${\mathit{b}}_l$ denote the weight and bias of the l-th layer, respectively.

3.3.2. Optimization

In the context of a classification task, the learning procedure of the SLN can be formulated as an optimization problem. Concretely, given a training set $\mathcal{D}=\{({\mathit{x}}_1,y_1),({\mathit{x}}_2,y_2),\cdots ,({\mathit{x}}_N,y_N)\}$, this optimization problem is mathematically expressed as minimizing the following function:

$$\underset{{\left\{{\mathit{W}}_l,{\mathit{b}}_l\right\}}_{l=1}^L}{min}\frac{1}{N}\sum _{k=1}^N\mathcal{L}\left(y_k,{\widehat{y}}_k\left({\mathit{x}}_k,{\left\{{\mathit{W}}_l,{\mathit{b}}_l\right\}}_{l=1}^L\right)\right),$$

(12)

where $y_k$ is the true label of the training sample ${\mathit{x}}_k$, ${\widehat{y}}_k$ denotes the predicted label, and ${\mathit{W}}_l$ and ${\mathit{b}}_l$ are the parameters to be learned in l-th layer of the SLN. $\mathcal{L}\left(·\right)$ represents the cross-entropy [54] loss function, which is given by

$$\mathcal{L}=\langle {\mathit{y}}_k,-ln{\mathit{p}}_k\rangle ,$$

(13)

where $\langle ·\rangle$ denotes the inner product, ${\mathit{y}}_k$ is the label vector of $y_k$ generated by the one-hot encoding, and ${\mathit{p}}_k$ represents the predicted output of the SoftMax layer. In this paper, the gradients of the loss function $\mathcal{L}$ with respect to the parameter to be updated are estimated by every mini-batch of training samples. Thus, the parameter ${\theta }^l$ in the l-th layer is updated using the following strategy:

$${\theta }_{m+1}^l={\theta }_m^l-\eta ·{\nabla }_{{\theta }^l}\left(\frac{1}{K}\sum _{k=1}^K\langle {\mathit{y}}_k,-ln{\mathit{p}}_k\rangle \right),$$

(14)

where ${\nabla }_{{\theta }^l}(·)$ represents the gradient of $\mathcal{L}$ with respect to parameter ${\theta }^l$, K is the number of training samples in a mini-batch, and $\eta$ is the learning rate.

For the quadratic primitive, the gradient of the loss function $\mathcal{L}$ with respect to the associated parameter is derived as follows. As illustrated in Figure 4, the output of the quadratic primitive is $z=z_1+z_2$, where $z_1={\mathit{w}}_1^T\mathit{x}$, and $z_2={\mathit{w}}_2^{T}Vec\left(\mathit{x}{\mathit{x}}^T\right)$; ${\mathit{w}}_1$ and ${\mathit{w}}_2$ represent the weight vectors that need to be updated.

Let $\mathcal{L}$ denote the loss function, the gradient of $\mathcal{L}$ with respect to ${\mathit{w}}_1$ is

$$\frac{\partial \mathcal{L}}{\partial {\mathit{w}}_1}=\frac{\partial \mathcal{L}}{\partial z}\frac{\partial z}{\partial z_1}\frac{\partial z_1}{\partial {\mathit{w}}_1},$$

(15)

where $z=z_1+z_2$, and $z_1={\mathit{w}}_1^T\mathit{x}$; $\frac{\partial z}{\partial z_1}=1$, $\frac{\partial z_1}{\partial {\mathit{w}}_1}=\mathit{x}$. Therefore, $\frac{\partial \mathcal{L}}{\partial {\mathit{w}}_1}$ can be given by

$$\frac{\partial \mathcal{L}}{\partial {\mathit{w}}_1}=\frac{\partial \mathcal{L}}{\partial z}\mathit{x}.$$

(16)

Please note that $\frac{\partial \mathcal{L}}{\partial z}$ in Equation (16) can be obtained by the back-propagation algorithm [61]. Using the same strategy described above, the gradient of $\mathcal{L}$ with respect to ${\mathit{w}}_2$ is

$$\frac{\partial \mathcal{L}}{\partial {\mathit{w}}_2}=\frac{\partial \mathcal{L}}{\partial z}Vec\left(\mathit{x}{\mathit{x}}^T\right).$$

(17)

For the CNN module, fully connected and SoftMax modules of the SLN and the gradients of $\mathcal{L}$ can be obtained straightforwardly by the back-propagation algorithm [61] with respect to the parameters. Then, the gradient descent algorithm, namely, Equation (14), can be used for optimization.

4. Experiments

In this section, experiments conducted on three data sets are presented, including both space- and airborne data. The performance of the SLN is quantitatively and qualitatively compared with the standard CNN and texture-based approaches, followed by a discussion and an analysis.

4.1. Datasets

The SLN is evaluated on three datasets, including both space- and airborne SAR data as described below.

Guangdong data: The first image, as shown in Figure 5, is multi-look and single polarized (VV channel) data acquired by TerraSAR-X in Guangdong Province, China. The image size is 7518 × 4656 pixels with 1.25 m × 1.25 m of spatial resolution. The dataset includes five classes: i.e., $Vegetation$, $High-densityResidentialAreas$, $Low-densityResidentialAreas$, $River$, and $Pool$.

Orchard data: The second is a PolSAR image, as shown in Figure 6. These data were provided by the China Electronics Technology Group Corporation (CETC) 38 Institute [62] and were acquired by an X-band airborne sensor in Hainan Province, China. This image is in the format of a scattering matrix, including the HH, HV, VH, and VV channels; the SPAN image is used for experiments. The spatial resolution is approximately 0.5 m × 0.5 m, and the image size is 2200 × 2400 pixels. Seven classes are considered for the experiment: $Mango1$, $Mango2$, $Mango3$, $BetelNut$, $Longan$, $Forest$, and $Building$.

Rice data: The image in Figure 7 also consists of PolSAR data provided by the CETC 38 Institute. It has a pixel resolution of 2048 × 2048 pixels and a spatial resolution of approximately 0.5 m × 0.5 m. This dataset is composed of 7 categories: $Rice1$ to $Rice6$ and $Grass$, where $Rice1$ to $Rice6$ represent rice at different growth stages. The SPAN image is used for the experiments.

For each dataset, the ground truth images are generated by manual annotation according to the associated optical image which can be found in the Google Earth by the information provided in the metadata files. 64 × 64-pixel image patches were randomly collected according to the associated ground truth; the number of collected image patches for each dataset is given in

Table 3. The number of collected image patches and data information in experimental datasets.

Dataset	Date	Band	#Class	#Samples per Class	#Image Patch
Guangdong	2008	X-band	5	1000	5 × 1000
Orchard	2010	X-band	7	200	7 × 200
Rice	2010	X-band	7	200	7 × 200

. The collected image patches were then randomly divided into training, validation, and test sets, accounting for $60\%$, $15\%$, and $25\%$ of the total image patches, respectively. The effectiveness of the SLN is evaluated on the test set.

4.2. Experimental Setup

To assess the effectiveness of the proposed SLN, it is compared with the standard CNN and traditional texture-based methods, including GLCM, Gabor, and LBP. The architecture of the SLN is presented in

Table 4. SLN experimental architecture.

Layer	Parameter	#Channel
Input	64 × 64	1
Quadratic Module	4 × 4, 2	4
conv1	3 × 3, 1
relu1	-	16
pooling1	2 × 2, 2
conv2	3 × 3, 1
relu2	-	64
pooling2	2 × 2, 2
conv3	3 × 3, 1
relu3	-	128
pooling3	2 × 2, 2
ip1	3 × 3, 1	256
relu3	-
ip2	1 × 1, 1	number of class

. Here, take the parameter of “conv1”, for example, where “3 × 3, 1” denotes that the kernel size and stride are set to $3\times 3$ and 1, respectively. The setup for the other compared methods is described below.

CNN: The structure of the CNN is similar to the SLN except for the second layer. The second layer in the CNN is a convolution layer, while it is a quadratic module in the SLN. It should be noted that the parameters associated with each layer in the CNN are identical to the ones in the SLN.

GLCM: The number of gray levels is set to 4, and the offsets for the row and column are set to 0 and 2, respectively. Four types of features are considered for GLCM, i.e., contrast, correlation, energy, and inverse different moment, resulting in a 4-D GLCM feature vector.

Gabor: Gabor filters are implemented on three linear scales (2, 3, 4) and eight orientations $\left(0,\frac{\pi }{4},\frac{\pi }{2},\frac{3\pi }{4},\pi ,\frac{5\pi }{4},\frac{3\pi }{2},\frac{7\pi }{4}\right)$. The ratio of the mean and variance of each sub-band generates a Gabor feature vector with a dimension of 24.

LBP: The input image is first uniformly divided into $4\times 4$ subpatches; then, the uniform LBP feature is extracted from these subpatches. A $16\times 59$-dimensional LBP feature vector is obtained for image representation.

For the texture-based methods, an SVM [27] is used for classification. The experiments were performed on a desktop PC with dual Intel Xeon E3-1200 v3 processors, a single Nvidia GTX Titan-X GPU, and 32-GB RAM, running Ubuntu 14.04 LTS with the CUDA 8 release.

4.3. Experiment Results

4.3.1. Classification Accuracy

Table 5. Classification accuracy (%) on the Guangdong dataset.

	GLCM	Gabor	LBP	CNN	SLN
Vegetation	80.00	66.80	89.60	89.20 ± 0.11	90.00 ± 0.10
Pool	89.20	89.20	78.00	91.20 ± 0.03	92.80 ± 0.03
River	88.00	74.00	93.20	93.60 ± 0.01	94.00 ± 0.01
LD Area	65.20	70.80	67.20	74.40 ± 0.19	72.40 ± 0.12
HD Area	82.00	78.80	76.40	70.40 ± 0.15	79.60 ± 0.20
AA	80.88	75.92	80.88	83.76	85.76
Kappa	0.76	0.70	0.76	0.79	0.82

,

Table 6. Classification accuracy (%) on the Orchard dataset.

	GLCM	Gabor	LBP	CNN	SLN
Mango1	86.00	94.00	82.00	98.00 ± 0.20	98.00 ± 0.16
Mango2	66.00	60.00	50.00	72.40 ± 0.14	72.80 ± 0.12
Mango3	86.00	78.00	68.00	86.86 ± 0.17	84.00 ± 0.25
Betel Nut	88.00	82.00	80.00	97.20 ± 0.25	97.60 ± 0.07
Longan	84.00	80.00	76.00	92.00 ± 0.05	94.00 ± 0.04
Forest	76.00	82.00	72.00	78.40 ± 0.20	80.40 ± 0.14
Building	74.00	78.00	80.00	82.40 ± 0.09	91.20 ± 0.15
AA	79.99	79.14	72.57	86.74	88.28
Kappa	0.76	0.75	0.67	0.84	0.86

and

Table 7. Classification accuracy (%) on the Rice dataset.

	GLCM	Gabor	LBP	CNN	SLN
Rice1	90.00	47.50	75.00	90.00 ± 0.17	95.00 ± 0.10
Rice2	82.50	62.50	85.00	100.00 ± 0.02	100.00 ± 0.02
Rice3	70.00	75.00	75.00	80.00 ± 0.26	87.50 ± 0.07
Rice4	27.50	45.00	55.00	67.50 ± 0.25	65.00 ± 0.16
Rice5	67.50	72.50	55.00	87.50 ± 0.19	82.50 ± 0.15
Rice6	82.50	80.00	90.00	95.00 ± 0.06	95.00 ± 0.06
Grass	100.00	82.50	100.00	97.50 ± 0.01	100.00 ± 0.01
AA	74.28	66.42	76.42	88.21	89.28
Kappa	0.70	0.60	0.72	0.86	0.87

show the classification accuracies on each class for the Guangdong, Orchard, and Rice datasets, respectively. The uncertainty is included in the study of CNN and SLN.

On the Guangdong dataset, the SLN and CNN outperform the texture-based methods as shown in Table 5. Compared with the CNN, the SLN’s accuracy on the Pool and HD Area classes is improved by 1.60% and 9.20%, respectively. The SLN also slightly improves the average accuracy (AA) and Kappa coefficient. Please note that the CNN performs better at recognizing the LD area class, and it shows a performance comparable to the SLN for River classification.

Table 6 displays the classification accuracies on the Orchard dataset. The results indicate that the SLN performs better than do the CNN and texture-based methods. The AA and Kappa coefficient achieved by the SLN reach 88.28% and 0.86, respectively. Concretely, the SLN shows potential for effectively classifying Mango2, Betel Nut, Longan and Building—especially Longan and Building—for which the classification accuracies are improved by approximately 2.00% and 8.80%, respectively, compared with the CNN. Interestingly, the Gabor performs best at recognizing the Forest class.

The results for the Rice dataset are reported in Table 7. As expected, the SLN achieves the best performance, with an AA and Kappa coefficient of 89.28% and 0.87, respectively. The CNN shows comparable performance to the SLN in classifying Rice2 and Rice6; moreover, the CNN performs best for the Rice4 and Rice5 classes.

4.3.2. Confusion Matrix

To evaluate the discriminative capability of the SLN, the confusion matrix is investigated. Figure 8, Figure 9 and Figure 10 display the results on the Guangdong, Orchard, and Rice datasets. Here, because GLCM and LBP perform better than Gabor, they are selected as candidates for comparison.

Figure 8 shows the results of the Guangdong dataset. (i) The GLCM and LBP have apparent confusions between the HD and LD areas as shown in Figure 8a,b. (ii) The CNN and SLN significantly reduce misclassifications of Vegetation, Pool and River as shown in Figure 8c,d. More specifically, the SLN shows better discriminative capability than the CNN in discriminating between LD and HD areas. In fact, the interclass variations between the HD and LD areas are non-salient due to the similar and complex scattering; therefore, it is difficult to eliminate these confusions.

The confusion matrices for the Orchard dataset are displayed in Figure 9. Three classes: Mango1, Forest and Building, are selected for investigation. (i) For the Mango1 class, Figure 9a,b indicate that GLCM and LBP misclassify Mango1 as Mango2 at a high probability (16.00% and 20.00%, respectively). The CNN and SLN reduce this misclassification rate as shown in Figure 9c,d where the probabilities are 1.60% and 2.00%, respectively. (ii) For the Forest class, the CNN and SLN perform better. The confusions between Forest and Betel Nut by the SLN is 2.40% which is lower than CNN. (iii) For the Building class, the SLN performs best as shown in Figure 9d, and the Building class is misclassified only as Longan or Forest at a low probability (0.80% and 8.00%, respectively).

Figure 10 shows the results of the Rice dataset. Here, Rice1, Rice3 and Rice4 are selected for comparison. (i) For the Rice1 class, LBP’s performance is poor compared with the GLCM and CNN, while the SLN performs better than CNN. As shown in Figure 10d, Rice1 is misclassified only as Rice3 with a probability of 5.00%. (ii) For the Rice3 class, the SLN achieves a better result than the other methods; confusions occur only between Rice1 and Rice4 at probabilities of 5.00% and 7.50%, respectively. (iii) For the Rice4 class which is difficult to precisely recognize, this confusion is reduced by the CNN and SLN and the CNN performs slightly better. However, the SLN’s misclassification accuracy for Rice3 is lower compared to the CNN (17.00% vs. 20.00%). In summary, these results suggest that the SLN is more competitive.

4.3.3. Classification Map

To perform a quantitative comparison, the classification maps shown in Figure 11 and Figure 12 were evaluated. These classification maps were generated with a post-processing method based on conditional random fields [63] for a better visualization.

Figure 11 displays the classification maps of the Guangdong dataset, and Figure 11a is the ground truth of this dataset. The three regions shown in Figure 11a were selected for convenience of comparison. Visually, the results by the SLN, CNN, and LBP in Figure 8b–d, respectively, are more consistent with the ground truth Figure 8a. (i) In the selected region 1, GLCM and Gabor show poor performance than the SLN, CNN, and LBP. (ii) In the highlighted regions 2 and 3, the SLN, CNN, and LBP achieve similar results but the GLCM and Gabor results are unsatisfactory. These results indicate that while the SLN works well, it does not achieve a remarkable improvement on this dataset.

The results of the Orchard dataset are shown in Figure 12. (i) In the highlighted region 1, the SLN and CNN provide more satisfactory performance, since they correctly recognize most parts of the Longans. (ii) In the region 2, the SLN and CNN results, as expected, are more consistent with the ground truth Figure 12a, while the texture-based methods misclassify a large portion of Mango3 as Mango2 and Betel Nut. (iii) For the region 3, the SLN and CNN provide almost identical results, and both are better than the GLCM, Gabor, and LBP. In summary, these results suggest that the SLN works best on this dataset, and further reduced the confusions between Mango3 and Betel Nut as compared to the CNN.

4.3.4. Average Accuracy vs. Training Samples

To further evaluate the effectiveness of the SLN, the average accuracies achieved with different training samples percentages are compared. Figure 13 shows the results for the Guangdong and Orchard datasets. (i) In Figure 13a,b, the traditional texture-based methods obtain better performance when the training data comprise less than 20%. This result occurs because of the data-driven natures of the SLN and CNN such that they are unable to learn a large number of parameters effectively with insufficient training data, thus having a dramatic impact on the performance of the SLN and CNN. (ii) As the amount of training data increases, the performance of all the compared methods improve. Moreover, when the percentage of training data exceeds 30%, the SLN and CNN outperform the texture-based methods. (iii) Compared with the CNN, the SLN achieves better performance when the amount of training data lies in the interval $\left(20\%,40\%\right)$. In contrast, when the training data are sufficient (e.g., above 40% in Figure 13a), the SLN and CNN obtain almost the same level of performance.

4.4. Analysis and Discussion

The experimental results indicate that the proposed SLN performs better than do the compared methods. An analysis of this improvement is that the SLN exploits more of the information contained in the SAR images. The randomness of the SAR signal calls for many DoFs when creating the SAR image description. In the SLN, the first, quadratic and cross terms are combined with the quadratic primitive to fit the elementary statistical features. The quadratic primitive extends the feature space and provides more flexibility in feature representation. This characteristic of the quadratic primitive is expected to provide potential for feature extraction in SAR images.

As possible future research directions some discussion is provided as follows:

• Given a random vector $\mathit{x}={[x_1,x_2,\cdots ,x_n]}^T$, its linear transformation $\mathit{y}=\mathit{W}\mathit{x}+\mathit{b}$ is still a random variable, because $\mathit{y}$ can be regarded as a statistic that is a function of some observable random variables. This fact implies that the strategy used for elementary statistical feature extraction can be extended to deeper layers of the SLN. Using such an extension is expected to be more effective in SAR image interpretation. However, this approach also increases the number of weights that must be learned. Therefore, a tradeoff should be considered, and the effectiveness of this extension is in need of investigation.
• The SLN is a patchwise approach; thus, when the SLN is used to process large images, it suffers from the same disadvantage as the patch-based methods [64,65]. For semantic labeling or segmentation, post-processing, e.g., smoothness constraints, or other end-to-end methods such as a fully convolutional network [66] should considered in further research.

5. Conclusions

The distinctive nature of SAR signals, such as high fluctuations, poses a large challenge to extending the potential of CNNs to understanding SAR images. In this paper, a statistics learning network(SLN) based on the quadratic form is presented for SAR image classification. The innovation of this approach lies in its capability to fit the statistical features for SAR image representation. By using the quadratic form in linear algebra theory, a quadratic primitive that features learnable capability is proposed for learning the elementary statistical features. The quadratic primitive extends the feature space by exploiting more of the information contained in the input, including the quadratic and cross terms. This primitive is an extension of the convolutional primitive, and it is expected to be an alternative to the convolutional primitive. The experiments conducted on three SAR images indicate that the SLN achieves better performance than do the standard CNN and texture-based methods. The scheme proposed in this paper has room for further improvement, and discussions on a general framework and semantic labeling based on this scheme are presented.