Complex-Valued 2D-3D Hybrid Convolutional Neural Network with Attention Mechanism for PolSAR Image Classification

Abstract

The recently introduced complex-valued convolutional neural network (CV-CNN) has shown considerable advancements for polarimetric synthetic aperture radar (PolSAR) image classification by effectively incorporating both magnitude and phase information. However, a solitary 2D or 3D CNN encounters challenges such as insufficiently extracting scattering channel dimension features or excessive computational parameters. Moreover, these networks’ default is that all information is equally important, consuming vast resources for processing useless information. To address these issues, this study presents a new hybrid CV-CNN with the attention mechanism (CV-2D/3D-CNN-AM) to classify PolSAR ground objects, possessing both excellent computational efficiency and feature extraction capability. In the proposed framework, multi-level discriminative features are extracted from preprocessed data through hybrid networks in the complex domain, along with a special attention block to filter the feature importance from both spatial and channel dimensions. Experimental results performed on three PolSAR datasets demonstrate our present approach’s superiority over other existing ones. Furthermore, ablation experiments confirm the validity of each module, highlighting our model’s robustness and effectiveness.

1. Introduction

The classification of objects using polarimetric synthetic aperture radar (PolSAR), which involves the transmission of electromagnetic waves and reception of scattering echoes under all-weather and all-time conditions, has proven to be a significant asset in the remote sensing field. Contrasting with conventional synthetic aperture radar (SAR), PolSAR captures four distinct polarimetric states, leveraging electromagnetic wave properties for an enhanced understanding of the target. Image classification is a committed step in interpreting PolSAR images, which determines the true category for each pixel by utilizing the available information. It is widely used in urban planning [1], agricultural production [2], geological survey [3], target detection [4], ocean exploration [5], and so on.

Traditional PolSAR image classification is primarily enhanced by scattering feature extraction and classifier structure improvement. There are two categories of feature extraction methods: those reliant on simple combinations and transformations, and those reliant on polarimetric target decomposition. The former includes the polarimetric scattering matrix and its corresponding vector expansion [6], polarimetric coherence matrix, and covariance matrix [7]. The latter decomposes these matrices into distinct components, each conveying specific scattering or geometric structure information of targets. Commonly used decomposition methods include Cloude decomposition [8], Krogager decomposition [9], Cameron decomposition [10], and Freeman–Durden decomposition [11]. The above methods provide more parameters of targets, but a lot of manpower and more experience are required for feature extraction and selection in a specific task. Regarding the design of classifiers, the main approach evolved from machine learning, including support vector machine (SVM) [12], random forest [13] and decision tree [14]. These methods excel in independent learning and tackling complex non-linear problems. However, the major limitation is the dependence on previous experience for artificial feature selection.

Deep learning (DL) techniques have recently shown impressive potential in computer vision by autonomously acquiring high-level abstract features through incremental learning [15]. Compared to machine learning, they eliminate the reliance on manual feature extraction. Typical DL methods mainly include deep belief network (DBN) [16], sparse auto-encoder (SAE) [17], and convolutional neural network (CNN) [18], where CNN is often used for image classification due to the characteristics of parameter sharing and sparse connections [19]. Multiple CNN models have been developed for PolSAR image classification with excellent results, such as 2D-CNN [20], 3D-CNN [21], and fully convolutional network (FCN) [22]. Generally, the network’s feature extraction capability improves as the computational dimension of the convolutional kernel expands. Specifically, networks based on 3D convolution work better than 1D and 2D in most cases. However, an increase in convolutional dimension also leads to a corresponding rise in model complexity, which means that the classification task requires more time and equipment cost for training. This is consistently deemed undesirable in practical applications.

Notably, all the above approaches entail projecting the complex-valued PolSAR data onto the real domain, serving as input for the model. Nevertheless, the projection will lead to some significant information being omitted. It is necessary to design a complex-valued CNN (CV-CNN) instead of a real-valued CNN (RV-CNN) to fully utilize the abundant information contained in complex data. To address this challenge, Zhang et al. [23] pioneered a CV-CNN that expands all elements to the complex domain, effectively exploiting the phase information in the coherence matrix of PolSAR data and achieving higher accuracy than the RV-CNN. Subsequently, methods such as CV-3D-CNN [24] and CV-FCN [25] have also been proposed in succession, achieving superior results to the real-domain methods. Nevertheless, the augmented quantity of parameters required for expanding the model from the real to the complex domain calls for more labeled samples for training. Unfortunately, this poses a challenge for PolSAR data, which are arduous to interpret manually, making the situation less favorable.

Attention mechanisms are inspired by the human ability to suppress unimportant stimuli and allocate neural computational resources to critical parts when processing complex visual information [26]. In CNN, the importance of each feature map is different. The central concept of attention mechanisms is to give larger weight to the important features and devote more computational resources of the network to more important tasks. Common attention mechanisms include SE [27], CBAM [28], ECA [29], CA [30], etc. Currently, several PolSAR classification studies have noticed the effects of attention mechanisms. They applied attention mechanisms for preprocessing input data and network architecture improvements [31,32,33,34]. Although these approaches acknowledge the efficacy of the attention mechanism in boosting model performance, most of them simply employ it to enhance model inputs or integrate it with sequence networks. How to apply the attention mechanism to the feature maps inside a CNN, especially CV-CNN, is a major challenge.

To tackle the aforementioned issues, this article explores a new complex-valued 2D-3D hybrid convolutional neural network with attention mechanism (CV-2D/3D-CNN-AM). The model is capable of directly processing phase information in a complex form, using a combination of 2D and 3D convolution to extract effective discriminative features and reduce model complexity, along with a specific attention block to improve classification performance.

The major contributions of our article can be summarized as follows:

• An innovative hybrid CV-CNN for PolSAR image classification is presented, which effectively incorporates the advantages of both 2D and 3D convolution. Specifically, the model first extracts spatial features using complex-valued 2D convolutions with fewer parameters and then integrates spatial and channel dimensional features using 3D convolutions. Compared with a single-dimensional CNN, this hybrid network effectively reduces the number of parameters to avoid overfitting while maintaining better performance.
• We design an attention block for CV-CNN to increase the model’s efficiency and accuracy. The proposed attention block utilizes both the real and imaginary parts of features to calculate weights that represent feature importance. The classification results can effectively be improved through the utilization of recalibrated features supplemented with appropriate weights. It is worth noting that the inputs and outputs of the block are in complex form.
• Experiments are performed on three authentic PolSAR datasets to confirm the advancement of our proposed method. Under the same conditions, CV-2D/3D-CNN-AM provides a competitive classification outcome, which effectively extracts discriminative features and focuses on more significant information.

The remainder of this article is as follows: Section 2 provides an overview of the related work. Section 3 delves into the detailed description of the proposed method. Section 4 analyzes the experimental results conducted on various datasets, and lastly, the conclusions are provided in Section 5.

2. Related Work

2.1. PolSAR Data Processing

PolSAR captures the polarimetric state of targets being examined through multiple polarization combinations. The scattering characteristics are represented by the complex two-dimensional scattering matrix S as follows:

$$S=\left[\begin{array}{cc}{S}_{HH}& S_{HV}\\ S_{VH}& S_{VV}\end{array}\right]$$

(1)

where H and V denote horizontal and vertical modes, respectively. $S_{pq}$ denotes that the transmitting wave is p and the receiving wave is q, $p,q\in \left\{H,V\right\}$. When the reciprocity condition is fulfilled, the equation $S_{HV}=S_{VH}$ is considered for the single station case.

The scattering matrix S can be vectored as K by the Pauli decomposition [7] as:

$$K={\displaystyle \frac{1}{\sqrt{2}}}{[S_{HH}+S_{VV},S_{HH}-S_{VV},2S_{HV}]}^T$$

(2)

where the superscript T denotes the transpose operation. The coherence matrix $\mathbf{T}$ after multi-look is calculated as:

$$\mathbf{T}={\displaystyle \frac{1}{L}}\sum _{i=1}^L{K}_i{K}_i^H=\left[\begin{array}{ccc}{T}_{11}& T_{12}& T_{13}\\ T_{21}& T_{22}& T_{23}\\ T_{31}& T_{32}& T_{33}\end{array}\right]$$

(3)

where L is the count of looks and the superscript H is the conjugate transpose operation. According to Equation (3), the coherence matrix $\mathbf{T}$ is a Hermitian matrix with real diagonal elements and complex off-diagonal elements. Therefore, it is sufficient to enter only the upper triangular elements into the model.

For real-valued models, the eigenvector is expressed as a vector of nine elements as follows:

$$\begin{array}{cc}\hfill [T_{11},T_{22},T_{33},\Re \left(T_{12}\right),\Im \left(T_{12}\right),& \Re \left(T_{13}\right),\Im \left(T_{13}\right),\Re \left(T_{23}\right),\Im \left(T_{23}\right)]\hfill \end{array}$$

(4)

where $\Re (·)$ and $\Im (·)$ represent the real and imaginary parts of the complex numbers, respectively.

For complex-valued models, the real and imaginary parts of the elements do not need to be separated, and the eigenvector is represented as a vector of six elements as follows:

$$\left[\begin{array}{c}{T}_{11}+0·j,T_{22}+0·j,T_{33}+0·j,T_{12},T_{13},T_{23}\end{array}\right]$$

(5)

where j is the imaginary unit satisfying $j^2=-1$.

Polarimetric feature extraction methods aim to provide more scattering properties of the target, among which Cloude decomposition is widely used and effective [8]. In this decomposition, eigenvalues and eigenvectors are obtained from the coherence matrix $\mathbf{T}$ by spectral decomposition as:

$$\mathbf{T}=U\Lambda U^H=U\left[\begin{array}{ccc}{\lambda }_1& 0& 0\\ 0& {\lambda }_2& 0\\ 0& 0& {\lambda }_3\end{array}\right]U^H$$

(6)

where $\Lambda$ is a diagonal matrix consisting of three eigenvalues ${\lambda }_1$, ${\lambda }_2$, and ${\lambda }_3$ of $\mathbf{T}$. U is the matrix consisting of the column vectors of the corresponding eigenvalues. And the entropy H, anisotropy A, and scattering angle $\alpha$ are defined as follows, respectively:

$$H=\sum _{i=1}^3-P_i{log}_3{P}_i,P_i={\displaystyle \frac{{\lambda }_i}{{\sum }_{j=1}^3{\lambda }_j}}$$

(7)

$$A={\displaystyle \frac{{\lambda }_2-{\lambda }_3}{{\lambda }_2+{\lambda }_3}}$$

(8)

$$\alpha =\sum _{i=1}^3{P}_i{\alpha }_i$$

(9)

where ${\alpha }_i$ represents the scattering angle corresponding to ${\lambda }_i$. In this way, considering the above decomposition features, the real-valued input eigenvector expands as:

$$\begin{array}{c}\hfill [T_{11},T_{22},T_{33},\Re \left(T_{12}\right),\Im \left(T_{12}\right),\Re \left(T_{13}\right),\Im \left(T_{13}\right),\Re \left(T_{23}\right),\Im \left(T_{23}\right),H,A,\alpha ]\end{array}$$

(10)

The complex-valued input eigenvector expands as:

$$\begin{array}{cc}\hfill [T_{11}+0·j,T_{22}+0·j,T_{33}+0·j,T_{12},T_{13},& T_{23},H+0·j,A+0·j,\alpha +0·j]\hfill \end{array}$$

(11)

2.2. CV-CNN-Based PolSAR Classification Methods

The distinctive phase information of PolSAR is important in their applications, and the CV-CNN has been extensively researched for its capacity to fully utilize the phase information present in the complex values of PolSAR. The CV-CNN was first employed by Zhang et al. [23] for PolSAR tasks. Recently, CV-CNN-based approaches have been continuously updated, which are broadly classified into driving features-based improvement, model structure-based improvement, and updating strategies-based improvement according to their main contributions.

(1) Driving features-based improvement: Utilizing the proficient understanding of the targeted scattering mechanism interpretation and efficient polarimetric feature extraction in guiding the model inputs contributes to enhancing the classification performance of the convolutional neural network. Qin et al. [35] fused the six-dimensional artificial features and the upper half elements of the coherence matrix into a twelve-dimensional complex vector as input for CV-CNN, which achieves better and more stable classification performance. Barrachina et al. [36] concluded that the data representation of the correlation matrix has limitations in the pixel-level task, and changed to use Pauli representation as input.

However, blindly increasing the dimension of polarimetric features will result in feature redundancy. Redundant inputs not only increase extra computation but also make the model prone to local overfitting. Han et al. [37] employed random forest to select the manually extracted seventeen-dimensional feature vectors and filter the six-dimensional most important features as the optimal set. Yang et al. [38] designed a 1D-CNN feature selection method and combined it with the Kullback–Leibler distance to determine the final feature combination.

(2) Model structure-based improvement: The upper limit of model performance and learning capacity primarily depends on the model structure. Adugna et al. [39] designed a more stable PolSARNet model that adopts a real-valued kernel function to process complex data and produce complex-valued outputs. Tan et al. [24] proposed CV-3D-CNN that combines the advantages of 3D-CNN and CV-CNN, which makes a breakthrough in classification accuracy but also consumes more time for model training. Ren et al. [40] made improvements in the non-convolutional layer part of CV-CNN to achieve better performance. Tan et al. [41] designed a triplet CV-CNN for acquiring the complex-valued representation through optimizing interclass distance and reducing intraclass distance.

Due to the decrease in computational efficiency caused by conventional CNN inputting pixels within each patch range to predict the category of central pixels [42], several studies have focused on developing pixel-to-pixel methods. Cao et al. [25] designed a deep CV-FCN for precise pixel-level annotation, together with a novel loss function to optimize the network. Mullissa et al. [43] explored a multi-branch CV-FCN, which is excellent for dealing with scattering noise in PolSAR images. Liu et al. [44] proposed a polarization coding that effectively preserves structural information in the scattering matrix and combines it with CV-FCN to enhance its performance and achieve greater resilience.

(3) Updating strategies-based improvement: Regular CNN models usually require a supervised learning strategy to model parameter updating. It is challenging to obtain a substantial amount of labeled samples for training in this scenario, particularly for PolSAR images that are intricate to decode manually. Qin et al. [45] presented a weakly supervised CV-CNN, which trains the model by iteratively selecting the labeled samples simultaneously and uses it directly for the classification task.

In recent times, scholars have shown significant interest in semi-supervised learning owing to its capacity to effectively handle a vast amount of unlabeled samples alongside limited labeled samples [46]. This strategy empowers the model to attain favorable classification performance. Xie et al. [47] designed a recurrent CV-CNN, which continuously expands the number of training sample sets through recurrent training and still achieves high accuracy with fewer labels. Zhu et al. [48] used multiple classifiers to generate strong and weak datasets and then used the former as a label to reclassify the latter. Zeng et al. [49] integrated the semi-supervised learning framework with data features to reduce the dependence on label annotation.

2.3. Attention Mechanisms

For image classification, attention mechanisms are mainly classified into the following categories depending on the data area of action: channel attention, spatial attention, and channel and spatial attention [50]. Concretely, Figure 1 shows their operations on features.

Channel attention assigns weights to each channel of feature maps, thereby letting the network know what it needs to focus on. Hu et al. [27] pioneered the channel attention concept and designed the SE network, which improves the model’s expressive power by squeezing and exciting the input features. Li et al. [51] introduced a selective kernel to improve the recognition efficiency based on channel attention.

Spatial attention transforms the spatial information in the feature map to another space with appropriate transformations and retains the key information, enabling the neural network to learn where it should pay attention. Minh et al. [52] devised a spatial attention model that employs the inner state of the model to determine the location of attention, which is capable of producing control messages in a dynamic context. Jaderberg et al. [53] designed a spatial transformer that can be used to obtain regions of interest in features through spatial transformations such as translation, rotation, and scaling.

Channel and spatial attention adaptively chooses representative objects and positions, which is more holistic than single-dimensional attention. Wang et al. [54] designed a residual attention network using channel and spatial attention for the first time. The model generates 3D attention maps by stacking attention modules that produce attention-aware features. However, it is restricted by the high computational workload and limited perceptual domains. To address the above limitations, Refs. [28,55] improved the calculation speed by importing global average pooling and decoupling attention. Refs. [56,57] used self-attentive modules to coordinate the two dimensions of channel and space. Refs. [58,59] improved the validity of attentional outcomes by increasing the receptive field.

In the PolSAR image domain, Dong et al. [31] employed the attention mechanism in polarimetric feature selection to assure the efficiency of high-dimensional data and inputted the selected data into a CNN for classification. Hua et al. [32] proposed an attention-based multi-scale sequential model with a new hybrid loss function to improve classification performance. Yang et al. [33] designed a polarization orientation angle (POA) attention to enable the network to focus on a more physically meaningful range of critical POA to enhance performance. Zhang et al. [34] explored a texture-based attention module that considers both channel and positional importance to learn enhanced features.

3. Proposed Method

3.1. Framework of CV-2D/3D-CNN-AM

In this article, we design a CV-2D/3D-CNN-AM for PolSAR image classification tasks, and Figure 2 shows its detailed structure. Overall, the framework architecture developed in this paper mainly consists of a 2D convolutional block, a specially designed attention mechanism module, a 3D convolutional block, and fully connected layers. The first three parts mainly act as feature extractors, while the last one is used as a classifier.

In terms of process, rich spatial features are first extracted from the processed PolSAR data by a 2D convolutional block consisting of two complex-valued 2D convolutional layers. The extracted features are dimensionality reduced by a complex-valued pooling layer and then fed into the attention module for computation, highlighting the more important information in the features. The 3D convolutional block is then used for feature extraction and integration in both scattering and spatial dimensions. Finally, the extracted feature tensor is sent to the dense linear layer to obtain the classified result. The following section presents more details of the CV-2D/3D-CNN-AM framework.

3.2. Complex-Valued 2D-3D Hybrid CNN

The hybrid 2D and 3D CNN, which is adept at processing complex-valued inputs, is employed for PolSAR image classification tasks to obtain ample discriminative information and valuable multi-scale features. The designed hybrid CNN belongs to the feed-forward neural network with a deep structure, which partially extracts features through the feature extractor in a data-driven manner and then inputs them into the classifier, without any manual design in the whole process.

All elements of the feature extraction part including the parameters and operations in this network are extended from the real field to the complex field. The calculation implementation process for each layer is as follows.

Convolutional layer: Assume that the feature maps of the lth convolutional layer are $F_m^{\left(l\right)}\in {\mathbb{C}}^{W_l\times H_l\times M}$ and the subsequent $(l+1)$th layer is $F_n^{(l+1)}\in {\mathbb{C}}^{W_{l+1}\times H_{l+1}\times N}$, where $\mathbb{C}$ denotes the complex field and the superscript is its corresponding dimension. The set of filters between the lth and $(l+1)$th layers is defined as $w_{nm}^{l+1}\in {\mathbb{C}}^{S\times S\times M\times N}$ in the 2D case, which increases by one dimension in the 3D case defined as $w_{nm}^{l+1}\in {\mathbb{C}}^{S\times S\times S\times M\times N}$. The additional added bias is defined as $b_n^{(l+1)}\in {\mathbb{C}}^N$. The specific procedure of the convolution operation is as follows:

$$\begin{array}{c}\hfill F_n^{l+1}=f(\Re \left(V_n^{(l+1)}\right))+jf(\Im \left(V_n^{(l+1)}\right))=\max (0,\Re \left(V_n^{(l+1)}\right))+j\max (0,\Im \left(V_n^{(l+1)}\right))\end{array}$$

(12)

$$\begin{array}{cc}\hfill V_n^{(l+1)}& =\sum _{m=1}^M{w}_{nm}^{(l+1)}\ast F_m^{\left(l\right)}+b_n^{(l+1)}\hfill \\ \hfill & =\sum _{m=1}^M(\Re \left(w_{nm}^{(l+1)}\right)\ast \Re \left(F_m^{\left(l\right)}\right)-\Im \left(w_{nm}^{(l+1)}\right)\ast \Re \left(F_m^{\left(l\right)}\right))\hfill \\ \hfill & +j\sum _{m=1}^M(\Re \left(w_{nm}^{(l+1)}\right)\ast \Im \left(F_m^{\left(l\right)}\right)+\Im \left(w_{nm}^{(l+1)}\right)\ast \Re \left(F_m^{\left(l\right)}\right))+b_n^{(l+1)}\hfill \end{array}$$

(13)

where $\max (·)$ denotes obtaining the maximum value. The operator ∗ refers to the convolution operation. $f(·)$ stands for the Relu non-linear activation function, which offers the advantages of computational simplicity as well as sparse representation [60]. The discrepancy between 2D and 3D convolution is directly illustrated in Figure 3, where the latter handles channel dimension information better than the former but with more parameters. Figure 4 explains how complex-valued convolution is performed.

In this model, the 2D convolutional block consists of two convolutional layers. The convolution kernels in each layer are 9, the size of the kernel is 3 × 3, and the move step is 1. The padding pattern is set as “same”. The 3D convolutional block contains two 3D convolutional layers, the number of 3D convolution kernels in each layer is 32, the size of the kernel is 3 × 3 × 3, and the move step is set to 1. The padding pattern is furthermore designated as “same”.

Pooling layer: The pooling layer downsamples features in the spatial dimension by averaging or taking maximal pairs, which merges locally similar features and reduces the feature dimension. In addition, performing pooling operations makes the DL network’s receptive field larger and more robust to changes in the input feature positions.

This model adopts average pooling for data dimension reduction. The average pooling operation under complex form is defined as

$$\begin{array}{cc}\hfill F_n^{(l+1)}(x,y)=& {\mathrm{ave}}_{a,b=0,\dots ,d-1}(\Re \left(F_i^{\left(l\right)}(x·s+a,y·s+b)\right))\hfill \\ \hfill & +j{\mathrm{ave}}_{a,b=0,\dots ,d-1}(\Im \left(F_i^{\left(l\right)}(x·s+a,y·s+b)\right))\hfill \end{array}$$

(14)

where $F_n^{(l+1)}(x,y)$ is the feature map cell of $F_n^{(l+1)}$ at location (x, y). d and s represent the dimension and stride of the pooling kernel, respectively. $\mathrm{ave}(·)$ refers to averaging.

Since the pooling operation will lead to some information loss, the pooling kernel size and stride should not be taken to be large. A suitable value refers to a size of 2 × 2 and a step size of 2.

Fully Connected Layer: After completing the convolution and pooling computation, the learned features are typically mapped from the feature space to the label space through one or more layers that are fully connected. The output of the fully connected layers is represented as:

$$\begin{array}{cc}\hfill & F_n^{(l+1)}=f(\Re \left(V_n^{(l+1)}\right))+jf(\Im \left(V_n^{(l+1)}\right))\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & V_n^{(l+1)}=\sum _{k=1}^K{w}_{nk}^{(l+1)}·F_k^{\left(l\right)}+b_n^{(l+1)}\hfill \end{array}$$

(16)

where M refers to neuron amount in the lth layer.

However, the complex tensors of the fully connected layer fed into the softmax classifier most commonly used by CNN networks will no longer result in a probability between 0 and 1. In this paper, we borrow from the approach of flattening complex features’ real and imaginary parts [24] before inputting them into the fully connected layer to cooperate with softmax classifiers for category determination. Between the flatten layer and the final layer, the total neuron count in the fully connected layer is 128.

Output layer: The output layer of the network is at the end and consists of several neurons that correspond to the overall categories. As is usual with neural networks, the softmax function is utilized to determine categories, and the cross-entropy loss ($\mathcal{L}$) is commonly applied to optimize the model. Specifically, separate losses are computed for each class separately and summed to obtain the final value. Model training is achieved by adjusting the parameters to minimize the loss values.

Complex weight initialization and backpropagation: The initialization of the complex weights obeys the derivation of [61,62]. In the complex-valued domain, the weight W is denoted as:

$$W=\left|W\right|e^{i\theta }=\Re \left(W\right)+j\Im \left(W\right)$$

(17)

where $\left|W\right|$ and $\theta$ are separately the amplitude and phase. $\left|W\right|$ conforms to the Rayleigh distribution.

The variance is calculated as:

$$\begin{array}{cc}\hfill & Var\left(W\right)=\mathbb{E}\left[W{W}^{\ast }\right]-{\left(\mathbb{E}\left[W\right]\right)}^2={\mathbb{E}\left[\right|W|}^2]-{\left(\mathbb{E}\left[W\right]\right)}^2\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & Var\left(\right|W\left|\right)={\mathbb{E}\left[\right|W\left|\right|W|}^{\ast }{]-(\mathbb{E}\left[\right|W\left|\right])}^2={\mathbb{E}\left[\right|W|}^2{]-(\mathbb{E}\left[\right|W\left|\right])}^2\hfill \end{array}$$

(19)

where $\mathbb{E}[·]$ is the expectation. When W is symmetrically distributed around 0, $Var\left(W\right)$ is equal to $\mathbb{E}\left[\right|W{|}^2]$. Combine the above:

$$Var\left(W\right)=Var\left(\right|W\left|\right)+{\left(\mathbb{E}\right[\left|W\right|\left]\right)}^2$$

(20)

Equation (20) shows that the variance of W is associated with $\left|W\right|$. The weight initialization is performed by adjusting the Rayleigh distribution parameter of $\left|W\right|$.

The fact that the real and imaginary parts of the objective and activation functions are differentiable for each complex-valued parameter is a prerequisite for model backpropagation. When the loss function $\mathcal{L}$ is real, the complex chain rule is expressed as follows:

$${\nabla }_{\mathcal{L}}\left(z\right)={\displaystyle \frac{\partial \mathcal{L}}{\partial z}}={\displaystyle \frac{\partial \mathcal{L}}{\partial \Re \left(z\right)}}+j{\displaystyle \frac{\partial \mathcal{L}}{\partial \Im \left(z\right)}}=\Re \left({\nabla }_{\mathcal{L}}\left(z\right)\right)+j\Im \left({\nabla }_{\mathcal{L}}\left(z\right)\right)$$

(21)

where z is a complex variable.

3.3. Improved Attention Block for Complex-Valued Tensors

In PolSAR image classification tasks, information from different scattering channels of the center pixel and its surrounding window-sized neighborhood pixels are fed into the network for processing. Nevertheless, the contribution of pixels at different positions and scattering information from different channels to tasks varies significantly. Therefore, it is worth stimulating valuable information while filtering out irrelevant information in various dimensions. Since some attention mechanisms lead to huge computational effort [63], this article is inspired by [30] to design an improved attention mechanism capable of handling complex-valued tensors in CV-CNN. Coordinate Attention integrates positional information into channel attention processing, enabling it to sift through information across both spatial and channel dimensions. Its parallel one-dimensional feature encoding allows the model to attend to more crucial features with minimal computational overhead. The schematic diagram of our improved attention module is illustrated in Figure 5.

The improved attention module filters the complex-valued tensor across space and channels at the cost of a small computational overhead. Assume the input feature tensor is $X\in {\mathbb{C}}^{C\times H\times W}$, which can be divided into $\Re \left(X\right)\in {\mathbb{R}}^{C\times H\times W}$ and $\Im \left(X\right)\in {\mathbb{R}}^{C\times H\times W}$ according to the real and imaginary parts, where $\mathbb{R}$ denotes the real domain. Taking $\Re \left(X\right)$ as an example, using average pooling to encode its position in both H and W dimensions, the output can be represented as:

$$\begin{array}{cc}\hfill & a^h={\displaystyle \frac{1}{W}}\sum _{i=0}^{W-1}\Re \left(X(C,H,i)\right),a^h\in {\mathbb{R}}^{C\times H\times 1}\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill & a^w={\displaystyle \frac{1}{H}}\sum _{i=0}^{H-1}\Re \left(X(C,i,W)\right),a^w\in {\mathbb{R}}^{C\times 1\times W}\hfill \end{array}$$

(23)

where $a^h$ and $a^w$ are the result of operating on the height H and the width W, respectively.

After that, we convert the $a^h$ dimension to ${\mathbb{R}}^{C\times 1\times H}$ and splice it with $a^w$. A shared convolutional operation $F_s$ with kernel size 1 × 1 is implemented on the splicing result as:

$$l=\psi \left(F_s\left([a^h,a^w]\right)\right),l\in {\mathbb{R}}^{C/r\times 1\times (W+H)}$$

(24)

where $[·,·]$ is the splicing operation, $\psi (·)$ denotes a non-linear activation function, and r represents a reduction ratio for controlling block size.

Then, $l^h\in {\mathbb{R}}^{C/r\times 1\times H}$ and $l^w\in {\mathbb{R}}^{C/r\times 1\times W}$ are obtained by separating l along the spatial dimension. It should be noted that the dimension of $l^h$ is reconverted to ${\mathbb{R}}^{C/r\times H\times 1}$ here. Two independent 1 × 1 convolution operations $F_h$ and $F_w$ are employed for $l^h$ and $l^w$, respectively, to transform them to have the identical channel count as input X:

$$\begin{array}{cc}\hfill & g^h=\sigma \left(F_h\left(l^h\right)\right),g^h\in {\mathbb{R}}^{C\times H\times 1}\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill & g^w=\sigma \left(F_w\left(l^w\right)\right),g^w\in {\mathbb{R}}^{C\times 1\times W}\hfill \end{array}$$

(26)

where $\sigma (·)$ represents the sigmoid activation function. So far, the attention weight $W_{re}$ computed from the real part of the input X can be obtained as:

$$W_{re}=g^h\times g^w,W_{re}\in {\mathbb{R}}^{C\times H\times W}$$

(27)

Similarly the attention weight $W_{im}$ calculated from the imaginary part can be obtained. Finally, the outcome of our attention block Y can be expressed as:

$$Y=X\odot {\displaystyle \frac{W_{re}+W_{im}}{2}},Y\in {\mathbb{C}}^{C\times H\times W}$$

(28)

where ⊙ is the matrix element-wise product.

4. Experiments

4.1. Datasets

The performance of our designed method is validated by performing classification tasks on three datasets: Flevoland, San Francisco, and Oberpfaffenhofen. The details of each dataset are given below.

(1) Flevoland: The Flevoland dataset was acquired by the ARISAR system of NASA/JPL laboratory sounding over the Flevoland region of Poland in 1989, which consists of 1024 × 750 pixels and contains fifteen defined categories. Its corresponding Pauli-RGB map, ground truth, and category diagrams are depicted in Figure 6a–c, respectively.

(2) San Francisco: The San Francisco dataset was acquired by the RadarSat-2 satellite during its earth observation over the San Francisco region of the United States in 2008, which consists of 1024 × 900 pixels and contains five categories. Its corresponding Pauli-RGB map, ground truth, and category diagrams are depicted in Figure 7a–c, respectively.

(3) Oberpfaffenhofen: The Oberpfaffenhofen dataset was acquired by the German Aerospace Center during ground exploration over the Oberpfaffenhofen region of Germany, which consists of 1300 × 1200 pixels and contains three defined categories. Its corresponding Pauli-RGB map, ground truth, and category diagrams are depicted in Figure 8a–c, respectively.

4.2. Experimental Setting

The approach designed in this article is independently experimented on three selected datasets. For the Flevoland dataset, training involves randomly choosing 9% of the labeled samples from each category, while 1% is allocated to validation, leaving the remaining 90% for testing. For the San Francisco and Oberpfaffenhofen datasets, only 1% of the randomly selected labeled pixels are used for training, 1% for validation, and the remaining 98% for testing. Since the scattering noise present in the original PolSAR images interferes with the classification results, this paper employs refined Lee filtering with a 7 × 7 window size to filter each dataset before formal sampling [64]. In addition, the data for each channel are standardized.

During the model adjustment phase, the Adam algorithm is employed to update parameters. The batch size is 64, while the initial learning rate is 0.001. Training consists of 50 iterations with an early stopping mechanism to avoid overfitting, where training halts prematurely when the validation data loss fails to show improvement within 5 epochs.

To illustrate the effectiveness and elevation of the designed approach, we select six other PolSAR image classification methods to compare the results with our method, including SVM [12], CV-MLP [65], CV-2D-CNN [23], CV-3D-CNN [24], CV-FCN [25], CCN-WT [66], and PolSF [67]. In particular, SVM, CCN-WT, and PolSF are parameterized by the real domain, while the other complex-valued methods demonstrate superior performance compared to the real-valued counterparts in the corresponding papers. The details of each method are as follows:

• SVM: SVM is a classical and powerful supervised machine learning technique. It finds an optimal hyperplane in pixel-level classification by maximizing the distance between pixels of different categories to determine the categories. Specifically, SVM is based on kernels that map the data into a high-dimensional space to make them linearly divisible, thus solving the non-linear problems well.
• CV-MLP: CV-MLP uses a stack of fully connected layers to map the input image data into high-dimensional features. It extracts and transforms the features through hidden layers to learn the non-linear relationship between data and labels.
• CV-2D-CNN/CV-3D-CNN: Both CV-2D-CNN and CV-3D-CNN are capable of hierarchically extracting local abstract features for pixel classification. They are major deep neural networks. However, CV-2D-CNN extracts features in spatial dimensions under channel independence, while CV-3D-CNN can simultaneously process information in three dimensions.
• CV-FCN: CV-FCN learns the mapping between each pixel and its label end-to-end during classification. It eliminates the computational redundancy arising from the need to input a fixed-size patch around the center pixel when identifying it. Moreover, CV-FCN uses inverse convolution to upsample the extracted features and adapts well to inputs of arbitrary size.
• CNN-WT: CNN-WT is a new method based on deep neural networks and wavelet transform. It notes that the amount of PolSAR data is not enough to support a network with many layers, and the data are noisy. CNN-WT uses the wavelet transform for feature extraction and denoising of the raw data and feeds them into a multi-branch deep network for classification.
• PolSF: PolSF is a recently proposed PolSAR classification method for extracting abstract features based on the transformer framework. It analyzes complex dependencies within a region through a special local attention mechanism. Compared with shallow CNN models, PolSF has deeper layers and stronger feature extraction capability.

In addition, the complexity of each neural network model is shown in

Table 1. Complexity of different models.

Model	Params	FLOPs	MACs
CV-MLP	171,012	85,341	42,671
CV-2D-CNN	10,794	355,840	177,920
CV-3D-CNN	2,754,095	94,568,557	47,284,279
CV-FCN	964,316	438,260,314	219,130,157
CNN-WT	4,714,043	195,928,265	97,964,133
PolSF	1,351,961	689,628,230	344,814,115
CV-2D/3D-CNN-AM	2,716,891	40,489,291	20,244,646

, including parameter quantity, floating point operations (FLOPs), and multiply–accumulate operations (MACs). The inclusion of the attention module does not burden our model complexity. Compared to CV-3D-CNN, the designed hybrid model has a notable decrease in FLOPs and MACs as reflected when compared to CNN-WT and PolSF. Four frequently chosen metrics, including overall accuracy (OA), average accuracy (AA), kappa coefficient, and per category precision, are applied to evaluate the classification results. Furthermore, the training time of each method on each dataset is recorded. We conduct ten independent replications of each method on each dataset to obtain the mean of the above four metrics and the variance of the first three metrics.

For several involved methods, CV-MLP, CV-2D-CNN, CV-3D-CN, CV-FCN, CNN-WT, PolSF, and the proposed CV-2D/3D-CNN-AM are implemented based on the TensorFlow framework in Python. SVM is implemented based on the machine learning package sklearn in Python. The classification experiments are run on a workstation featuring a 13.75 M cache, 2.40 GHz CPU, a GTX 3070 graphics processor, and 64 GB onboard RAM.

4.3. Experimental Results of Flevoland

The result graphs of all involved methods operating on the Flevoland dataset are displayed in Figure 9. It can be visualized from the result maps that the accuracy of our proposed CV-2D/3D-CNN-AM outperforms the other compared approaches. The machine learning method SVM misjudges most regions of the building category due to its small labeled samples and has non-negligible error points in determining the other categories. CV-MLP and CV-2D-CNN improve classification accuracy for building identification compared to SVM, yet they erroneously classify a significant portion of bare soil pixels as water. CV-FCN adjudicates massive pixels to other classes in barley, water, and rapeseed, indicating that its learned abstract features are unstable. CV-3D-CNN is capable of learning the discriminative features for each category, but there is still room for improvement in the accuracy, such as the occurrence of continuous errors in the rapeseed region. CNN-WT exhibits numerous misclassified regions in the determination of wheat and forest, which is undesirable. PolSF has some misclassified pixels in the distribution area of classes such as potatoes, which limits its accuracy level. The CV-2D/3D-CNN-AM has excellent recognition ability for all categories in the dataset. In other words, the proposed method has a high recognition rate for categories with small distribution areas while ensuring high classifying precision for categories with more samples.

The specific experimental data of all approaches on the Flevoland dataset are displayed in

Table 2. Classification results of different methods on the Flevoland dataset.

Class	SVM	CV-MLP	CV-2D-CNN	CV-3D-CNN	CV-FCN	CNN-WT	PolSF	Proposed
Stembeans	85.01	98.31	97.40	99.21	97.97	99.73	98.90	99.84
Peas	92.83	96.81	99.01	99.46	98.27	99.95	99.65	99.97
Forest	92.52	91.65	98.54	99.02	97.19	98.96	95.29	99.92
Lucerne	97.25	97.74	98.54	99.96	95.25	98.79	98.76	99.91
Wheat	93.48	91.96	97.53	98.37	99.71	97.21	99.48	99.72
Beet	94.47	95.14	98.98	99.68	89.41	99.67	99.83	99.76
Potatoes	78.87	89.41	97.86	99.21	96.25	99.12	99.55	99.65
Bare Soil	99.16	16.45	44.67	97.50	96.08	99.79	99.01	99.99
Grass	92.80	81.13	94.22	93.33	93.89	88.90	98.88	99.37
Rapeseed	85.58	83.28	93.83	98.33	91.71	97.57	98.51	99.44
Barley	97.83	96.97	98.10	99.32	69.95	99.60	99.86	99.94
Wheat2	86.40	79.62	97.12	97.76	99.83	97.07	98.78	99.69
Wheat3	95.71	96.06	99.50	99.89	99.21	99.37	99.36	99.94
Water	99.79	98.59	99.20	98.83	75.53	99.09	99.99	99.98
Buildings	10.08	79.78	61.97	98.32	55.41	87.82	93.42	96.05
Training time (s)	39.42	45.62	91.78	1303.64	227.85	1434.63	3332.12	480.24
OA (%)	91.65 ± 0.358	90.64 ± 0.114	96.73 ± 0.564	98.81 ± 0.011	93.49 ± 4.197	98.34 ± 0.386	98.90 ± 0.145	99.78 ± 0.003
AA (%)	86.78 ± 0.316	86.19 ± 1.119	91.77 ± 3.622	98.55 ± 0.025	90.73 ± 6.413	97.51 ± 1.265	98.62 ± 0.009	99.55 ± 0.018
Kappa (×100)	90.88 ± 0.428	89.76 ± 0.138	96.43 ± 0.676	98.71 ± 0.013	92.75 ± 4.991	98.19 ± 0.459	98.80 ± 0.173	99.76 ± 0.004

. Figure 10 displays the loss and accuracy curve trends of the proposed method during the training and validation phases to verify its convergence. From the curve diagram, the proposed method rapidly attains a high level of accuracy following the initial few epochs of training, indicating its efficiency in capturing critical information. It is known that our CV-2D/3D-CNN-AM obtains the most favorable values for OA, AA, and Kappa from the table, corresponding to the results presented in Figure 9. Compared with CV-2D-CNN, although we consume more training time, the improvement achieved in the accuracy metric is very obvious. The increases in OA and AA are 3.05% and 7.78%, respectively. With the hybrid convolution and attention module, the proposed method takes less than half the training time of CV-3D-CNN, reducing training costs. Even in comparison to CNN-WT and PolSF, both of the latest techniques, our method exhibits significantly enhanced OA, AA, and Kappa. Notably, these improvements are achieved without any additional training time burden. In detail for each category, CV-2D/3D-CNN-AM achieves the highest accuracy in all classes except lucerne, beet, wheat2, and water. It is important to mention that the designed approach not only achieves the highest mean in the three metrics of OA, AA, and Kappa but also possesses the lowest variance in OA and Kappa, which implies that our method possesses high classification performance along with strong stability.

4.4. Experimental Results of San Francisco

The result graphs of all involved methods operating on the San Francisco dataset are displayed in Figure 11. The classification results provided by the other comparison methods are less desirable as seen in the result graphs. SVM and CV-MLP have limited fitting ability and contain numerous misclassification points within the vegetation and mountain zones. CV-MLP and CV-2D-CNN tend to confuse bare soil samples with their neighboring water samples resulting in low accuracy of bare soil category. The limited ability of CV-FCN to process border information leads to many errors in its classification results around the boundaries. Although CV-3D-CNN can handle mountain, water, and building categories satisfactorily, the discrimination accuracy for bare soil and vegetation pixels still needs to be raised. CNN-WT struggles to precisely discern the pixels belonging to green spaces, ultimately leading to a substantial accumulation of errors within the category’s region, compromising the accuracy of its classification results. In the PolSF classification diagram, while continuous and large bare soil areas are relatively well recognized, the precision of recognition for fragmented and small areas is less satisfactory. Compared with the above approaches, the designed method in this article accurately discriminates samples across all categories. In particular, it demonstrates exceptional proficiency in efficiently categorizing small, geographically scattered areas.

The specific experimental data of all approaches on the San Francisco dataset are displayed in

Table 3. Classification results of different methods on the San Francisco dataset.

Class	SVM	CV-MLP	CV-2D-CNN	CV-3D-CNN	CV-FCN	CNN-WT	PolSF	Proposed
Bare Soil	68.41	30.13	51.88	68.14	60.78	81.01	73.06	87.60
Mountain	87.57	84.22	96.26	98.15	95.92	97.87	97.21	97.75
Water	99.18	98.72	98.92	98.95	95.47	99.01	98.80	99.07
Building	95.73	95.71	97.83	97.52	96.23	98.61	97.38	98.18
Vegetation	68.78	67.42	79.85	82.73	90.07	65.00	79.81	86.10
Training time (s)	38.52	41.71	59.54	737.81	122.54	411.30	1950.58	374.43
OA (%)	94.24 ± 0.002	93.04 ± 0.081	96.17 ± 0.052	96.67 ± 0.058	94.88 ± 1.688	96.17 ± 0.032	96.36 ± 0.059	97.53 ± 0.016
AA (%)	83.93 ± 0.042	75.24 ± 8.746	84.95 ± 3.028	89.10 ± 1.349	87.69 ± 6.881	88.30 ± 1.157	89.25 ± 4.150	93.74 ± 0.859
Kappa (×100)	90.96 ± 0.006	89.03 ± 0.247	93.98 ± 0.132	94.78 ± 0.142	92.06 ± 3.825	93.96 ± 0.099	94.30 ± 0.125	96.13 ± 0.040

. The loss and accuracy curves are illustrated in Figure 12. Similarly, it can be obtained that the transition from low to high accuracy is efficiently accomplished by our model in only a few epochs. In comparison to CV-3D-CNN and PolSF, our method significantly excels in terms of training time. This superiority stems primarily from the integration of multi-level feature and attention modules, which not only empower the model with robust feature extraction capabilities but also facilitate swift identification of the optimal solution. Based on Table 3, it is evident that the proposed approach attains the highest scores for OA, AA, and Kappa with 97.53%, 93.74%, and 0.9613. Since the water pixels occupy a large portion of the graph and are easily discriminated by all methods, the lead of our method is not obvious on OA. However, our method exhibits a minimum improvement of 4.49% in AA compared to others. Particularly in the bare soil category, CV-2D/3D-CNN-AM acquires far superior accuracy than other methods, which indicates that the discriminative features acquired by our method are both generalized and resilient.

4.5. Experimental Results of Oberpfaffenhofen

The result graphs of all involved methods operating on the Oberpfaffenhofen dataset are displayed in Figure 13. Several comparative methods exhibit varying degrees of error rates in the ground object discrimination of each category. SVM incorrectly classifies a large proportion of built-up areas as open areas and there is also some misclassification within the woodland area. The CV-MLP recognition rates are unsatisfactory for all three surface feature types contained in the data, and there is a significant error region located at the bottom left corner of the image. The resultant plots of CV-2D-CNN are relatively cleaner than the previous two, but the discriminative correctness for built-up areas is still not high. CV-3D-CNN misidentifies badly in the area with more complex surface feature types at the lower right corner of the image. CV-FCN has low accuracy in classifying pixels near category boundaries, where the category changes are prone to massive and continuous misclassification. CNN-WT exhibits error dispersion across all three feature domains, with a pronounced tendency to inaccurately categorize numerous pixels within the built-up area region as open area, impeding its overall performance and accuracy. PolSF discriminates woodland areas well but encounters difficulties in classifying the image central region. By comparison, CV-2D/3D-CNN-AM enables better differentiation of samples in each category and performs more satisfactorily in boundary processing and region coherence.

The specific experimental data of all approaches on the Oberpfaffenhofen dataset are displayed in

Table 4. Classification results of different methods on the Oberpfaffenhofen dataset.

Class	SVM	CV-MLP	CV-2D-CNN	CV-3D-CNN	CV-FCN	CNN-WT	PolSF	Proposed
Built-up Areas	68.07	65.87	80.57	82.05	84.70	84.31	84.61	88.28
Woodland	84.97	83.07	92.41	93.25	89.80	91.72	95.10	96.07
Open Areas	97.86	95.93	97.54	98.02	98.45	93.45	95.80	98.23
Training time (s)	62.75	77.86	98.45	1319.88	129.27	1007.62	3251.98	521.56
OA (%)	87.98 ± 0.041	85.74 ± 0.081	92.33 ± 0.110	93.12 ± 0.212	90.57 ± 2.918	93.64 ± 0.006	92.87 ± 0.312	95.34 ± 0.043
AA (%)	83.63 ± 0.068	81.29 ± 0.340	90.17 ± 0.242	91.10 ± 0.517	89.31 ± 3.806	91.49 ± 0.019	91.84 ± 0.084	94.20 ± 0.036
Kappa (×100)	78.98 ± 0.124	75.27 ± 0.277	86.79 ± 0.326	88.15 ± 0.637	84.03 ± 7.725	89.02 ± 0.016	87.89 ± 0.708	92.01 ± 0.114

. The loss and accuracy curves are illustrated in Figure 14. Specifically, the curve’s change rate transitions from rapid to gradual, ultimately trending towards a continuously optimized state. Our model surpasses CV-3D-CNN, CNN-WT, and PolSF in terms of efficiency, achieving significantly more precise classification outcomes while requiring significantly less training time, underscoring its superior performance. Since this dataset contains fewer surface feature types, all methods can achieve a certain level of OA. However, there are still differences in the AA and per-category accuracy. It is evident that our model excels in terms of accuracy, achieving a remarkable 88.28% in the built-up zones and an outstanding 96.07% in the woodland zones. This represents a substantial improvement of at least 3.58% and 0.97%, respectively, over the other models. In addition, the proposed method yields superior values of 95.34%, 94.20%, and 0.9201 for three metrics, indicating that it has stronger classification performance.

4.6. Ablation Study

To verify whether the expansion to complex domains and the attention block have any impact on model performance, we execute RV-2D/3D-CNN, RV-2D/3D-CNN-AM, and CV-2D/3D-CNN on the three datasets, respectively. Among them, RV-2D/3D-CNN possesses the same network structure as the proposed method but removes the attention module and can only process real data. What is more, the attention block in RV-2D/3D-CNN-AM removes the calculation of complex parts. CV-2D/3D-CNN just masks out the attention module compared to our model. The ablation experiments on Flevoland, San Francisco, and Oberpfaffenhofen datasets are shown in

Table 5. Ablation experiments on the Flevoland dataset.

Methods	Complex	Attention	OA (%)	AA (%)	Kappa
RV-2D/3D-CNN	×	×	96.65	96.23	0.9634
CV-2D/3D-CNN	✓	×	98.20	97.32	0.9803
RV-2D/3D-CNN-AM	×	✓	98.53	98.65	0.9840
CV-2D/3D-CNN-AM	✓	✓	99.81	99.77	0.9979

,

Table 6. Ablation experiments on the San Francisco dataset.

Methods	Complex	Attention	OA (%)	AA (%)	Kappa
RV-2D/3D-CNN	×	×	95.46	85.74	0.9278
CV-2D/3D-CNN	✓	×	95.99	87.22	0.9365
RV-2D/3D-CNN-AM	×	✓	96.35	89.38	0.9428
CV-2D/3D-CNN-AM	✓	✓	97.32	93.71	0.9580

and

Table 7. Ablation experiments on the Oberpfaffenhofen dataset.

Methods	Complex	Attention	OA (%)	AA (%)	Kappa
RV-2D/3D-CNN	×	×	92.32	90.46	0.8683
CV-2D/3D-CNN	✓	×	93.39	91.39	0.8858
RV-2D/3D-CNN-AM	×	✓	93.13	91.14	0.8816
CV-2D/3D-CNN-AM	✓	✓	95.53	94.20	0.9232

, respectively, in which, “Complex” represents the complex computational domain, and “Attention” represents the attention block.

The experimental findings indicate that incorporating the attention module and directly inputting complex-valued data lead to an increase in classification accuracy. RV-2D/3D-CNN and RV-2D/3D-CNN-AM achieve inferior results to the complex-valued models due to their inability to utilize the phase information contained in complex elements. The model’s performance with the added attention module is superior to the non-added one in both real and complex domains. This confirms that the attention block helps the model to identify more important information and thus enhances the feature extraction ability. Notably, even without the addition of an attention module, only CV-2D/3D-CNN achieves accuracy comparable to a single 2D or 3D model. It shows that the hybrid model itself has a decent feature extraction capability. Moreover, our model performs best on all datasets by incorporating the improved attention block and expanding to the complex domain. In particular, the AA of CV-2D/3D-CNN-AM is notably improved on all three datasets by at least 1.08%, 5.36%, and 2.62%, respectively. This suggests that multi-level and targeted abstract features help our model to recognize all types.

5. Conclusions

We propose a simple, practical, and novel PolSAR image classification method called CV-2D/3D-CNN-AM, which combines the attention mechanism with a complex-valued CNN framework that hybridizes 2D and 3D convolutions. First, the model is extended to the complex domain to fully exploit the amplitude and phase information in complex-valued PolSAR data. Secondly, a hybrid of 2D and 3D convolutions extracts rich discriminative features while reducing the computational parameters. In addition, the attention mechanism module serves as a filter, allowing the model to focus on information that is more critical to the practical task. The effectiveness of each module in the designed framework is verified in ablation experiments. CV-2D/3D-CNN-AM is tested on three PolSAR datasets and achieves more advanced results than the other methods. Future work will focus on PolSAR image classification under limited-labeled samples or unsupervised scenarios. By employing self-supervised learning, domain generalization, and other methods to reduce the dependence on labeled samples, it seeks to effectively tackle the inherent complexities involved in the interpretation of PolSAR images.