Sparse Weighting for Pyramid Pooling-Based SAR Image Target Recognition

Abstract

In this study, a novel feature learning method for synthetic aperture radar (SAR) image automatic target recognition is presented. It is based on spatial pyramid matching (SPM), which represents an image by concatenating the pooling feature vectors that are obtained from different resolution sub-regions. This method exploits the dependability of obtaining the weighted pooling features generated from SPM sub-regions. The dependability is determined by the residuals obtained from sparse representation. This method aims at enhancing the weights of the pooling features generated in the sub-regions located in the target and suppressing the weights of the background. The feature representation for SAR image target recognition is discriminative and robust to speckle noise and background clutter. Experiments performed on the Moving and Stationary Target Acquisition and Recognition public dataset prove the advantageous performance of the presented algorithm over several state-of-the-art methods.

1. Introduction

Synthetic aperture radar (SAR) imagery has become a significant research object in many areas, such as civilian and military fields [1,2,3]. It takes advantage of acquiring images in all weather conditions and during the night as well as the day. Image target recognition is a basic step in understanding and interpreting SAR images [4]. In this context, it is important to develop discriminative and robust methods for automatic target recognition (ATR) systems, and tremendous research attention has been paid to the study of ATR for SAR images [5,6,7,8,9].

Recently, sparse representation (SR) has become a focus and has been used in many areas [10]. It is robust to noise and can maintain natural discrimination without any prior information. Even in SAR target recognition, SR could remove the need for the pose estimating process. Recently, Thiagarajan et al. [11] applied SR to realize SAR image target recognition and applied a local linear approximation to generate a classification prediction for every target class manifold. This algorithm did not demand any specific pose estimation or preprocessing, but the use of random projections in the high-dimensional space discarded some discriminative locality information, thus making occlusion handling more difficult. In another study [12], descriptors from local patches were extracted, and an image was treated as a collection of the unordered descriptors; then, sparse representation was applied to represent the local patches for SAR image target classification in the framework of SPM. Additionally, the application of spatial pyramids was confirmed to be an effective classification method for SAR images.

This model of spatial pyramid matching [13] for image classification is a statistics-based model with the objective of providing a better image representation. In order to obtain discriminant details of the images, the SPM needs to extract the low-level local features through, for example, scale-invariant feature transform (SIFT) [14] and histogram of oriented gradient (HOG) [15]. However, the local features are not provided directly to image classifiers due to their sensitivity to noise and computational complexity. One solution is to represent the images by integrating the local features into the midlevel features. This image representation works well with linear classifiers, and the results have achieved a competitive performance in many image classification tasks. Nonetheless, the SPM model is not perfect when implemented on SAR images. This is attributed to the fact that the variety of targets posed in SAR images hinders the advantages of SPM. Noteworthy, locality is more essential than sparsity [16], locality-constrained linear coding was presented in place of the vector quantization (VQ) coding, and a good approximation was obtained. Recently, Zhang et al. [17] proposed to apply a locality constraint to ensure similar patches shared similar codes in the coding scheme for SAR image target recognition. However, its complete codebook was obtained after preprocessing of the estimation of target poses.

In a literature study [5], a complementary spatial pyramid coding method was used in change detection, and good performance was gained. The SAR targets suffer from the effects of the speckle noise and the background; therefore, different parts of an image play different roles in image representation. Combining the advantages of SPM and SR, a novel SAR image target recognition method is proposed herein, which makes use of the dependability obtained by SR to obtain weighted sub-regions at every pyramid level. Some sub-regions in each level of the spatial pyramid may consist of background noise, and other ones represent the target. Based on the sparse representation theory, the target parts could be represented by the training samples from the same class [18]. Therefore, there could be a small residual value corresponding to its class, which shows the dependability of the sub-region. We apply the dependability to weight the pooling features obtained from the SPM sub-regions [19]. Therefore, the pooling feature located in the target is enhanced. Meanwhile, the pooling feature located in the background is suppressed. The results obtained using real SAR Moving and Stationary Target Acquisition and Recognition (MSTAR) database demonstrate that the method presented herein is more robust to variant unconstrained conditions than the methods reported in other recent related studies.

The organization of this paper is as follows: Section 2 introduces the presented sub-regions weighting method. Section 3 reports the experimental results of the presented algorithm and compares the approach used herein with some classical approaches. Finally, Section 4 includes the conclusion.

2. SAR Image Recognition with Sparse Weighting Spatial Pyramid Pooling

The method proposed herein simultaneously utilizes the SPM model and SR to deal with SAR image recognition. Enlightened by the idea that different parts of an image play different roles, a sparse weighting spatial pyramid pooling method is proposed to extract a new type of feature. The main objective of utilizing this method is to reduce the influence of background clutter and enhance the target. Figure 1 exhibits the flowchart of the proposed SAR image target recognition method. Firstly, an image was divided into gradual fine sub-regions; then, the dense local features were calculated, followed by coding and pooling according to SPM, in order to obtain feature vectors of the sub-regions, respectively. Additionally, the pooling feature vector of the sub-regions at each pyramid level was weighted based on the dependability, which was determined according to the residuals obtained by the SR. Finally, the representation for SAR images was built by systematically concatenating the weighted feature vectors. With sparse representation classification, the method is robust, in particular, in dealing with the speckle noise and large clutter background.

2.1. Local Feature Extraction and Descriptor Quantization

Considering that the SIFT feature has the characteristic of invariance to scale, orientation, and affine distortion, in this study, dense-SIFT descriptors were extracted to express an SAR image [14]. The image $I$ can be denoted by a set of local features’ descriptors as: $I=[d_1,d_2,\dots ,d_N]\in {\mathrm{R}}^{D\times N}$, where $d_i$ represents the $D-\mathrm{dimensional}$ SIFT description vector. Given a codebook $B=[b_1,b_2,\dots ,b_M]\in {\mathrm{R}}^{D\times N}$, the K-Means clustering algorithm [13] was adopted with the Euclidean distance to cluster local features into groups, and the generated centers of each group were taken as the codebook.

Sparse coding [10] was applied to encode the local feature vectors, as follows:

$$\arg \underset{V}{\min }{\displaystyle \sum _{i=1}^N\left|\right|d_i-B{v}_i{\left|\right|}_2+\lambda \left|\right|v_i{\left|\right|}_1}.$$

(1)

The feature vector for a descriptor $d_i$ becomes an M-dimensional vector $v_i$. $v_i$ is the corresponding vector to the descriptor $d_i$. The sparse coding vectors are obtained by the feature-sign search algorithm [20] by solving Equation (1) here.

The next step is the spatial pooling step aiming at obtaining more discriminative image representation from each sub-region. This study applied the construction of a three-level spatial pyramid. At every resolution $e$, $e$ = 0, 1, 2, a grid was constructed such that there were $2^e$ resolution cells along every dimension, i.e., the 1 × 1, 2 × 2, 4 × 4 grid structures; thus, the sub-regions were $K$ = 21 in total. We let $V$ be a collection containing $T$ local feature codes acquired from a sub-region. In addition, the max-pooling strategy was used to concatenate all the descriptors for the sub-region, which can acquire many discriminative features to SAR image spatial variations. The expression of the max-pooling is as follows:

$$t_k=\max (v_1,v_2,\cdots ,v_T)$$

(2)

where max represents the element-wisely maximization for the involved vectors. The local features were pooled in all the sub-regions and concatenated to form the image representation $f=[t_1;t_2;\cdots ;t_K]$.

Considering a set of $G$ training images from $C$ classes, $F=[f_1,f_2,\cdots ,f_G]$, where $f_g$ is the feature vector, which is the pooling result of the $g\mathrm{th}$ image. Correspondingly, $F$ is partitioned as $F={[F_1^T,F_2^T,\cdots ,F_K^T]}^T$ and $F_k\in R^{d\times G}$ with $d<G$. Distinctly, the $g\mathrm{th}$ column of $F_k$ is the feature vector for the $k\mathrm{th}$ sub-region of the $g\mathrm{th}$ image, and $G$ represents the image number of the training dataset. Analogously, a test image $y$ is divided into $y=[y_1;y_2;\cdots ;y_K]$.

2.2. Determination of Sub-Region Weights

The traditional SPM models use all the extracted local descriptors for image representation; therefore, some of them might not be the objects to be recognized. Thus, it was required to learn the spatial weights to boost the target part and weaken the background parts. Taking the sparse representation into account, the observation to be followed was that the target part was sparser than the background one. The feature vectors of the sub-regions at every pyramid level were weighted on the residual basis to determine their dependability [21]. Then, the following optimization problem [10] was obtained:

$${\widehat{x}}_k={\mathrm{argmin}}_{x_{{}_k}}\left\{\right||F_k{x}_k-y_k{\left|\right|}_2^2+\lambda \left|\right|x_k{\left|\right|}_1\}\begin{array}{cc}\mathrm{s}.\mathrm{t}.& 1\le k\le K\end{array},$$

(3)

where $y_k$ is the pooling feature for each sub-region, and $F_k$ is the dictionary.

The residuals of a sub-region can be measured simply by using the L2-norm as follows:

$$r_{kc}=\left|\right|F_k{\delta }_c({\widehat{x}}_k)-y_k{\left|\right|}_2,$$

(4)

where $r_{kc}$ is the residual corresponding to the $c\mathrm{th}$ class of the $k\mathrm{th}$ sub-region, ${\delta }_c\left({\widehat{x}}_k\right)$ is the discriminative function, which selects the elements related with the $c\mathrm{th}$ class in ${\widehat{x}}_k$, $c=[1,2,\cdots ,C]$, and $C$ is the class number of the training dataset.

All the residuals of the $k\mathrm{th}$ sub-region were concatenated to generate the residual vector $r_k$ as follows:

$$r_k=[r_{k1},r_{k2},\cdots ,r_{kC}].$$

(5)

According to the theory of SR [10], target sub-regions can be accurately represented only by the training samples from the same class. Therefore, the residual vector produced contains only one small element. The background sub-regions could be far away from every subspace spanned by the training samples of each class but nearly within the subspace spanned by all training samples of all classes. The residual vector produced could include almost similar elements. According to the abovementioned analysis, the following function was applied for evaluating the sub-region sparsity:

$${\varsigma }_k=\frac{\min (r_k)}{\mathrm{mean}(r_k)}.$$

(6)

When $r_k$ has only a single zero or near zero residual, ${\varsigma }_k$ reaches the minimum value 0. This shows that the sub-region could be well represented by its subspace. When all residuals in $r_k$ are nonzero value and equal, ${\varsigma }_k$ reaches the maximum value 1. This indicates that the sub-region could contain noise or it could be the background part.

This is verified by the numerical value results shown in Figure 2. Figure 2a,b are example images, which illustrate the distribution of sub-regions in the construction of a three-level pyramid. Figure 2c,d show ${\varsigma }_k$ of the sub-regions corresponding to Figure 2a,b respectively. The obtained residuals indicated that the residuals’ sparsity ${\varsigma }_k$ generated in the sub-regions located in the target parts (such as the sub-regions marked by 11 and 12 in Figure 2a) was smaller than the background parts (such as the sub-regions marked by 9 and 10 in Figure 2a) in the third-level pyramid. Additionally, a difference existed between the residual values in the different resolutions, such that the residuals were much greater in the low resolution level than in the high ones, in particular, the target parts. The reason could be that more background cluster and speckle noise was included in a larger region. Therefore, in order to differentiate the target sub-regions from the background ones, the residual could be employed to obtain the weighted feature vector.

To tackle this problem, a simple way was to impair the feature representation of the background sub-regions for the classification. This was achieved by weighting all sub-regions with the reciprocal of the sparsity: ${\omega }_k$, which is viewed as the dependability:

$${\omega }_k=\frac{1}{{\varsigma }_k}$$

(7)

2.3. Weighted SPM Sparse Representation

After weighting the sub-regions, the weighted sub-regions were integrated to reconstruct a global feature vector according to the SPM model. Each sub-region vector $y_k$ was weighted by a corresponding weight to be represented as a sub-region weighted query vector:

$$y_k^{\omega }={\omega }_k{y}_k.$$

(8)

We connected all sub-regions’ weighted query vectors in series to produce a weighted feature vector as

$$y^{\omega }={[y_1^{\omega T},y_2^{\omega T},\cdots ,y_K^{\omega T}]}^T.$$

(9)

Consider that $y^{\omega }$ can be represented by $F$ in a linear manner as follows:

$$y^{\omega }=F{u}^{\omega }.$$

(10)

To obtain the primary label information from the training dataset matrix and weighted query feature vector, the SR classification method [13] was applied to complete the target image classification:

$${\widehat{u}}^{\omega }={\mathrm{argmin}}_{u^{\omega }}\left\{\right||F{u}^{\omega }-y^{\omega }{\left|\right|}_2^2+\lambda \left|\right|u^{\omega }{\left|\right|}_1\}.$$

(11)

Linear approximation [15] is capable of selecting significant training vectors to provide a good discrimination among all the classes.

2.4. Classification Procedure

After the representation coefficient ${\widehat{u}}^{\omega }$ was obtained, the given weighted global query vector $y^{\omega }$ and the global training matrix $F$ were applied to calculate the minimum reconstruction residual; then, the query image to the subject $c$ that had the global minimum residual [12] was allocated:

$$c={\mathrm{argmin}}_j\left|\right|F{\delta }_j\left({\widehat{u}}^{\omega }\right)-y^{\omega }{\left|\right|}_2.$$

(12)

3. Contrast Experiment Results and Analysis

3.1. Dataset and Experimental Conditions

This section presents the experimental results on the MSTAR [22] public database to assess the performance of the presented algorithm. Three types of vehicles are included in the dataset. They are BMP2 armored personnel carriers and BTR70 and T72 main battle tanks with depression angles at 17 degrees and 15 degrees. Every image in the dataset is 128 × 128 pixels, covering the full 0–to–360 degree aspect range. The resolution of each image is 0.3 m × 0.3 m. The training set contained the SAR target images whose depression was 17 degrees. The other images with a depression of 15 degrees were regarded as the testing set. The visible light images are illustrated in Figure 3.

Table 1. The dataset of training and testing targets.

Target	Name of Training Set	Training Set	Name of Testing Set	Testing Set
BTR70	Sn-c71	233	Sn-c71	196
BMP2	Sn-9663	233	Sn-9663	195
			Sn-9666	196
			Sn-c21	196
T72	Sn-132	232	Sn-132	196
			Sn-812	195
			Sn-s7	191

lists the details of the training set and testing set.

To reduce the negative interaction of the redundant background, a 64 × 64 pixels sub-image was cropped from the center of every image. Moreover, the amplitude of the sub-image was normalized. Herein, the experiment result, which is reported, was the average classification result of running 10 times.

In this experiment, the SIFT [14] was applied as the local feature. We extracted the SIFT features from patches densely located by six pixels on the image, under the scales of 16 × 16. Furthermore, every SIFT descriptor had the dimension of 128.

3.2. Experiment 1: Investigation of the Proposed Method

In order to verify the superiority of the presented algorithm, the presented sparse weighting sparse pyramid pooling (SWSPP) algorithm was compared with the state-of-the-art methods: namely sparse coding SPM (ScSPM) [13] and locality-constrained linear coding SPM (LLC) [16]. The classification results are listed in

Table 2. MSTAR classification results (%).

$\mathit{d}\mathit{i}\mathit{m}$	Data	ScSPM	LLC-SPM	SWSPP
100	BMP2	77.27	76.17	80.99
	BTR70	92.35	95.77	97.04
	T72	85.36	88.24	85.02
	av	84.99	86.72	87.68
200	BMP2	79.66	80.82	83.79
	BTR70	94.13	95.36	98.03
	T72	86.17	88.09	87.29
	av	86.65	88.09	89.70
400	BMP2	83.65	83.48	87.67
	BTR70	96.73	96.73	99.39
	T72	86.87	87.79	88.14
	av	89.08	89.33	91.73
600	BMP2	84.16	85.33	89.27
	BTR70	97.19	97.86	100
	T72	87.63	88.36	89.00
	av	89.66	90.52	92.76
800	BMP2	84.67	86.51	91.31
	BTR70	98.16	97.81	100
	T72	88.38	88.95	90.03
	av	90.41	91.09	93.78
1024	BMP2	87.05	86.46	90.46
	BTR70	96.94	98.78	100
	T72	89.18	89.78	92.27
	av	91.06	91.67	94.24

. The performance of the codebook was compared under different dimensions $dim$ from 100 to 1024. The greater the dimensionality, the more the feature extracted was discriminative. Therefore, the results improved as the dimension increased. In all three scenarios, the method presented in this study was obviously superior to the ScSPM and LLC methods under all the dimensions of the codebook. Since the proposed SWSPP took the sub-regions dependability into account, it enhanced the discrimination of the recognition target. From the experiment results, we verified that the dependability generated according to the sparse representation was effective for classification. The curves of the classification average results under the three algorithms are shown in Figure 4.

3.3. Experiment 2: Comparison of SWAPP with Related Algorithms

In order to show a more rounded analysis, the presented approach was compared with the following methods: the sparse representation classification (SRC) method [11], the PCA feature [23], and the LDA feature [24].

Table 3. Classification results (%) of different methods.

Algorithm	BMP2	BTR70	T72	Average
SRC [11]	84.48	98.33	94.59	92.47
PCA [23]	94.54	85.66	96.79	92.33
LDA [24]	98.47	67.80	95.86	87.38
SWSPP	90.46	100	92.27	94.24

gives the average classification results. According to this table, we can see that the presented method had advantages over all the other methods. The average classification result of our proposed method was 2% higher than the other approaches. On the basis of the results, we can see that the classification accuracy of the presented method was better; in particular, the correct rate of the BTR70 was 100%. The result validates the effectiveness of the proposed dependability weighted feature learning method for SAR image recognition.

4. Conclusions

In this study, a robust synthetic aperture radar (SAR) automatic target recognition approach was presented that applied sparse representation to produce a weighted global feature vector based on spatial pyramid matching. SAR image target recognition was performed on the reconstructed image representation. The experimental results clearly and consistently showed that the proposed framework was significantly more discriminative than the traditional SPM methods. Moreover, the feature coding–pooling framework was shown to perform well in image classification tasks, the unavoidable information loss incurred by the feature quantization in the coding process and the undesired dependence of pooling on the image spatial layout, however, may severely limit the recognition performance. Therefore, more systematic investigations are still required to find a new coding method to improve the classification performance in future research.