FEMSFNet: Feature Enhancement and Multi-Scales Fusion Network for SAR Aircraft Detection

Abstract

Aircraft targets, as high-value subjects, are a focal point in Synthetic Aperture Radar (SAR) image interpretation. To tackle challenges like limited SAR aircraft datasets and shortcomings in existing detection algorithms (complexity, poor performance, weak generalization), we present the Feature Enhancement and Multi-Scales Fusion Network (FEMSFNet) for SAR aircraft detection. FEMSFNet employs diverse image augmentation and integrates optimized Squeeze-and-Excitation Networks (SE) with residual network (ResNet) in a SdE-Resblock structure for a lightweight yet accurate model. It introduces ssppf-CSP module, an improved pyramid pooling model, to prevent receptive field deviation in deep network training. Tailored for SAR aircraft detection, FEMSFNet optimizes loss functions, emphasizing both speed and accuracy. Evaluation on the SAR Aircraft Detection Dataset (SADD) demonstrates significant improvements compared to the contrasted algorithms: precision rate (92%), recall rate (96%), and F1 score (94%), with a maximum increase of 12.2% in precision, 12.9% in recall, and 13.3% in F1 score.

1. Introduction

Synthetic Aperture Radar (SAR) stands out as an active microwave remote sensing technology, offering uninterrupted, all-day, and all-weather Earth surface observation [1]. It remains unaffected by conditions such as illumination, clouds, or weather, making it indispensable in remote sensing [2]. SAR has found extensive applications in both military and civilian domains, emerging as a pivotal tool for information acquisition [3]. In military contexts, the detection of aircraft holds a central position in air defense research, leveraging the distinctive advantages offered by SAR images [4]. Consequently, there is a global research emphasis on enhancing aircraft target detection in SAR imagery.

SAR imaging, distinct from optical methods, poses challenges in detecting and identifying aircraft targets due to its longer wavelength and complex mechanism [5]. The irregular distribution of land clutter, marked by bright backscattering points, introduces interference [6]. SAR images often showcase intricate terrain features, complicating aircraft target detection as these features may mimic the representation of targets. Targets in SAR images manifest as irregular bright spots, necessitating spot integration for effective recognition. The varied imaging characteristics of aircraft targets in SAR images, combined with fluctuating scattering conditions, reduce the relevance of traditional manually designed features [6,7,8,9]. Detecting aircraft targets in SAR images is, therefore, a significant but intricate research direction.

Traditional SAR image target detection relies heavily on model features, encompassing characteristics like the target’s outline, size, texture, and scattering center [10,11,12,13]. A common traditional algorithm is the constant false alarm rate (CFAR) based on clutter statistics and threshold extraction [14]. Scholars have delved into statistical features and non-uniform backgrounds, proposing improved CFAR algorithms like CA-CFAR [15], SOCA-CFAR [16], GOCA-CFAR [17], OS-CFAR [18], and VI-CFAR [19]. Ai et al. [20] proposed an SAR detection algorithm using bilateral fine-tuning thresholds, enhancing performance in ocean backgrounds by fitting the target to clutter with higher contrast. Chen et al. [21] introduced an improved constant false alarm rate detection algorithm based on multiscale contrast and variable windows, elevating target detection accuracy in SAR images. Model-based recognition methods achieve heightened target recognition accuracy with an evolving template database. However, this approach demands multiple iterations for high-precision simulated images, taxing computation speed, and model accuracy. Furthermore, model-based methods suffer from high computational complexity and low efficiency. Consequently, researchers are increasingly exploring machine learning algorithms, such as support vector machines, neural networks, and adaptive enhancement, for automatic interpretation of SAR targets.

Recently, deep learning-based target detection has experienced rapid development across various fields [22,23,24,25]. Notable strides have been achieved in aircraft target recognition in Synthetic Aperture Radar (SAR) images through deep learning [26,27,28]. This progress is largely credited to the automatic learning and pattern recognition capabilities inherent in deep learning methods for handling complex features. Zhao et al. [29] introduced an SAR aircraft detection algorithm leveraging dilated convolution and attention mechanisms, creating a novel pyramid dilation network to optimize aircraft feature extraction in SAR images. Wang et al. [30], utilizing the SSD object detection framework, applied a strategy integrating transfer learning and data augmentation to enhance SSD’s target detection performance in SAR images. In SAR aircraft target detection, scholars commonly modify existing algorithms to meet specific requirements for satisfactory results [31,32]. However, achieving a balance between model complexity and detection accuracy can be challenging, and existing algorithms may struggle in such scenarios. Moreover, aircraft in SAR images may experience deformation due to radar geometry effects, altering the target shape and complicating detection. Additionally, deep learning methods rely on ample samples for supervised training, and inadequate samples can result in overfitting, adversely affecting detection performance.

To address these challenges, we propose a Feature Enhancement and Multi-Scales Fusion Network (FEMSFNet). This network aims to improve detection accuracy while minimizing model complexity, achieving a balance between the two. Firstly, FEMSFNet employs a diverse image enhancement technique [33,34,35], applying methods like noise, mosaic, mixup, rotation, and cropping to enhance image features. This addresses the scarcity of SAR aircraft image data, enhancing sample diversity for improved generalization and robust network training. Secondly, drawing inspiration from Yolov4-tiny [36,37,38], FEMSFNet utilizes the CSPDarknet53-tiny network [39] as the backbone to create a lightweight model. It incorporates a residual module based on an improved attention mechanism, focusing more on critical image regions for enhanced recognition accuracy. Thirdly, the paper introduces superior CSP [40] structures based on an improved feature pyramid [41], preventing a reduction in the network’s receptive field and the loss of target feature information in deep structures. Finally, tailored for SAR aircraft target detection, FEMSFNet optimizes loss functions [42,43] while implementing cosine annealing learning rate decay [44], and includes label smoothing [45] techniques to prevent overfitting, expedite convergence, and improve regression accuracy.

The main contributions of our work are as follows:

• FEMSFNet, as proposed, prioritizes both speed and accuracy in target detection. It leverages the lightweight CSPDarknet53-tiny as the backbone for efficient feature extraction. The model’s performance has undergone evaluation using the SAR Aircraft Detection Dataset (SADD).
• To maintain a lightweight model without compromising detection accuracy, we integrate the optimized Squeeze-and-Excitation Networks (SE) attention module with the ResNet module in the backbone, forming the SdE-Resblock structure.
• To prevent the deep network from causing a deviation in the receptive field during training, leading to ineffective global feature fusion and loss of feature information, we propose a CSP structure based on an improved pyramid pooling model, called ssppf-CSP.
• Considering the unique characteristics of SAR aircraft target detection, FEMSFNet optimized the network’s loss functions while implementing techniques such as learning rate cosine annealing decay and label smoothing to prevent overfitting, ultimately enhancing convergence speed and regression accuracy.

The rest of the paper is arranged as follows. Section 2 describes the proposed aircraft detection network in detail. Experimental results, as well as performance evaluation, are presented in Section 3, and a detailed discussion of the results is provided at the end of this section. Section 4 briefly summarizes this paper.

2. Materials and Methods

2.1. Overview of FEMSFNet

Yolov4-tiny, a lightweight variant of the YOLO (You Only Look Once) object detection series, is tailored for real-time object detection on devices with constrained computational resources. In contrast to Yolov4 [36], Yolov4-tiny boasts a smaller model size and reduced computational complexity. With outstanding performance in natural settings, we adopt Yolov4-tiny as the baseline for our work. The overall structure of FEMSFNet is illustrated in Figure 1.

FEMSFNet comprises five key modules: Input, Image Enhancement, Backbone, Neck, and Prediction. Recognizing the pivotal role of sample quality in network training effectiveness, the Image Enhancement module employs various methods, including mosaic, mixup, rotation, scaling, and cropping, with a certain probability. These diverse image enhancement techniques augment the dataset, addressing limitations in quantity and diversity in SAR aircraft image datasets. The results of image enhancement are illustrated in Figure 2. Drawing inspiration from Yolov4-tiny, we select the lightweight CSPDarknet53-tiny as the Backbone feature extraction network for FEMSFNet to strike a balance between detection accuracy and efficiency. The Backbone network integrates the Basic-Conv convolutional module, SdE-Resblock residual module, and ssppf-CSP deep feature fusion module. The Basic-Conv convolutional module consists of a convolution block, batch normalization (BN) [46] block, and the SiLU [47] activation function block.

Following image enhancement and feature extraction by the Backbone network, FEMSFNet channels feature maps of different scales to the Neck layer. This layer incorporates a feature pyramid structure with an attention mechanism, fusing feature maps from the Backbone at three scales. During feature down-sampling, an attention mechanism prioritizes the target region for high-resolution, semantically rich features. The resulting feature maps are then input into the Prediction module for inference. Subsequently, predictions are compared with ground truth labels, and loss is computed. The error undergoes backpropagation through the network, and the Adam optimizer [48] with a cosine annealing learning schedule adjusts weights and parameters iteratively until the end of the training loop. To address the nuances of SAR image aircraft single-object detection and prevent issues like model overfitting, we utilize an optimized loss function in the calculation. The subsequent section provides a detailed explanation of our methods.

2.2. SdE-Resblock

Residual Network [49,50] (ResNet) is a deep learning model architecture specifically crafted to mitigate challenges like vanishing gradients and exploding gradients encountered in the training of deep neural networks. ResNet facilitates the construction of exceptionally deep networks without experiencing performance degradation. The fundamental concept behind ResNet is the incorporation of residual blocks, allowing the input signal to be directly forwarded to the block’s output via skip connections. This mechanism, termed residual learning, enables the model to learn residuals rather than mapping directly. This approach simplifies the training of deep networks and enhances their effectiveness. The success of ResNet has served as inspiration for the design of numerous subsequent deep learning architectures, establishing it as a classic example in the construction of deep neural networks [51]. The residual structure in the Yolov4-tiny network is depicted in Figure 3. In the illustrated residual network, there are four DarkNet blocks and one MaxPooling block. The DarkNet module consists of fundamental convolutional layers, Batch Normalization (BN) layers, and ReLU activation layers. It serves as one of the essential building blocks of the entire FEMSFNet architecture. Here, the convolutional layers are tasked with extracting features from images. The BN layers address the issue of increasing training difficulty and slower convergence as the depth of the neural network grows. The ReLU activation layers map the input from neurons to the output, introducing sparsity within the network by rendering outputs of some neurons to zero. This sparsity reduces the interdependency of parameters, thereby mitigating the risk of overfitting. The input data undergo the first convolutional block, producing Map1, which is then split into two parts. The second half passes through the second convolutional block, resulting in Map2. Map2 undergoes further computations in the third convolutional block, generating results. The obtained result is concatenated with Map2 and input into the fourth convolutional block, producing Map3. Finally, Map1 and Map3 are concatenated, and the result undergoes max-pooling to obtain the final output.

Attention mechanisms, known for selectively focusing on important information, improve model performance on relevant tasks [52,53]. Integrating attention mechanisms into ResNet enhances the network’s attention to crucial features [54]. Hence, we devised the SdE-Resblock structure by combining the optimized Squeeze-and-Excitation (SE) attention [55] module with the ResNet module in the Backbone. This structure assigns varied weights to different channels of the network’s feature maps, prioritizing the target region for enhanced training efficiency. To counter overfitting during the Excitation phase, where non-linear mapping and adjustment of squeezed features occur, we introduced Dropout [56] techniques for regularization into SE module. The detailed network architecture is depicted in Figure 4.

Compared to the residual structure in Figure 3, the SdE-Resblock introduces an attention mechanism into the residual branch. Without altering the size and number of channels of the feature map, it employs Squeeze-and-Excitation along with regularization operations to assign varying weights to different channels in the residual branch. This enables the network to concentrate more attention on the target region, facilitating easier and more emphasized recognition of target features.

To gain deeper insights into the model’s decision-making process, the distribution of the attention mechanism was visualized through heatmaps. As illustrated in Figure 5, the heatmap unveils the areas of focus while the model processes the input data. In this figure, warmer colors (such as red) denote higher degrees of attention by the model to those areas, whereas cooler colors (like blue) indicate lower levels of focus. This visualization method clearly demonstrates the model’s tendency to concentrate on specific parts of the input data. Notably, the model significantly zeroes in on the airplane regions within the input image, aligning with expectations, as these areas typically contain crucial information necessary for object recognition. Moreover, the heatmap further reveals the model’s ability to effectively ignore background noise or information irrelevant to the task at hand, underscoring the efficacy of the attention mechanism in enhancing the model’s focus on vital information and improving overall performance. In summary, the attention mechanism’s heatmap not only provides an intuitive view of the model’s learning and decision processes but also affirms the model’s capability to efficiently identify and utilize key information within the input data for accurate predictions.

2.3. ssppf-CSP

In typical scenarios, deep neural networks often exhibit a bias in the receptive field during training, hampering effective integration of global features and causing loss of feature information. To address this challenge, we propose the ssppf-CSP structure (soft-spatial pyramid pooling-fast-CSPnet), leveraging an enhanced pyramid pooling model. This structure employs multi-level pooling operations to extract and merge local and global features, thereby improving the model’s receptive field. The sppf (spatial pyramid pooling-fast) structure [57] achieves pooling effects of large-sized layers through stacking smaller layers, enhancing the network’s expressive capability through multi-scale feature fusion. In contrast to traditional sppf modules using max-pooling, we employ soft-pooling in the ssppf module to minimize information loss during pooling, preserving detailed information for detection. Integrated with the final feature extraction module, CSPnet, in FEMSFNet, the ssppf module uses soft-pooling with four window sizes ($6\times 6, 3\times 3, 2\times 2, 1\times 1$) to map high-level feature information to low-level features. Skip connections concatenate high-level semantic information with shallow-level information after pooling, overcoming performance loss in deeper network structures. The detailed architecture of ssppf-CSP is illustrated in Figure 6.

In the ssppf-CSP structure, the input image undergoes convolutional operations, leading to the division of feature channels into two segments. One part directly forms a larger residual branch through skip connections, becoming an integral part of the final output feature map. The other segment undergoes pooling operations and contributes to a feature pyramid structure comprising four differently sized soft pooling layers. This process maps high-level feature information to low-level feature maps. Finally, it is cascaded and fused with high-level semantic information that has not undergone pooling, preserving global information to the maximum extent. This approach helps avoid performance loss caused by the limited feature representation capability of deep layers in the network.

2.4. Loss Function

The loss function plays a crucial role in deep learning models, quantifying the disparity between predicted values and true labels [58]. Yolov4-tiny’s loss function includes bounding box (L_CIoU), confidence (L_conf), and class components (L_class). The detailed process is shown in Equations (1)–(7).

$$L_{\mathrm{l}\mathrm{o}\mathrm{s}\mathrm{s}}=L_{\mathrm{C}\mathrm{I}\mathrm{o}\mathrm{U}}+L_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}}+L_{\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}}$$

(1)

$$L_{\mathrm{C}\mathrm{I}\mathrm{o}\mathrm{U}}=\sum _{\mathrm{i}=0}^{\mathrm{S}\times \mathrm{S}}\sum _{\mathrm{j}=0}^{\mathrm{M}}{I}_{\mathrm{i}\mathrm{j}}^{\mathrm{o}\mathrm{b}\mathrm{j}}(2-w_{\mathrm{i}}\times h_{\mathrm{i}})(1-A_{\mathrm{C}\mathrm{I}\mathrm{o}\mathrm{U}})$$

(2)

$$A_{\mathrm{C}\mathrm{I}\mathrm{o}\mathrm{U}}=1-A_{\mathrm{I}\mathrm{o}\mathrm{U}}+\frac{{\rho }^2(b,\widehat{b})}{c^2}+\alpha v$$

(3)

$$\alpha =\frac{v}{1-A_{\mathrm{I}\mathrm{o}\mathrm{U}}+v}$$

(4)

$$v=\frac{4}{{\pi }^2}{(\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{t}\mathrm{a}\mathrm{n}\frac{\widehat{w}}{\widehat{h}}-\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{t}\mathrm{a}\mathrm{n}\frac{w}{h})}^2$$

(5)

$$\begin{gathered} L_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}}=-\sum _{\mathrm{i}=0}^{\mathrm{S}\times \mathrm{S}}\sum _{\mathrm{j}=0}^{\mathrm{M}}{I}_{\mathrm{i}\mathrm{j}}^{\mathrm{o}\mathrm{b}\mathrm{j}}\left[\widehat{C_{\mathrm{i}}}\log \left(C_{\mathrm{i}}\right)+\left(1-\widehat{C_{\mathrm{i}}}\right)\log \left(1-C_{\mathrm{i}}\right)\right] \\ -{\lambda }_{\mathrm{n}\mathrm{o}\mathrm{o}\mathrm{b}\mathrm{j}}\sum _{\mathrm{i}=0}^{\mathrm{S}\times \mathrm{S}}\sum _{\mathrm{j}=0}^{\mathrm{M}}{I}_{\mathrm{i}\mathrm{j}}^{\mathrm{n}\mathrm{o}\mathrm{o}\mathrm{b}\mathrm{j}}\left[\widehat{C_{\mathrm{i}}}\log \left(C_{\mathrm{i}}\right)+\left(1-\widehat{C_{\mathrm{i}}}\right)\log \left(1-C_{\mathrm{i}}\right)\right] \end{gathered}$$

(6)

$$L_{\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}}=-\sum _{\mathrm{i}=0}^{\mathrm{S}\times \mathrm{S}}\sum _{\mathrm{j}=0}^{\mathrm{M}}{I}_{\mathrm{i}\mathrm{j}}^{\mathrm{o}\mathrm{b}\mathrm{j}}\sum _{\mathrm{c}\in \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{e}\mathrm{s}}[\widehat{p_{\mathrm{i}}}\left(c\right)\log \left(p_{\mathrm{i}}\left(c\right)\right)+(1-\widehat{p_{\mathrm{i}}}\left(c\right)\left)\log \left(1-p_{\mathrm{i}}\left(c\right)\right)\right]$$

(7)

For FEMSFNet, focusing on SAR image aircraft single-object detection with a single category output, the class loss function is omitted for computational simplicity. In object detection, the imbalance between positive and negative samples, where most pixels are background, is addressed by introducing the focal loss function in the confidence component of the loss function. This helps the model handle class imbalance and concentrate on challenging examples. To expedite model convergence and prevent overfitting, L2 regularization [59] is incorporated into the loss function for weight decay of model parameters. Here, $\phi$ represents the regularization coefficient. The detailed process is shown in Equations (8)–(11).

$${Loss}_{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}=L_{\mathrm{C}\mathrm{I}\mathrm{o}\mathrm{U}}+L_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}+\mathrm{f}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}}+\phi \sum _{i=0}^n{\omega }^2$$

(8)

$$L_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}+\mathrm{f}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}}=L_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}}+{\lambda L}_{\mathrm{f}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}}4$$

(9)

$$\begin{gathered} L_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}}=-\sum _{i=0}^{S\times S}\sum _{j=0}^M{I}_{\mathrm{i}\mathrm{j}}^{\mathrm{o}\mathrm{b}\mathrm{j}}\left[\widehat{C_{\mathrm{i}}}\log \left(C_{\mathrm{i}}\right)+\left(1-\widehat{C_{\mathrm{i}}}\right)\log \left(1-C_{\mathrm{i}}\right)\right] \\ -{\lambda }_{\mathrm{n}\mathrm{o}\mathrm{o}\mathrm{b}\mathrm{j}}\sum _{i=0}^{S\times S}\sum _{j=0}^M{I}_{\mathrm{i}\mathrm{j}}^{\mathrm{n}\mathrm{o}\mathrm{o}\mathrm{b}\mathrm{j}}\left[\widehat{C_{\mathrm{i}}}\mathit{log}\left(C_{\mathrm{i}}\right)+\left(1-\widehat{C_{\mathrm{i}}}\right)\mathit{log}\left(1-C_{\mathrm{i}}\right)\right] \end{gathered}$$

(10)

$$L_{\mathrm{f}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}}=-{\left(1-{\rho }_{\mathrm{t}}\right)}^{\gamma }\mathrm{l}\mathrm{o}\mathrm{g}\left({\rho }_{\mathrm{t}}\right)4$$

(11)

where $I_{\mathrm{i}\mathrm{j}}^{\mathrm{o}\mathrm{b}\mathrm{j}}$ is used to determine whether there is a target object in the network—if yes, it is set to 1; otherwise, it is set to 0; $A_{\mathrm{I}\mathrm{o}\mathrm{U}}$ represents the intersection over union ratio of the area between the true box and the predicted box; ${\rho }^2(b,\widehat{b})$ is the Euclidean distance between the center points of the predicted and true boxes; $c$ is the diagonal distance of the minimum closed region between the true box and the predicted box; $\widehat{w}, \widehat{h}$ and $w, h$ represent the width and height of the true and predicted boxes, respectively; $\widehat{C_{\mathrm{i}}}$ and $C_{\mathrm{i}}$ are the confidences of the true and predicted samples; $I_{\mathrm{i}\mathrm{j}}^{\mathrm{n}\mathrm{o}\mathrm{o}\mathrm{b}\mathrm{j}}$ is the inverse of $I_{\mathrm{i}\mathrm{j}}^{\mathrm{o}\mathrm{b}\mathrm{j}}$—it is set to 0 if there is an object in the grid, and 1 if there is no object; $\lambda$ is an adjustment parameter used to balance the importance of two losses; ${\rho }_{\mathrm{t}}$ is the model’s probability of a sample being positive; $\gamma$ is the parameter controlling attention.

3. Experiments and Results

3.1. Datasets

Currently, there is a scarcity of publicly available datasets for SAR image aircraft detection. Consequently, we exclusively utilize the SAR Aircraft Detection Dataset (SADD) [60] to validate our approach.

SADD is derived from the German Terra-SAR-X satellite, operating in the x-band and HH polarization mode. It provides image resolutions ranging from 0.5 to 3 m. Expert SAR Automatic Target Recognition (ATR) analysts manually annotate the ground truth of aircraft based on prior knowledge and corresponding optical images. After cropping large images, the SADD comprises 2966 non-overlapping 224 × 224 slices, containing 7835 annotated aircraft targets with clear structures, outlines, and main components. The dataset includes aircraft targets of varying sizes, with a significant number being small-scale targets. The SADD backdrop features a complex environment with diverse scenes such as airport runways, aprons, and civil aviation facilities. Negative samples are predominantly found in areas surrounding the airport, including open spaces and forests. Refer to Figure 7 for visual representations of sample images within the SADD.

In this article, to validate our method and enable comparison with other relevant papers using the same dataset, we randomly divide the SADD images into the training and test sets at a 5:1 ratio [60]. The training set comprises 799 positive samples and 1673 negative samples, totaling 6948 annotated aircraft boxes. The test set includes 85 positive samples, 409 negative samples, and a total of 887 annotated aircraft boxes, as summarized in

Table 1. Division of the dataset.

Dataset	Positive Samples	Negative Samples	Ground Truth
Train	799	1673	6974
Test	85	409	861

.

Figure 8 showcases sample images from the SADD. The images exhibit notable variations in the size of aircraft targets, and the backgrounds surrounding specific aircraft targets are particularly intricate, presenting challenges for accurate aircraft positioning. Furthermore, the intricate background clutter points may be misleadingly identified as aircraft components, further complicating the detection process.

3.2. Evaluation Metrics

In SAR aircraft detection, precision rate and recall rate serve as common evaluation criteria. Yet, there is often a trade-off between precision and recall, meaning enhancing one may reduce the other. To mitigate this, we introduce the F1 score as a supplementary metric, providing a holistic indicator that balances accuracy and recall. The calculation formulas are detailed in Equations (12)–(14):

$$Precision=\frac{TP}{TP+FP}$$

(12)

$$Recall=\frac{TP}{TP+FN}$$

(13)

$$F1=\frac{2\times Precision\times Recall}{Precision+Recall}$$

(14)

where the meaning of TP, FN, and FP are as shown in

Table 2. The meaning of TP, FN, and FP.

Prediction	Positive	Negative
Label
Positive	TP	FN
Negative	FP	TN

.

3.3. Experimental Setup

FEMSFNet utilizes a 224 × 224 image resolution for both training and testing phases. During the training stage, random rotation, random mixup, and mosaic data-enhancement methods are employed. All these methods are validated on an NVIDIA 4060Ti GPU. The configuration details of experimental parameters are outlined in

Table 3. Hyperparameter settings during model training.

Hyperparameters	Value
Optimizer	Adam
Learning rate	0.001
Learning decay method	Cosine Annealing
Batch size	128
Momentum	0.937
Worker number	4

.

3.4. Comparison to Existing Algorithms

To showcase the efficacy of FEMSFNet, we visualized its detection confidence maps in comparison to the baseline Yolov4-tiny, depicted in Figure 9. FEMSFNet demonstrates superior accuracy in locating aircraft targets and effectively suppressing background clutter interference, resulting in agiler detection confidence compared to the baseline Yolov4-tiny.

To rigorously validate our algorithm, we compare it against two two-stage methods (Faster R-CNN [61] and Cascade R-CNN [62]) and four one-stage methods (SSD [63], Yolov3 [64], SEFEPNet [60], and Yolov4-tiny [36]), as outlined in

Table 4. Comparisons to the state-of-the-art models.

Indicator	P	R	F1	Model Size (MB)
Model
faster R-CNN	0.86	0.89	0.87	160
cascade R-CNN	0.90	0.95	0.92	319
SSD	0.84	0.89	0.86	96
Yolov3	0.83	0.97	0.89	236
Yolov4-tiny	0.82	0.85	0.83	19
SEFEPNet 1	0.89	0.98	0.93	$\times$
FEMSFNet (ours)	0.92	0.96	0.94	23

¹ The data come from the original article [60], and there are no data about the SEFEPNet’s model size.

. To specify, the model complexities are as follows: Faster R-CNN at 160MB, Cascade R-CNN at 319 MB, SSD at 96 MB, Yolov3 at 236 MB, Yolov4-tiny at 19 MB, and FEMSFNet at 23 MB. The results demonstrate that FEMSFNet excels across various metrics, particularly in Precision and Model Size, with a maximum improvement of 92.7% in model size (compared to cascade R-CNN), 12.2% in precision, 12.9% in recall, and 13.3% in F1 score compared to the contrasted algorithms. This success is attributed to enriching dataset diversity through image enhancement, incorporating attention modules in the residual structure, and integrating pyramid modules in the CSP network within the FEMSFNet architecture. These components enable FEMSFNet to capture deeper, broader, and more accurate target information. Figure 10 illustrates the detection results of FEMSFNet and other methods. In support of further research in Synthetic Aperture Radar (SAR) aircraft target detection, we plan to open-source FEMSFNet soon at https://github.com/WenboEth/Sar-Aircaft-Target-Detection, accessed on 10 May 2024.

The results below highlight that while FEMSFNet may make mistakes in complex target recognition scenarios, it consistently outperforms the compared algorithms, especially in terms of precision, recall, and model size. We anticipate that FEMSFNet has untapped potential, and with further optimization, broader expansion, and deeper evolution, it can achieve even better results in the future.

In the realm of deep learning-based object detection, assessing a network’s generalization ability is crucial. A network model with robust generalization performance excels on unseen, complex, or even distorted data, serving as a key metric of the model’s practical utility. The FEMSFNet network enhances its training data through methods including rotation, scaling, and cropping, aiding the model in learning a broader range of variations to boost generalization performance. Moreover, techniques like dropout are utilized within the SdE-Resblock module, randomly “dropping” a portion of neurons during training to decrease the model’s dependency on specific data and enhance its generalization capability. To validate the generalization performance of the FEMSFNet network, a non-learned dataset (n-LD) was created. This dataset consists of targets the network has never “learned” from, not included in training, validation, or test sets. To challenge the network’s generalization and robustness, the n-LD was complicated and distorted, increasing the difficulty of target recognition. Examples of the n-LD cases are shown in Figure 11. The n-LD was then fed into both the FEMSFNet and state-of-the-art models for comparative testing and validation of FEMSFNet’s generalization ability, with results presented in

Table 5. Generalization performance comparison.

Model	Faster R-CNN	Cascade R-CNN	SSD	Yolov3	Yolov4-Tiny	FEMSFNet
Indicator
Accuracy Rate	0.91	0.93	0.89	0.88	0.85	0.94
Error Rate	0.04	0.04	0.06	0.04	0.06	0.02
Omission Rate	0.05	0.03	0.05	0.08	0.08	0.04

. The charts reveal that FEMSFNet performs exceptionally well across various metrics, including accuracy, error rate, and omission rate. Such outcomes are attributable to the superior SdE-Resblock module and the ssppf-CSP structure of FEMSFNet.

3.5. Ablation Study

To enhance readers’ understanding of each module’s impact, we conducted an ablation study. Yolov4-tiny serves as the base framework for evaluating the effects of various feature modules on aircraft detection in SAR images, as outlined in

Table 6. Ablation study.

	ssppf-CSP	SdE-Resblock	Loss Function	P	R	F1
(a)	×	×	×	0.86	0.90	0.88
(b)	√	×	×	0.90	0.93	0.91
(c)	√	√	×	0.92	0.95	0.93
(d)	√	√	√	0.92	0.96	0.94

. In the ablation comparison experiments, the “×” mark in the loss function column indicates the use of the pre-improvement Yolov4-tiny loss function, detailed in Equations (1)–(7). The “√” mark indicates the use of the improved loss function, as detailed in Equations (8)–(11). Simultaneously, Figure 12 presents the results of the ablation study conducted on the SADD dataset. The charts illustrate sequential application of different modules leading to improvements across various metrics. Precision sees the highest enhancement, with a 7.0% increase, while recall and F1 score exhibit improvements of 6.9% and 6.8%, respectively. It highlights that the ssppf-CSP module, integrating a soft feature pyramid into the CSP architecture, enhances network depth and incorporates features from multiple scales. This results in a substantial improvement in overall accuracy and recall metrics. The SdE-Resblock module, integrating an optimized attention mechanism into the residual structure, enhances the network’s focus on target detection, leading to improvements in recognition rate and recall. While the improved loss function module may not show a significant increase in evaluation metrics, its main impact lies in accelerating convergence and reducing computational complexity. As illustrated in Figure 13, we have compared the iteration process of the loss function before and after improvements. It is evident from the figure that the improved loss function demonstrates an overall reduction in loss values compared to its prior state, along with a notably faster convergence rate. Furthermore, when comparing the same iteration round, there is a maximum enhancement of 26% in the loss value.

3.6. Result and Discussion

From the comparative validation analysis presented earlier, it is clear that in the domain of aircraft target detection in remote sensing SAR imagery, the FEMSFNet model excels across several metrics, including detection precision, recall rate, F1 score, and model size, achieving varying degrees of improvement. Specifically, it surpasses other algorithms, with a maximum increase of 12.2% in precision, 12.9% in recall, and 13.3% in F1 score. Moreover, in terms of model size, FEMSFNet achieves an 88% reduction compared to the largest model size benchmarked. The primary reason for this performance is that, unlike general-purpose state-of-the-art neural networks, FEMSFNet is a unique network specifically tailored for SAR imagery of aircraft targets. It eliminates unnecessary components for this task, such as the Region Proposal Network and multiple cascading detection heads, refining the network architecture. Initially, to address the challenges of limited remote sensing SAR image data, complex backgrounds due to noise, and various obstructions making target detection difficult, FEMSFNet incorporates a data augmentation module. This module complexifies or even distorts the source image data, tackling the issue of data scarcity while training the network to enhance its generalization capability and robustness with complicated images. Furthermore, addressing the diversity in target scales and complex angles in SAR imagery, FEMSFNet introduces the ssppf-CSP and SdE-Resblock modules. Optimized residual modules are employed to extract more feature information, and multi-scale fusion modules ensure the detection and recognition of targets across various scales, thereby improving the network’s accuracy. Lastly, in the specialized task of single-object detection for aircraft in SAR images, focusing on a singular category output, the class loss function is omitted to simplify computations. This approach addresses the imbalance between positive and negative samples—mostly background—by incorporating the focal loss function into the confidence measure of loss calculation. This modification significantly enhances the model’s capability to manage class imbalances and prioritize complex examples. Moreover, to ensure faster convergence and prevent overfitting, L2 regularization is integrated into the loss function, facilitating the weight decay of model parameters.

4. Conclusions

This paper introduces FEMSFNet, an SAR aircraft detection model prioritizing both speed and accuracy. FEMSFNet utilizes the lightweight CSPDarknet53-tiny as its backbone for efficient feature extraction. To maintain a lightweight yet accurate model, we integrate the optimized SE attention module with the ResNet module, forming the SdE-Resblock structure. A novel CSP structure, ssppf-CSP, prevents deviations in the receptive field during training, enhancing global feature fusion. Addressing unique characteristics in SAR aircraft target detection, FEMSFNet optimizes loss functions and employs techniques like learning rate cosine annealing decay and label smoothing to prevent overfitting, improving convergence speed and regression accuracy. For increased sample diversity, FEMSFNet employs multi-faceted image augmentation with techniques like noise addition, mosaic, mixup, rotation, and cropping, enhancing training generalization and robustness. Experiments on the SADD dataset demonstrate FEMSFNet’s effectiveness, surpassing state-of-the-art object-detection algorithms. FEMSFNet exhibits significant improvements compared to the contrasted algorithms in terms of precision rate (92%), recall rate (96%), and F1 score (94%). Notably, it surpasses contrasted algorithms with a maximum increase of 12.2% in precision, 12.9% in recall, and 13.3% in F1 score. Anticipating untapped potential, further optimization, broader expansion, and deeper evolution are expected to propel FEMSFNet to even better results in the future.