1. Introduction
Non-cooperative space targets usually refer to malfunctioning satellites and space debris without cooperative interfaces, artificial labels, and information communication capabilities [
1]. With the increase in space accidents and malfunctioned spacecrafts, many space non-cooperative targets will occupy orbit resources and threaten the safety of spacecrafts in orbit. Because of their non-cooperative nature, non-cooperative target on-orbit recognition technology has become a complex problem in space situational awareness (SSA) [
2,
3].
Compared with expensive and heavy LIDAR systems, optical cameras provide an economical alternative with less mass, power requirement, and geometric constraints [
4]. As a result, the research of a non-cooperative space target recognition algorithm based on vision has a broader application prospect. With the development of convolutional neural networks, image classification and recognition based on deep learning has made a breakthrough in the field of natural images, producing ResNet series [
5,
6], YOLO series [
7,
8,
9], Faster R-CNN [
10], and other mainstream deep learning models. Deep learning methods can learn target features autonomously and are more robust than traditional methods. Thus, current space non-cooperative target recognition algorithms are shifting from conventional to deep learning [
11].
However, the space non-cooperative target recognition task is different from the natural image recognition task. On the one hand is the problem of the dataset. The lack of datasets has consistently been a significant difficulty in space situational awareness (SSA) since it is rare, costly, and sensitive to obtain realistic on-orbit space surveillance images. Published datasets are mainly aimed at spacecraft classification tasks [
12,
13], component identification, and pose estimation tasks [
14], and there are no publicly available datasets for the multi-target detection task of distinguishing between satellites and debris in the space environment [
15]. While the SPARK dataset [
16] is suitable for target detection missions, including ten satellites and five space debris, the scene pictures contain only single targets of satellites or debris. On the other hand, the problem of on-orbit deployment. Although YOLO and R-CNN series models belong to the end-to-end high-performance target detection models, they are large in scale and arithmetic demand. Under the constraints of arithmetic and memory resources of onboard hardware platforms [
17], it is difficult for them to realize accurate and efficient on-orbit recognition.
In addition, in terms of observation modes, with the development of satellite formation technology, multi-satellite cooperative observation has been gradually applied to space surveillance, space capture [
18,
19], and other tasks because it can obtain more comprehensive wide-area space images. Typically, such as the autonomous maneuvering space flying net capture system [
20,
21], as shown in the
Figure 1. The satellite formation carries the photoelectric camera, and based on the multi-satellite cooperative observation, it can obtain a high-resolution large field-of-view space image and improve the scope and efficiency of deep space exploration. Unfortunately, to the best of our knowledge, most current studies employ single spacecraft to monitor targets, and there are a lack of research results on multi-satellite cooperative observation for non-cooperative targets.
To address the above issues, this paper proposes a space non-cooperative target recognition method that offers an extensive monitoring range and higher detection efficiency compared to single-satellite systems. Firstly, we employ the idea of feature matching to realize the stitching of multi-satellite images and generate smooth, wide-area space images. Secondly, we perform target localization and recognition on the generated wide-area space images. In this part, considering the need for lightweight and fast inference of models for on-orbit recognition, inspired by the idea of two-stage recognition, we propose a lighter CNN model with a distance merge threshold (termed as LCNN-DMT). This model first performs foreground extraction from the scene to locate and extract all the suspicious targets and then sends each target region into a CNN classification network to distinguish between satellites and debris. In conclusion, we propose a simpler, lighter, and more hardware-friendly method for non-cooperative multi-target recognition within multi-satellite cooperative observation while maintaining high recognition accuracy and speed.
The contributions of this paper are mainly as follows:
- (1)
This paper proposes a space non-cooperative target recognition method, including wide-area image generation, as well as target recognition and localization.
- (2)
A public space target dataset and a multi-target space scene dataset are constructed, encompassing 15 types of satellites and 39 types of debris. In particular, a randomized augmentation strategy is proposed by randomly assigning the parameters and transformation combinations of the samples.
- (3)
Inspired by the minimum bounding rectangle with threshold (MBRT) [
22,
23], we propose a new foreground extraction model based on a minimum bounding rectangle with threshold for distance merging (MBRT-D) to achieve high-precision target localization and extraction. We replace the IOU threshold with a bounding box distance merge threshold, effectively addressing the issue of redundant extraction of satellite bodies and solar wing parts.
- (4)
We propose an improved lightweight SqueezeNet model, which integrates separable convolutions [
24] to reduce the model’s parameter count and computational demands. Furthermore, the incorporation of the efficient channel attention (ECA) [
25] mechanism significantly enhances the model’s capacity for inter-channel information interaction, thereby improving its recognition performance.
- (5)
The proposed LCNN-DMT adopts the two-stage idea of first positioning and then identification by combining MBRT-D with the improved SqueezeNet model. This approach not only reduces the data labeling time but also maintains extremely low parameter counts and computational requirements.
The remainder of this paper is organized as follows. In
Section 2, we briefly discuss some research work related to ours.
Section 3 presents the space target dataset, scene dataset preparation, and augmentation methods.
Section 4 details our proposed multi-satellite non-cooperative target recognition method, including multi-satellite image stitching and LCNN-DMT model design. Extensive experiments are conducted in
Section 5, with our work and future outlook presented in
Section 6 and
Section 7.
3. Datasets
Since the space non-cooperative target recognition model proposed in this paper adopts the two-stage idea of foreground extraction followed by classification, it is necessary to construct a space target dataset containing failed satellites and debris for the training of the classification network and a space scene dataset for the testing of the overall model recognition capability. Considering the variation of illumination angle and brightness of the natural space environment, we propose a randomized data augmentation strategy to improve the data richness.
3.1. Space Target Dataset
The space target dataset constructed in this paper contains two types of targets: malfunctioning satellites and space debris. We obtain satellite models from publicly accessible online resources and the BUAA-SID-share1.0 public dataset. The BUAA dataset has a total of 4600 grayscale images with a resolution of 320 × 230, including 20 types of satellite targets, each containing 230 multi-angle attitude images generated by 3D MAX software, which provides data support for the space target classification task. Since space debris often accompanies satellites in the real universe, we collect various space debris online, mainly parts of spacecraft or rockets.
The limited availability of publicly accessible space target images makes it challenging to meet the training requirements of deep learning models. To overcome the problem of poor generalization ability caused by small sample datasets, this paper proposes a randomized data augmentation strategy to achieve target dataset augmentation.
We use label-preserving transformations for data enhancement, including horizontal flipping, vertical flipping, random rotation, random cropping, and other geometric transformations. In addition, considering that the target’s luminance and background change with varying angles of light source incidence, we randomly add varying degrees of brightness changes based on contrast enhancement and Gamma correction to prevent the overfitting of the recognition model in the uneven illumination environment.
Instead of using only one transform or a fixed order of multiple transforms, we apply randomly generated combinations and coefficients of transforms to each image. Due to the random transformations, such as rotation angle, cropping scale, and Gamma coefficients, the uncertainty of the generated image samples is higher, improving the whole dataset’s sample richness. This will enhance the model’s recognition ability for non-cooperative targets.
Figure 2 illustrates the effect of some of the data transformations in this paper.
3.2. Space Scene Dataset
According to the characteristics of space scenes with wide frames and small target pixels, considering the pixel values of satellite cameras in recent years, the resolution of our simulated images of space scenes is unified as 2048 × 2048. Considering the effect of illumination during real detection, we perform a luminance transformation on a pure black background. The targets within the scene are randomly taken from the above-established space target dataset and randomly placed in the background. The pixels of the targets are set to 0.6 times, 0.75 times, and 0.9 times resolution (320, 240), corresponding to the different stages of approaching the target during the detection or capture tasks. The simulated images in our space scene dataset are shown in
Figure 3. In the end, a total of 2900 space scene images were synthesized and marked, containing all target types in the space target dataset. Compared with the publicly available space datasets containing only a single spacecraft or space debris target, our scenes are equipped with satellites and debris multi-targets, which are closer to the actual space environment.
5. Experiments
5.1. Implement Details
Our experiments are all performed on the Windows 11 operating system, and the programming environment incorporates Pycharm IDE with Python 3.8, Pytorch 1.8, and CUDA 11.1. The computing resources are Intel Core i7-13700KF CPU and NVIDIA RTX 4090 GPU.
In order to fairly compare the recognition algorithms and show the outstanding of our method, the datasets are set up as follows in the experiments. The space target dataset contains 17,000 samples containing 12,500 satellite targets and 4500 debris targets of varying sizes. The ratio of the training and testing set is 8:2 for the training and testing of the binary classification network, and the target image size is standardized to 320 × 240 when sent to the network. In the space scene dataset, the total number of samples is 2900 with a resolution of 2048 × 2048, of which 2000 are used as the training set while 900 are regarded as the test set. The scene images are randomly placed with different kinds of satellites and debris in our established space target dataset, which contains a total of 1649 satellite targets and 1031 debris targets in the test set. To compare the recognition performance of different algorithms on space non-cooperative targets in natural scenes, in the two datasets above, the targets in the test set are almost non-cooperative, i.e., their configurations have not appeared in the training set.
5.2. Image Stitching Results
To measure the performance of our algorithm, we compare the image registration algorithm presented in this paper, which is based on ORB features, with other classical feature extractors [
27,
29,
30,
31]. Uniformly, the KNN method is used for feature matching, the PROSAC algorithm for matching point correction, and the weighted fusion method for image fusion. We apply the different feature registration algorithms to 10 sets of simulated satellite images to be stitched and compare their correct feature point matching pairs (
), root mean squared error (RMSE), precision, and stitching speed. In order to evaluate the impact of different sensor imaging differences on the stitching performance, we have designed a transformation for each set of image pairs. Specifically, rotation transforms, luminance transforms, and Gaussian noise are added to simulate different viewing angles, illumination, and signal-to-noise ratio (SNR) situations, respectively. The RMSE calculates the deviation between the reference image and the matched image. The precision refers to the ratio of the number of correct feature point matches to the number of all matches after matching point correction and is defined as follows:
where
denotes the number of correct feature point matching pairs after matching correction, and
N denotes the number of all feature matches after matching correction. Stitching speed refers to the time from the start of feature extraction to the end of image fusion between two images.
As can be observed in
Table 1, since the space background is simpler than the ground background, these feature matching algorithms show high robustness even for image pairs with differences in imaging, and the matching algorithms based on SIFT, AKAZE, and ORB features can reach up to 100% precision after PROSAC correction. In addition, regardless of which feature descriptor is used, the feature point matching correction method based on PROSAC outperforms RANSAC in terms of precision and RMSE and successfully eliminates the redundant matching pairs of the RANSAC algorithm so as to reduce the computational burden. Among them, the SIFT+PROSAC algorithm has the lowest matching error. However, the ORB+PROSAC algorithm we introduce achieves a larger number of correct matched points and reduces the image matching speed by nearly 90 milliseconds, with only a 0.137 unit increase in RMSE compared to the SIFT+PROSAC algorithm.
Figure 12 presents the matching and stitching results of the representative three-treatment alignment images, joining various degrees of rotation transformation, luminance transformations, and Gaussian noise interference. It can be seen that the PROSAC algorithm can obviously eliminate the wrong feature pairs after the initial alignment of KNN. Meanwhile, when there are imaging deviations in different sensors, our ORB+PROSAC stitching algorithm still works well and realizes smooth transition and seamless splicing of multi-satellite observation images in overlapping regions without ghosting. It proves the environmental adaptability of our stitching algorithm.
5.3. Randomized Data Augmentation Results
To verify the performance enhancement of the randomized data augmentation strategy on the target classification network, based on the constructed space target dataset, we compare the recognition accuracy of no augmentation strategy, fixed-order augmentation strategy, and randomized data augmentation strategy under the SqueezeNet model. To ensure the network recognition accuracy is not affected by the amount of training data, for the original dataset without augmentation strategy, we perform 10× replication of the samples without image geometry and color transformation. Finally, the space target dataset of this experiment contains 10,000 satellite samples and 4000 debris samples in the training set and 2500 satellite samples and 500 debris samples in the test set.
As shown in
Table 2, it can be seen that using the data augmentation strategy significantly improves the recognition accuracy of the network compared to using the original dataset. Among these strategies, due to adding the diversity and randomness of the samples, the randomized data augmentation strategy improves the recognition accuracy by 21.97% and 5.43% compared with the original dataset replica expansion strategy and the fixed order augmentation strategy, respectively. This experiment demonstrates the effectiveness of our strategy of randomized data augmentation.
5.4. Detection Performance of LCNN-DMT
5.4.1. Performance on Our Space Scence Dataset
Figure 13 demonstrates the space non-cooperative target detection results based on our proposed LCNN-DMT model. It can be seen that our model has multi-target identification and localization capabilities with high confidence. In practical application, after target detection, the three-dimensional position of the target can be solved based on the principle of the multi-camera positioning system. Further, the target position information can be provided to the control system to perform the approach capture.
5.4.2. Performance on Spark Dataset
The Spark dataset [
16] is a widely used dataset for space target detection nowadays. To demonstrate the generalization capability and the identification capacity for space non-cooperative targets of our model, we conduct detection experiments on the Spark dataset. The dataset has never been used during the model’s training or validation phases. As displayed in
Figure 14, our model successfully locates and identifies the satellite target containing earth background interference under the SPARK dataset. This proves the environmental adaptability of our model.
5.4.3. Noise Case
As we all know, Gaussian noise usually appears during space image acquisition and transmission. To test the anti-noise performance of our LCNN-DMT model, the recognition results under different degrees of Gaussian noise are presented in
Figure 15. The means (
) of all the noises are 0, and the standard deviations (
) range from 0 to 1. To evaluate image quality, we also calculate the signal-to-noise ratio (SNR) for images with varying levels of noise.
It is observed that image quality significantly decreases as the noise level increases. Our LCNN-DMT model can correctly extract and identify satellite and debris targets with high confidence when the SNR remains above −10 dB. However, when the SNR is below −12 dB, the model, despite being able to detect targets normally, is unable to accurately identify them due to the overwhelming noise proportion in the image. It should be noted that the extreme scenarios depicted in
Figure 15e–h, where the SNR is negative, are not commonly encountered in actual detection. Consequently, when considering practical applications, our model performs with commendable robustness.
5.5. Ablation Study
In this section, we conduct a series of ablation studies to systematically validate the contribution of each component within our proposed recognition approach.
5.5.1. Effectiveness of Foreground Extraction Model Performance
To demonstrate the advantages of the proposed MBRT-D foreground extraction algorithm,
Table 3 presents a comparison of recognition performance when different foreground extraction algorithms are applied to the SqueezeNet model. The experiments are based on test images from our space scene dataset. It is evident that our proposed MBRT-D foreground extraction algorithm has the best performance. The mAP has increased by 17.89% and 10.42% relative to the MBR and MBRT. This indicates that the proposed method of removing redundant detection boxes based on distance merge threshold has a better extraction effect and target localization.
As shown in
Figure 16, we compare the effect of different distance merge thresholds on the recognition performance, with a range of thresholds covering 0 to 250. When the threshold is 0, no merging is performed on any detection frames, the detection results at this point are consistent with the MBR strategy, with a poor recognition accuracy. Most of the targets in our test images are small in size, belonging to long-distance detection simulation images, which are more sensitive to distance. As a result, when the threshold is set small, the recognition accuracy is effectively improved, and the best recognition performance is achieved when the threshold is 26, with an accuracy of 97.06%. As we analyzed, with the distance threshold continuing to increase, the recognition accuracy declines. When the distance merge threshold is too large by more than 250 pixels, the recognition performance degrades to below the MBR strategy. The reason for this is that a distance merge threshold that is too large will cause other target detection boxes to be merged together, resulting in incorrect recognition and location. Experiments have proven that the distance merge threshold is related to the detection distance, and the threshold needs to be set in a reasonable interval according to the detection distance. Meanwhile, in the subsequent experiments, our distance merge threshold is set to 26.
Figure 17 demonstrates the effect of foreground extraction based on MBR, MBRT, and MBRT-D for the same image. The image background has been processed with brightness change, and salt-and-pepper noise is added randomly to each image. We introduce OTSU [
44] to realize adaptive threshold segmentation in the three algorithms to adapt to the brightness variation of the environment. It can be seen that based on our MBRT-D, the redundant neighboring detection boxes after MBRT are merged. This indicates that MBRT-D reduces the redundancy of detection boxes and improves target localization accuracy with a certain degree of anti-noise ability.
5.5.2. Effectiveness of INFire Module
In this paper, we design the INFire module with parallel 3 × 1 and 1 × 3 convolutions instead of 3 × 3 convolutions. In order to check the impact of our designed INFire module, we replace the Fire module at different locations in the SqueezeNet with the INFire module, training and testing the model based on our space target dataset. As seen in
Table 4, the spatially separable convolutions idea in the INFire module can effectively reduce the number of model parameters and computations, and the more INFire modules are set, the lighter the model is. Compared with replacing all the Fire modules or replacing Fire4-Fire5, the model accuracy is highest when replacing the four positions of Fire6-Fire9, which can reach 98.23%, and the model can be compressed to 75.14% of the base SqueezeNet model.
5.5.3. Effectiveness of Attention Mechanism
To verify the effects of the ECA attentional mechanism in our optimized SqueezeNet. We set the ECA module at the end of the Fire module in different layers, and
Figure 18 shows the effect of the ECA module and its position on the recognition accuracy. It can be seen that the introduction of the ECA mechanism after the Fire module at most locations can improve the recognition performance.
However, the original recognition ability may be weakened in case of improper location selection. By adding the ECA mechanism after Fire9, the recognition accuracy is reduced from 98% to 97.63% due to the backward processing position. When the ECA mechanism is added sequentially after Fire2-Fire9, the recognition accuracy is reduced by 0.33%, probably because the Fire9 layer combined with the ECA mechanism has a greater impact on the overall model. After experimentation, the model recognition accuracy is highest when the ECA module is embedded behind both Fire3 and Fire7, up to 98.47%. In addition, the ECA attention mechanism improves the model recognition accuracy while basically unchanging the amount of computation and the number of parameters.
5.5.4. Effectiveness of Improved Squeezenet
After confirming the effectiveness of the INFire and ECA modules, we introduce the combination of both into SqueezeNet. In particular, based on the experimental results of the single module, we use the INFire module to replace the Fire6-Fire9 module in the original model to achieve the light weight of the model. We deploy the ECA module after the Concat operation of the Fire3 and INFire7 modules for channel information fusion. According to
Table 5, our improvement increases the recognition accuracy to 98.83% while reducing the number of model parameters by 24.86% and computation by 15.02%. It proves the positive effect of INFire module and ECA module on the model recognition performance.
5.6. Comparative Experiments
5.6.1. Comparison of Classification Model Performance
To represent the excellent performance of our improved classification model based on SqueezeNet, we compare it with the LeNet-5 model used in the article [
38], the ResNet-50 model used in the article [
22,
23], and other state-of-the-art (SOTA) models in recent years [
6,
45,
46,
47,
48]. The comparison experiments are based on our space-targeted dataset, and the training epochs are uniformly set to 32. As presented in
Figure 19, due to the sufficient amount of training samples after data augmentation, all models’ recognition accuracies are higher than 97%, except for the LeNet-5 model, which has a shallower number of layers. Although the LeNet-5 model performs the fastest inference with the least amount of computation, since this model employs the fully connected layers, resulting in an overly large number of parameters and model size.
The comparison results show that our model with the introduction of separable convolution and the ECA attention mechanism is superior to the basic SqueezeNet model in terms of both accuracy and the number of parameters and computations at the cost of a speed delay of no more than 1 ms. Among the models, our model has the highest recognition accuracy and the smallest number of parameters, which makes it more lightweight and hardware-friendly for on-orbit recognition tasks. In addition, although MobileNetv3 and ShuffeNetv2 also have outstanding recognition performance with lower computing power requirements, our model inference speed is only 44.83% and 56.52% of theirs, respectively. Therefore, the performance of our improved classification model has superior recognition performance compared to the current SOTA model, proving the effectiveness of the improved module.
5.6.2. Comparison of Recognition Method Performance
For further clarification about the superiority of our LCNN-DMT model, five recent SOTA models are used for comparisons, including lightweight models of the YOLO series [
7,
8,
9], Faster R-CNN [
10], and our benchmark model T-SCNN [
22]. All experiments are uniformly conducted under our space scene dataset. The YOLO series and faster R-CNN are trained for 300 epochs with 16 batch sizes.
The results are illustrated in
Table 6. Obviously, our LCNN-DMT model is lightweight enough, even though the lightest of the mainstream models, the YOLO v5n model, has 1.9 times the number of parameters and 9.1 times the amount of computation more than our model. Moreover, our LCNN-DMT model is lightweight while meeting space detection’s accuracy and speed requirements. The mAP of our model reaches 97.50%, which is only slightly lower than YOLO v8n, but the average inference speed of a single frame is improved by 17.6% compared to YOLO v8n and is much faster than Faster R-CNN.
In practical detection, targets in space tend to be sparse. Our model uses the two-stage strategy, so the image inference time mainly depends on the number of targets in the scene. It means that when the number of targets in the scene is less than three, our model will have a significant speed advantage, with an average single-target inference time of only 2 ms. In addition, considering the resource limitations of the onboard computing platform, it is usually required to recognize the wide-area space images in blocks after segmentation. Mainstream models such as YOLO and faster R-CNN need to be inferred for each sub-image, which will waste a lot of time in the blank area. In contrast, this paper adopts image processing methods to search the target area first quickly, and only the sub-images containing the target are processed, which has a higher on-orbit recognition efficiency.
On the other hand, our LCNN-DMT only needs the classification model to be trained, which saves a lot of marking and training time; the training time for network convergence is only 15 min. Compared with YOLO v5n and faster R-CNN, the training time is saved 6 times and 168 times, respectively.
Finally, compared to our baseline model, T-SCNN, our LCNN-DMT model has a notable performance improvement. This is due to our optimized foreground extraction algorithm design and redesigned the SqueezeNet model for classification. Specifically, the mAP increased by 10.86%, and the model size was sharply reduced by 96.3%, while the inference speed has been increased by 66.5%, which verifies the effectiveness and advantages of our proposed method once again.
5.6.3. Visualization Comparison
The visualization results of each detection model under the space scene dataset are shown in
Figure 20. Obviously, our LCNN-DMT model shows the best recognition performance for space non-cooperative targets and can even recognize targets that are difficult to judge by human eyes.
Particularly in
Figure 20a, due to the dark brightness of the debris target located at the top of the scene, it is misidentified as a satellite target by other mainstream recognition models except our LCNN-DMT model and T-SCNN model, highlighting the advantages of the strategy we adopt.
On the other hand, since the foreground extraction algorithm designed in this paper considers the distance merge threshold, there is almost no detection box redundancy in the visualization results of the LCNN-DMT compared to other models. As shown in
Figure 20a–d, both the YOLO series models and the T-SCNN model expose the problem of redundant detection of solar panels, which leads to misjudgment of the number of targets and localization errors. Although faster R-CNN does not have such a problem, the inference speed is too slow to be realistically applied to on-orbit recognition. Therefore, considering the recognition precision, speed, number of parameters, computation cost, and virtualization effect, our proposed recognition model, i.e., LCNN-DMT, has more outstanding recognition performance and is more suitable for on-orbit deployment.
6. Discussion
The experimental results demonstrate that our method can be applied to multi-satellite cooperative observation for missions such as space capture and space surveillance. It should be noted that our recognition model is not only applicable to spatial targets but also to conventional targets. Furthermore, the model is not limited to optical images; it can also be extended to synthetic aperture radar (SAR) images for the task of non-cooperative target detection.
As depicted in
Figure 21, we trained and tested our LCNN-DMT model based on the SAR-Ship dataset [
49]. Fortunately, our algorithm can successfully detect ship targets in SAR images. Moreover, our proposed foreground extraction algorithm successfully merged redundant extraction frames caused by pixel discontinuities, which are annotated with green ellipses in
Figure 21b,c.
However, our cooperative recognition algorithm has certain limitations. On the one hand, our algorithm is more suitable for fast processing of scenes with simple backgrounds, such as space or clean sea, while for interferences such as harbors or islands, as in
Figure 21c, our foreground extraction algorithm extracts them together. Although we have included the background as negative samples in our classification model to mitigate this issue, the recognition performance is likely to degrade when faced with excessively complex backgrounds.
On the other hand, in practical applications, the imaging differences between satellite detectors due to various viewing angles, light intensities, and SNR will directly affect the stitching and recognition accuracy.
Figure 22 shows the cooperative recognition result of our model for satellite images with different SNR. Due to the image preprocessing stages, including mean filtering and contrast enhancement adopted in our model, the quality difference between different images is reduced, and the space targets are successfully recognized by cooperative detection. However, more significant imaging differences pose a greater challenge to the model, which can adversely affect the recognition results.
In the future, we will consider utilizing super-resolution reconstruction techniques to enhance the algorithm’s robustness against variations in viewing angles, lighting conditions, and noise levels. Additionally, we will consider utilizing neural network hardware acceleration techniques to achieve the fast inference of our model on FPGA platforms and verify its feasibility for on-orbit deployment.
7. Conclusions
In contrast to terrestrial natural image detection tasks, space imagery is characterized by a simplistic background, large scale, low pixel, high noise levels, and dispersed targets with minimal pixel representation in long-range detection scenarios. Given the constraints on memory and computational resources of onboard hardware platforms, this paper has proposed a lightweight on-orbit space non-cooperative target recognition method with multi-satellite cooperation, aiming to achieve coordinated sensing across multiple satellites and reduce the parameter count and computational complexity of the recognition model. The input is the collected images from multiple satellites. Through the image stitching algorithm and recognition model proposed in this paper, the wide-area image is obtained and the target category and location information are output.
Concretely, we employed the feature-based image registration algorithm to realize high-precision stitching of multi-satellite images, taking into account the effects of perspective, illumination, and noise. Then, we proposed a novel space non-cooperative target recognition model, LCNN-DMT. For the foreground extraction step, we proposed the MBRT-D model, which solves the problem of redundant detection boxes by introducing a distance merge threshold so as to achieve high-precision location and extraction of suspicious targets in the scene. For the target classification stage, we introduced the spatially separable convolutions and ECA attention mechanism into the SqueezeNet model to obtain an improved model with higher accuracy and lighter weight. In addition, we constructed publicly available space target datasets and space scene datasets to provide convenience for other researchers to carry out space target classification and multi-target detection studies.
In conclusion, our method enables cooperative recognition for images obtained through multi-satellite observation, thereby enhancing both the field of view and detection efficiency. The LCNN-DMT recognition model we designed is sufficiently lightweight enough to mitigate the high computational demands of deep learning models, making it feasible for deployment on satellite-borne platforms. Moreover, compared to mainstream deep learning models such as YOLO and Faster R-CNN, our model achieves high recognition accuracy and inference efficiency without the need for large-scale scene datasets, thereby reducing the time required for labeling and training.