3.1. YOLOv8
In 2023, the Ultralytics team developed the next-generation, state-of-the-art model YOLOv8, building on the early YOLO series with multiple innovations to improve its performance and adaptability. YOLOv8’s versatility allows it to handle various computational tasks, ranging from object detection to instance segmentation, while being compatible with multiple hardware platforms, from CPUs to GPUs. YOLOv8 has been divided into several editions (n, s, m, l, and x) depending on the network architecture’s depth and breadth to accommodate varying application requirements. Exceptionally, YOLOv8n is crafted to be compact, which is ideal for usage in environments with limited resources.
The YOLOv8 architecture mainly comprises four elements: the input, the backbone for feature extraction, the neck for feature enhancement, and the head dedicated to detection tasks. The configuration of the model is depicted in
Figure 1.
In the input stage of YOLOv8, advanced data augmentation techniques, including Mosaic data augmentation, are employed. Mosaic augmentation boosts the model’s adaptability and generalization across diverse scales and backgrounds by randomly amalgamating various segments from multiple training imageries. Additionally, the model optimizes anchor box sizes and ratios through adaptive anchor technology, better matching the shapes of actual detection targets and improving detection accuracy. The anchor-free strategy simplifies the prediction process and increases processing speed by directly predicting objects’ center points and scales.
The backbone section focuses on feature extraction, with YOLOv8 incorporating several structural optimizations. The Conv module employs advanced convolutional networks to sequentially extract features from imageries while incorporating BatchNorm and activation functions at each level to augment the model’s capability for capturing non-linear characteristics. The C2f module integrates residual connections with extended convolutional operations, enhancing the transmission of features and addressing the issue of gradient disappearance often encountered in deep neural architectures. Simultaneously, the SPPF module utilizes multi-scale spatial pooling to amalgamate features across various scales, thereby boosting the model’s flexibility in adapting to varying object dimensions.
The neck section effectively fuses features extracted by the backbone. It uses FPN and PAN structures to optimize information flow between feature layers. FPN builds a top-down feature pyramid, combining high-level semantic information with low-level details. PAN enhances feature association through a bottom-up path, improving detail capture. These designs enable the neck to output high-quality feature maps rich in semantics and details.
The head section functions as the decision-making nucleus of the model and is tasked with the final classification and localization of objects. Its decoupled head configuration distinguishes classification and localization duties, tailoring the network architecture and parameters to enhance each function separately. This distinct separation results in more targeted and effective classification and localization. In terms of loss functions, YOLOv8 adopts a variety of methods for loss computation, such as VFL loss, along with DFL loss and CIOU loss. These methods refine the accuracy of regression and classification, thereby elevating the model’s proficiency in detecting targets of varying sizes and degrees of overlap.
3.2. NanoSight–YOLO
Detecting micro-vehicle targets in satellite remote sensing imagery poses unique challenges. Vehicle targets in high-resolution satellites typically occupy micro-pixel areas, significantly limiting the extraction of practical features and making conventional object detection methods challenging. Additionally, satellite imageries’ complex and variable backgrounds, including natural landscapes and artificial structures, can visually confuse vehicle targets. Furthermore, since satellite imageries often cover extensive areas, targets within the same image may appear at different angles and scales, requiring detection models to possess strong multi-scale and multi-view adaptability. These factors collectively constitute the main challenges in detecting micro-vehicle targets in satellite remote sensing imagery, necessitating advanced image processing technologies and algorithmic innovations to address them effectively.
Figure 2 shows this paper’s improved NanoSight-YOLO model structure diagram.
YOLOv8, renowned for its robust data processing and feature extraction capabilities, has shown considerable promise in detecting micro-vehicle targets within satellite remote sensing imagery. Several factors complicate vehicle detection, adversely affecting the model’s accuracy. In response, this paper introduces a detection model termed NanoSight–YOLO, an enhancement based on YOLOv8. Tailored for the specific traits of micro-vehicle targets in such imagery, NanoSight–YOLO seeks to boost the model’s efficiency and flexibility in recognizing these micro-scale vehicles.
3.2.1. FasterNet Module
This study selects the lighter YOLOv8n as the base model to reduce the computational burden caused by using complex models for simple tasks and improve detection speed. This ensures a balance between detection accuracy and speed. Although the original CSPDarknet backbone network performs excellently in feature extraction, it has limitations in detecting micro targets due to potential information loss during the convolution and pooling processes.
FasterNet, an efficient backbone network designed by Chen et al. in 2023 for object detection tasks, emphasizes the optimization balance between speed and accuracy [
50].
It achieves this goal through three core modules: the basic network module, the fast feature fusion module, and the efficient upsampling module. The essential network module extracts and processes features through convolutional layers, batch normalization layers, and activation function layers; the fast feature fusion module optimizes the integration of multi-scale features, enhancing the model’s ability to recognize targets of different scales; and the efficient upsampling module ensures computational efficiency while improving resolution, accurately restoring target positions. These designs enable FasterNet to quickly and accurately perform complex visual tasks in resource-constrained environments while maintaining a lightweight structure, making it suitable for the high-precision detection of micro targets in complex environments. The working principle and model structure are shown in
Figure 3.
FasterNet typically consists of four hierarchical stages, each composed of multiple FasterNet blocks. Each block contains one PConv layer and two PWConv layers. Additionally, before each stage begins, the network includes either an embedding layer (used for dimension reduction and channel expansion) or a merging layer (used for further spatial downsampling).
PConv is the core of FasterNet. It applies regular convolution on a subset of input channels while keeping the remaining channels unchanged, thereby reducing unnecessary computations and memory accesses. The computation and memory access details of PConv are as follows.
In the equations, h and w represent the height and width of the feature map, k is the size of the convolution kernel, and cp is the subset size of input channels used for computation.
In FasterNet, PWConv serves as the bridge between PConv and advanced network structures. It integrates features extracted by PConv, adjusts the number of channels to meet different network requirements, and enhances the network’s nonlinearity. The effectiveness of PWConv allows FasterNet to improve information integration and expressive capacity while reducing computational expenses. This makes it especially appropriate for use in environments with limited resources. Details on the computational processes and memory access associated with PWConv are outlined below.
Through the innovative PConv and the carefully designed hierarchical architecture, FasterNet significantly enhances the network’s running speed and efficiency while ensuring accuracy. The micro-vehicle target detection task in satellite remote sensing imagery effectively processes and retains critical spatial information and operates swiftly in resource-constrained environments. This makes it an ideal choice for micro-target detection in high-resolution satellite data.
3.2.2. Dynamic Head Module
The decoupled head in the YOLOv8 model has limitations in detecting micro-vehicle targets in satellite remote sensing imagery. While it performs well for larger targets, its high downsampling rate reduces its effectiveness in capturing the details of micro targets. This is especially problematic in satellite remote sensing, where features are not prominent, leading to missed and false detections. The primary reason is that the model’s feature extraction layers fail to retain sufficient detail when processing micro targets in highly complex backgrounds, and its integration of classification and localization is also limited for micro targets.
Dai et al. introduced a cutting-edge framework for object detection known as a dynamic head. This system incorporates multi-dimensional attention mechanisms within the model’s head structure for detecting objects, enabling comprehensive processing awareness of scale, space, and task-specific dimensions [
51]. This method addresses the limitations of traditional object detection head designs by dynamically adjusting the distribution of attention to optimize performance without adding extra computational burden.
The dynamic head enhances the head structure by combining three distinct attention strategies: scale-aware, spatial-aware, and task-aware attention mechanisms. Scale-aware attention modifies layer weights within the feature pyramid to suit objects of varying sizes. Spatial-aware attention employs deformable convolution networks to sharpen the model’s emphasis on critical spatial points within images. Task-aware attention tailors the activation intensity of feature channels to boost performance across different detection tasks, such as classification and localization. These mechanisms enable the dynamic head to elevate detection precision and adaptability while preserving processing efficiency. The detailed implementation steps are outlined below.
In this equation, F denotes the feature tensor, typically output from the backbone of a neural network, used for subsequent object detection tasks; L indicates the number of levels in the feature pyramid, reflecting different scales of feature layers; K is the number of sparse sampling locations, used for applying deformable convolution and executing attention weight focusing; S represents the number of spatial locations in the feature tensor, including the dimensions of width and height; C denotes the number of channels in the feature tensor, with each channel carrying a type of feature information; πL, πS, and πC, respectively, represent attention functions for different dimensions: levels, spatial, and channels; σ represents the hard sigmoid function, used to constrain inputs within the range of 0 to 1, commonly for normalization; f denotes the linear function, often implemented through a 1 × 1 convolution layer, used to process feature data; W(F) represents the weighted feature tensor, which is the feature tensor processed by the attention mechanism; pk and Δpk denote the original and offset spatial locations, used in deformable convolution to adjust spatial positions; Δmk signifies the importance weights of spatial locations, used to emphasize specific features; and α1, α2, β1, and β2 represent learned parameters for task-aware attention, used to control channel activations under different tasks.
Equation (5) demonstrates how fundamental self-attention weights enhance significant features while diminishing those of lesser importance. Equation (6) describes the decomposed attention operation, fine-tuning the feature tensor. Equation (7) represents scale-aware attention, dynamically adjusting the weights of features from different levels to accommodate objects of varying scales. Equation (8) denotes spatial-aware attention, achieved through deformable convolution to enable spatial awareness, allowing the model to focus more on critical regions within the imagery. Equation (9) illustrates task-aware attention, which optimally adjusts feature channels by comparing and selecting the results of different parameter combinations.
As seen in
Figure 4, the workflow of the dynamic head begins with feature inputs extracted by the backbone network. These features first undergo adjustments in the scale-aware module to adapt to targets of different sizes within the imagery. The adjusted features then enter the spatial-aware module, which further refines the spatial positioning of the features, emphasizing important spatial locations. Finally, the task-aware module optimizes channel features for specific detection tasks, providing more precise feature representations for subsequent tasks such as classification or regression. The dynamic head effectively addresses different scale, location, and task requirements through this hierarchical application of attention mechanisms, thereby enhancing performance across various object detection frameworks.
Therefore, the comprehensive application of the dynamic head enables exceptional performance in detecting micro-vehicle targets in satellite remote sensing imagery. This is particularly significant in scenarios requiring accurately identifying and localizing micro-scale targets within extensive, complex backgrounds. By dynamically adjusting attention distribution, the dynamic head enhances the model’s ability to perceive micro-vehicle targets in satellite imagery. It optimizes detection, making it especially effective and practical for satellite remote sensing imagery analysis.
3.2.3. Bi-Level Routing Attention Module
In satellite remote sensing imagery analysis, vehicle targets typically occupy only a tiny portion of the imagery and often face challenges such as low resolution, insufficient detail, and target clustering. During feature extraction, many negative samples are usually generated, leading to a highly uneven sample distribution, which affects the efficiency of model training and feature learning. The model incorporates an attention mechanism to tackle these challenges and concentrate on essential regions within the remote sensing imagery. This approach successfully consolidates data across various scales and levels, boosting the analysis of feature channels with vital target information. Consequently, this enhances the precision and speed of identifying micro-vehicle targets.
However, their fixed receptive fields often limit traditional attention mechanisms, lacking flexibility and adaptability in handling diverse target distributions. This limitation can lead to unstable performance in different application scenarios, making it difficult to effectively capture global information while facing high computational complexity. Bi-level routing attention (BiLRA) is an attention mechanism designed for visual transformers, aimed at addressing the computational burden traditional transformers face when processing large-scale or high-resolution imagery [
52]. The specific implementation details are shown in
Figure 5.
The BiLRA model enhances performance via three principal elements: partitioning regions and projecting inputs, routing from one area to another using a directed graph, and computing attention from token to token.
First, by segmenting the input feature map into multiple micro-regions and extracting regional features, BiLRA reduces the amount of data processed and localizes the computation. This not only speeds up data processing but also improves efficiency. Next, by constructing an inter-regional association graph and selecting key regions, BiLRA can dynamically focus on the most informative areas, avoiding ineffective and redundant computations. Finally, by performing fine-grained token-level attention calculations within these critical regions, BiLRA accurately captures essential visual features, enhancing the model’s accuracy and performance across various visual tasks. This series of steps enables BiLRA to improve computational efficiency while ensuring high-quality output, making it a powerful tool for handling complex visual tasks. The computational flow is as follows.
In the equations, Xr represents the re-partitioned feature map; Wq, Wk, and Wv are the projection matrices for query, key, and value, used to transform the input feature X into query, key, and value; Qr and Kr are the region-level query and key, used to calculate the affinity between regions; Ar is the region-to-region affinity matrix, used to determine the attention routing; Ir is the routing index matrix, selecting which other regions each region should focus on based on the highest values in the affinity matrix Ar; Kg and Vg are the gathered key and value tensors, collected from K and V based on the routing index Ir; O represents the output feature map, which is the value after being weighted by the attention mechanism; and LCE(V) stands for local equation enhancement, which typically uses deep convolution to enhance local information in the feature map.
Equation (10) forms the basis for implementing the attention mechanism. Equation (11) calculates the inter-regional affinity to determine the strength of associations between regions. Equation (12) focuses the attention mechanism’s computation on the most relevant areas selected through the region-to-region affinity graph, optimizing computational resources and processing efficiency. Equation (13) performs the final attention calculation, combining it with local equation enhancement to output the adjusted feature map.
3.2.4. Wise-Intersection over Union Module
In object detection, the loss function for bounding box regression is crucial. It aims to accurately predict the bounding box location to approximate the ground truth, achieving precise localization and recognition of the target area. The CIoU loss function, employed in YOLOv8, extends the DIoU loss function by incorporating the bounding box aspect ratio, thus enhancing regression precision. CIoU refines the prediction of bounding boxes by assessing three key factors: the area of overlap, the distance between center points, and the aspect ratio. Nonetheless, while it offers a thorough assessment of the bounding box’s shape and position, the CIoU encounters challenges in addressing minor variations among micro targets, which limits its utility in detecting micro-sized objects in densely populated scenes. The implementation of CIoU proceeds as described below.
In the equation, ρ represents the distance from the center point, Cpred is the center of the predicted bounding box, Cgt is the center of the ground truth bounding box, v denotes the aspect ratio loss, and wpred and hpred are the width and height of the predicted bounding box, respectively. At the same time, wgt and hgt are the width and height of the ground truth bounding box, respectively. Variable c is the diagonal length of the most micro-enclosing box, which contains predicted and ground truth bounding boxes. The parameter α is used to balance the aspect ratio loss, typically a coefficient that is dynamically adjusted based on the IoU. The computation method for α is shown in Equation (15), where the IoU is the intersection over the union between the predicted bounding box Bpred and the ground truth bounding box Bgt.
Equation (14) calculates the distance between the centers of the predicted and ground truth bounding boxes. Equation (16) considers the consistency of the aspect ratios of the predicted and ground truth bounding boxes. Equation (17) represents the final CIoU loss.
Wise-intersection over union (Wise-IoU) introduces a dynamic non-monotonic focusing mechanism that dynamically adjusts the gradient gain by evaluating the outlier degree of anchor boxes. This mechanism optimizes traditional loss functions’ performance when handling varied-quality anchor boxes [
53]. This technique markedly enhances the precision of object localization in detection models using medium- and low-quality anchor boxes and diminishes the excessive emphasis on high-quality anchor boxes. It effectively balances the learning focus of the model, thereby enhancing overall detection performance in complex environments. The implementation process is as follows, and
Figure 6 shows how the IOU works.
From Equations (18)–(21), LIoU calculates the loss based on the overlap between the predicted and ground truth bounding boxes. β represents the outlier degree, which assesses the current performance of the anchor box relative to its historical performance. denotes the exponential moving average of the IoU loss, used to smooth out the historical values of the IoU loss. r represents the gradient weight, which controls the attention given to anchor boxes of different quality in the loss function. α and δ are hyperparameters used to adjust the gradient weight, directly influencing the strength and manner in which the outlier degree regulates the gradient weight.
Equation (18) calculates the IoU loss, minimizing the overlapping area between the predicted and ground truth bounding boxes. Equation (19) measures the deviation of the current IoU loss of each anchor box from its historical average IoU loss to assess the quality of the anchor box. Equation (20) computes the gradient weight, dynamically adjusting the gradient allocation based on the outlier degree of the anchor box to optimize the model’s learning focus on anchor boxes of different quality. Equation (21) represents the final form of the Wise-IoU loss function, combining the basic IoU loss with the gradient weight adjusted by the outlier degree, used to train the object detection model more effectively.
3.2.5. Focaler-IoU Bounding Box Module
The conventional IoU loss function measures the overlap between predicted bounding boxes and those that are ground truth. Although it is generally effective, it encounters issues with disappearing gradients in cases of zero overlap, which obstructs the model’s learning in instances with notable positional differences. Furthermore, the IoU loss does not differentiate between simple and challenging samples; it applies the same criteria uniformly to all. This may lead the model to favor optimizing easily detectable samples while neglecting those that are more difficult to detect but crucial for improving overall performance.
Focaler-IoU is an improved loss function for bounding box regression in object detection, designed to enhance the model’s focus on complex samples by introducing a focal mechanism [
54]. This mechanism is inspired by the application of focal loss in classification problems. By adjusting the form of the loss function and using linear interval mapping to improve the traditional IoU loss function, Focaler-IoU effectively enhances the model’s learning efficiency and localization accuracy when dealing with difficult-to-locate objects, such as micro or partially occluded targets. The design of Focaler-IoU is based on a thorough reflection and improvement of the shortcomings of existing IoU-based loss functions in handling complex samples, enabling superior performance in micro-object detection tasks. The specific implementation process is as follows.
Equation (22) is the core of Focaler-IoU. Setting two thresholds, d and u, it adjusts the impact of the IoU values, causing the model to pay more attention to IoU values that fall between d and u during training. This modification helps sharpen the focus on challenging samples. Equation (23) details the loss function derived from Focaler-IoU, which targets reducing discrepancies between predicted and actual bounding boxes. Equations (24)–(28) integrate Focaler-IoU into the traditional IoU-based loss function. This integration retains the advantages of the original loss function while further enhancing the model’s ability to focus on and handle specific samples, such as hard-to-classify samples.
3.2.6. Micro-Object Detection Module
The head structure of the YOLOv8 model is a crucial component of the object detection architecture. It directly processes features extracted by the backbone network to make final predictions. A sophisticated design of upsampling and multi-level feature fusion layers effectively integrates features from different levels, improving the model’s accuracy in handling multi-scale information. This design includes multiple upsampling steps to restore high-resolution features and uses concatenation layers to fuse outputs from different depths, significantly enhancing the model’s capability to detect micro-objects. Adding C2f layers further optimizes feature representation, improving the efficiency of capturing local details. The final detection layer uses these enriched features for efficient object localization and classification. Overall, the head configuration of YOLOv8 ensures quick response times while adapting to complex and variable application environments, particularly excelling in detecting objects of different sizes and complex backgrounds.
While the original head of the YOLOv8 model has capabilities for multi-scale detection and feature fusion, it has certain limitations in micro-object detection due to constraints in feature capture, feature layer fusion methods, and sensitivity to micro-object details.
The micro-object detection layer enhances the original YOLOv8 model by introducing advanced upsampling and multi-layer feature fusion strategies, significantly improving its ability to detect micro-objects. Multiple upsampling operations ensure the quality of high-resolution feature maps, allowing the model to maintain and capture critical detail information even when the object size is extremely small. The newly added C2f layers and other feature fusion mechanisms strengthen the information flow between different levels of the network, optimizing the capture of micro-scale variations. These integrated technologies enhance the accuracy of micro-object recognition and localization, enabling the model to effectively utilize detailed information from various network layers, ensuring that even the most minor object features are not overlooked. This significantly improves micro-objects’ recognition precision and localization accuracy in complex backgrounds.