DVIF-Net: A Small-Target Detection Network for UAV Aerial Images Based on Visible and Infrared Fusion

Highlights

What are the main findings?

• A novel visible-infrared fusion network for small-target detection in UAV aerial images, DVIF-Net, is proposed, achieving mAP50 of 85.8% and 62.0% respectively on two cross-modal datasets DroneVehicle and VEDAI, with parameters of only 2.49 M.
• The proposed P4-level cross-modal feature fusion strategy, dual context-guided fusion module (DCGF), and edge information enhancement module (CSP-EIE) have significantly enhanced the ability of visible-infrared feature fusion and the expression ability of small-target edge features.

What is the implication of the main finding?

• The study provides a lightweight and high-precision solution for real-time UAV-based small-target detection in complex environments such as low illumination, fog, and occlusion.
• The proposed fusion strategy and module design offer a valuable reference for deploying multimodal detection models on resource-constrained embedded platforms.

Abstract

During UAV aerial photography tasks, influenced by flight altitude and imaging mechanisms, the target in images often exhibits characteristics such as small size, complex backgrounds, and small inter-class differences. Under single optical modality, the weak and less discriminative feature representation of targets in drone-captured images makes them easily overwhelmed by complex background noise, leading to low detection accuracy, high missed-detection and false-detection rates in current object detection networks. Moreover, such methods struggle to meet all-weather and all-scenario application requirements. To address these issues, this paper proposes DVIF-Net, a visible-infrared fusion network for small-target detection in UAV aerial images, which leverages the complementary characteristics of visible and infrared images to enhance detection capability in complex environments. Firstly, a dual-branch feature extraction structure is designed based on YOLO architecture to separately extract features from visible and infrared images. Secondly, a P4-level cross-modal fusion strategy is proposed to effectively integrate features from both modalities while reducing computational complexity. Meanwhile, we design a novel dual context-guided fusion module to capture complementary features through channel attention of visible and infrared images during fusion and enhance interaction between modalities via element-wise multiplication. Finally, an edge information enhancement module based on cross stage partial structure is developed to improve sensitivity to small-target edges. Experimental results on two cross-modal datasets, DroneVehicle and VEDAI, demonstrate that DVIF-Net achieves detection accuracies of 85.8% and 62%, respectively. Compared with YOLOv10n, it has improved by 21.7% and 10.5% in visible modality, and by 7.4% and 30.5% in infrared modality, while maintaining a model parameter count of only 2.49 M. Furthermore, compared with 15 other algorithms, the proposed DVIF-Net attains SOTA performance. These results indicate that the method significantly enhances the detection capability for small targets in UAV aerial images, offering a high-precision and lightweight solution for real-time applications in complex aerial scenarios.

1. Introduction

With the rapid advancement of unmanned aerial vehicle (UAV) platforms and sensor technologies, target detection technology based on UAVs has demonstrated significant application potential in fields such as military reconnaissance [1], search and rescue [2], agricultural crop monitoring [3], power line inspection [4], and security monitoring [5]. However, when UAVs are performing aerial photography tasks, their flight altitude usually varies in accordance with the specific mission environment and requirements. The variation in the flight altitude of UAV can cause targets in images they capture to exhibit diverse target characteristics in terms of scale. Furthermore, in most application scenarios, targets in UAV aerial images will also present characteristics such as small size, large quantity, and dense distribution, which undoubtedly increases the difficulty of detecting small targets in UAV aerial images. Especially in complex scenes, factors such as low resolution and environmental interference will also increase missed and false detection rates of small targets [6]. Images obtained by a single sensor are usually not effective at capturing the complete information of complex scenes, and often require multiple sensors or imaging devices to generate complementary information. Visible images are imaged by reflecting visible light, featuring rich textures and high spatial resolution. However, under extreme conditions such as insufficient lighting and target occlusion, the imaging effect is poor, which has an adverse impact on the model detection performance. In contrast, infrared images rely on the thermal radiation of the target itself for imaging, thus being less affected by lighting and weather conditions and having the ability to work all-weather. However, they suffer from severe loss of texture details and have a low contrast between target and background, making it impossible to fully reflect scene information. Therefore, relying solely on one type of image to perform object detection tasks has significant limitations, especially for UAV missions that depend on visual information. Given the complementary characteristics of visible images and infrared images, effectively fusing the two can not only retain the color, texture and edge information of visible images, but also highlight the thermal radiation information of infrared targets relative to their backgrounds, thereby improving the perception and discrimination ability of small targets in UAV aerial image in complex environments. Therefore, building an efficient end-to-end dual-modal fusion detection network is not only the key to improving the performance of small-target detection, but also an inevitable choice to promote the practical engineering application of UAV visual perception systems.

Over the past few decades, visible and infrared image fusion (VIIF) technology has undergone continuous advancement. It can be roughly divided into two categories: traditional VIIF methods and deep learning-based VIIF methods. Currently, traditional VIIF mainly includes multi-scale transform-based methods [7], sparse representation-based methods [8], subspace decomposition-based methods [9], and hybrid methods [10], etc. These methods rely heavily on manually designed prior knowledge, which not only leads to inefficient performance and high consumption of time and labor, but also results in limited detection accuracy [11]. In contrast, deep learning-based VIIF methods can automatically learn and extract key information from source image, thus eliminating cumbersome process of manually designing features that traditional methods require. Deep learning-based VIIF methods can generally be classified into three types: CNN-based methods [12,13], Transformer-based methods [14,15], and GAN-based methods [16]. For instance, Cai et al. [17] designed an unsupervised image fusion network DCSFuse guided by the correlation of image features, which can adaptively integrate complementary and long-distance context information. Park et al. [18] proposed a Cross-Modal Transformer Fusion (CMTFusion) algorithm based on infrared and visible image fusion, which captures global interactions by extracting complementary information from source image. Qi et al. [19] proposed a pseudo-supervised generative adversarial network (GAN) and single-scale retinal resolution (SSR) embedding method for infrared and visible fusion. This method significantly enhances both contrast and naturalness in fused image by utilizing a residual GAN embedded in the SSR module and a dense hybrid modal input strategy. Although these existing VIIF methods have achieved advanced performance, there are still some issues that need to be further addressed. Specifically, most fusion methods typically adopt elementary fusion strategies, such as simple connection or addition. This overly simplistic fusion approach neglects the differences between modalities, resulting in insufficient interaction between two modalities. This deficiency creates a clear boundary between the processing of single-modal images and feature fusion, resulting in insufficient utilization of cross-modal information and making it difficult to improve the object detection model’s performance. In addition, at present, most models either build a four-channel input or keep visible images and infrared images in two separate branches, respectively, and then merge their features in the downstream network. This method leads to a large amount of redundant information in fusion results, which brings high computational complexity and limited model generalization ability, posing a severe challenge to achieving efficient object detection on embedded devices with limited computing resources.

To address these challenges, we propose a small-target detection method for UAV aerial images based on the fusion of visible and infrared images, DVIF-Net, to enhance the detection effect of small targets in complex scene conditions of UAV aerial images. The main contributions of this paper are as follows:

• This paper proposes the DVIF-Net, which performs cross-modal feature fusion between the backbone network and the neck network. Unlike other methods that use “Concat” or “Add” to fuse bimodal images, DVIF-Net fuses bimodal information in feature space and focuses on small targets detection task for UAV aerial images.
• A P4-level cross-modal feature fusion strategy is designed, which achieves better fusion effect by performing one fusion at appropriate positions (rather than multiple fusions), and has fewer model parameters.
• A novel dual-context information-guided fusion module (DCGF) is proposed, which uses SE attention mechanism to capture and utilize important context information during dual-modal feature fusion, and then guides model to learn this information, thereby enhancing the model’s detection performance. In addition, DCGF employs an element-wise multiplication strategy to enhance the interaction ability between visible and infrared feature information.
• An edge information enhancement module CSP-EIE based on a cross stage partial structure is proposed, which employs AdaptiveAvgPool to expand the receptive field of the network, enabling the extraction of high-frequency image information across different scales. Meanwhile, an edge enhancer is incorporated to extract target edge features, thereby improving the backbone network’s capability in feature extraction for small targets.

The remaining part of this paper is arranged as follows. Section 2 reviews the related work. Section 3 introduces the proposed DVIF-Net and analyzes and explains the structural framework and working principle of this model. Section 4 verifies the effectiveness and generalization of the method proposed in this paper through experiments. Lastly, Section 5 summarizes and looks forward to the research results of this paper.

2. Related Work

2.1. Small Object Detection Method Based on Deep Learning

Early small-target detection methods mainly relied on manually designed feature detectors. In the field of visible image detection, traditional methods such as using Histogram of Oriented Gradient (HOG) [20] and Support Vector Machines (SVM) [21,22,23] for detection. These methods largely rely on manual labor in the design and selection of features, and their accuracy, objectivity, robustness and generalization are all restricted to a certain extent. Eventually, detection accuracy and speed are not ideal, which affects practical application. In the field of infrared image detection, traditional methods such as filter-based methods [24,25], local contrast augment-based methods [26], human visual system (HVS) -based methods [27,28], and low-rank methods [29,30], all of which are widely used in infrared small targets detection. These methods are often limited by manually adjusting hyperparameters. They are not robust enough to variations in target size and shape and can only be used in specific environments and tasks, lacking adaptability and objectivity.

With the development of deep learning, target-detection algorithms based on convolutional neural networks have gradually become a research hotspot. They can automatically learn image features without human intervention, greatly improving both accuracy and efficiency of detection. Deep learning object detection algorithms are mainly divided into two categories based on the different processing paradigms: two-stage algorithms and one-stage algorithms. Two-stage detection algorithm first generates candidate regions and then uses convolutional neural networks to classify and regress them. Two-stage algorithms have high accuracy and robustness when dealing with complex scenes and small-target detection, but their main disadvantages are high computational complexity and poor real-time performance. Common two-stage algorithms include R-CNN [31], SPP-Net [32], Fast R-CNN [33], and Faster R-CNN [34]. In contrast, one-stage detection algorithms such as YOLO [35,36,37] and SSD [38] transform localization problem into a regression problem, directly predicting bounding box and category of the target object from input image, thus having a faster detection speed and lower computational complexity. Among them, the YOLO algorithm is widely popular in the field of small object detection in UAV aerial images due to its outstanding object detection capability. For instance, Zhang et al. [39] proposed a lightweight feature enhancement, fusion, and context-aware YOLO detector, which solved the problem of low detection accuracy of small targets in aerial images. Di et al. [40] proposed a lightweight and high-precision model UAV-YOLO based on YOLOv8s, which incorporates Dual Separable Convolution (DSC) and an SPPL module to improve multi-scale target robustness through cross-level feature correlation. In the field of infrared target detection, Luo et al. [41] designed the YOLO-SMUG model to address issues such as high computational requirements and accuracy limitations in UAV infrared target detection. Wu et al. [42] enhanced infrared small-target feature extraction by designing a Swin-convolution backbone. Meanwhile, they created a multi-scale void space pyramid pooling module (MASPPM) to improve feature representation, thereby achieving more accurate detection. Zhang et al. [43] proposed a method called NOC-YOLO, which integrates attention mechanisms and multi-scale feature fusion to boost detection accuracy for small vehicles in aerial infrared images. Although above methods have made significant progress in accuracy and lightweight of single-modal object detection, their ability to detect objects under extreme lighting and weather conditions still has fundamental limitations, especially for small-target detection. Therefore, researchers have gradually extended detection methods based on single-modal images to multi-modal image fusion detection.

2.2. Infrared and Visible Light Image Fusion Method

Image fusion is a process that utilizes the complementary characteristics of multimodal image features to fuse effective information from multi-source data, thereby obtaining fused images that incorporate multimodal image features. The fused images usually conform better to the visual characteristics of humans and machines, and are also more conducive to further target recognition and detection of images [44]. Infrared images and visible images exhibit different features due to their different bands. The fusion of infrared images and visible images, with complementary information, offers richer features than single-source information and can enhance target detection accuracy. At present, based on the fusion level, methods for fusing infrared and visible images can be classified into three types: pixel-level fusion, feature-level fusion, and decision-level fusion.

Pixel-level fusion method first registers infrared and visible images, then selects a fusion algorithm to perform pixel-level processing on infrared and visible images to obtain the fused image, which is then input into detector for target detection. Schnelle et al. [45] conducted research using spatial domain and pyramid-based pixel-level visible and infrared fusion methods, demonstrating that the fusion method based on low-pass pyramid ratio has a lower computational cost and superior detection performance. Zhao et al. [46] proposed an infrared and visible image fusion method based on target detection meta-feature embedding. The core idea is to design a meta-feature embedding model and generate target semantic features according to fusion network capability, thereby making semantic features naturally compatible with fused features. In addition, by simulating the optimization strategy of meta-learning, adaptability of the model is further enhanced. Que et al. [47] proposed a fusion algorithm for “fast local salient region detection”. This algorithm first adopts structured decomposition method to decompose infrared and visible images into a detail layer and base layer, and then uses the weight parameter map obtained by visual saliency detection (VSD) to guide base-layer fusion. Finally, detail layer and base layer are fused according to principle of entropy maximization to obtain the final fused image. This method retains and highlights detailed information in image, enriching image data. Although the pixel-level fusion method retains the most complete feature information when performing image fusion, its computational complexity is relatively high, and it is not suitable for application on embedded devices with limited resources.

Feature-level fusion methods extract the input infrared and visible data, first converting them into high-dimensional feature representations and then fusing them through different fusion rules. Finally, the high-dimensional features obtained through fusion are input into the detection model for target detection. Under normal circumstances, fused features offer richer semantic and more detailed information than single features, thereby enhancing detection accuracy and robustness of the model. Cheng et al. [48] proposed a lightweight dual-modal fusion network SLBAF Net based on feature-level fusion for UAV target detection under complex lighting and weather conditions. This network utilizes a dual-modal adaptive fusion module (BAFM) to adaptively fuse visible and infrared feature maps, enhancing the robustness of the model for small-target detection. Wang et al. [49] proposed a cross-scale dynamic convolution driven YOLO fusion network (CDC-YOLOFusion), which innovatively introduced a cross-scale dynamic convolution fusion module (CDCF) to achieve adaptive bimodal feature extraction and fusion based on data distribution. Sun et al. [50] proposed a new TSFusion image fusion framework, which uses a teacher–student learning approach to achieve higher-quality fusion and high computational performance. In addition, they also introduced multi-scale loss functions to guide student networks to learn from teachers at the semantic and pixel levels. Feature-level fusion methods not only achieve a balance between accuracy and speed, but also can flexibly adjust fusion levels and methods, thus becoming a key research direction in infrared and visible fusion target detection.

Decision-level fusion method is the highest-level fusion approach. It extracts features from different modal data, respectively, to obtain deep semantic information, conducts training to obtain detection results, and then scores the data of different modalities to determine detection results of different modalities and obtain final correct detection result. Yao et al. [51] proposed a decision-level fusion object detection model IVF-Mask R-CNN based on Mask R-CNN. This model adopts strategies such as reducing the learning rate and magnifying test image to enhance model’s accuracy and stability, making up for the limitations of a single modal in the field of object detection. Chen et al. [52] proposed a non-learning multimodal detection result fusion method: the probabilistic ensemble technique ProbEn. ProbEn does not require any multimodal data for training and can handle missing modalities through probabilistic marginalization. The decision-level fusion method has characteristics of good fault tolerance and high flexibility compared with other two methods. However, there is a large loss of information during fusion, which is not conducive to the precise positioning of small targets.

3. Methodology

3.1. Overall Framework

Building upon YOLOv10 [53], this paper proposes a visible and infrared image fusion network DVIF-Net for small-target detection in UAV aerial images. Its structure is shown in Figure 1. This network can effectively integrate the features of visible and infrared images and focus on small-target detection tasks in UAV aerial images. Firstly, a novel P4-level cross-modal feature fusion strategy is proposed for fusing visible and infrared images. Compared with traditional mid-term fusion methods, this strategy not only achieves better fusion effects but also has smaller model parameters. Secondly, to better utilize complementary information between two modalities, we introduce a dual-context information guide fusion module, DCGF (see Appendix A). The element-wise multiplication strategy in DCGF not only enhances the original features of the input image, but also introduces the feature information of another mode. Finally, we propose an edge information enhancement module EIE and apply it to the cross stage partial structure. This module significantly enhances boundary perception ability of small-target features through two-stage processing of AdaptiveAvgPool and EdgeEnhancer. In addition, it also features cross-modal compatibility, which not only enhances the detailed information of visible images but also improves the thermal radiation profile of infrared images. Furthermore, Figure 2 presents the detailed structure of DVIF-Net backbone network and each component module.

3.2. P4-Level Cross-Modal Feature Fusion Strategy

Most fusion networks typically employ traditional mid-fusion methods for visible and infrared image fusion [54,55,56], as illustrated in Figure 3a. This approach first uses two separate backbone networks to extract features from visible and infrared images, generating multi-scale feature pyramids. In the downstream network, feature maps from the two modalities are fused at stages P3 to P5 (in some cases across all backbone stages) using operations such as “Concat” or “Add”. The fused features are then passed to the detection head for prediction. Although this fusion strategy can leverage features from different levels, it still has certain limitations. Due to the physical differences between visible and infrared images, excessive fusion across multiple levels may introduce interfering information, thereby reducing fusion effectiveness. Additionally, multi-level fusion can lead to significant redundancy in the fused outputs, increasing computational costs and making it difficult to meet the real-time detection requirements of UAV.

To address these issues, this paper proposes a P4-level cross-modal feature fusion strategy, which effectively utilizes P4 features to enhance interaction between the two modalities while reducing redundant information. As shown in Figure 3b, fusion is performed at the specificP4 level. This is because earlier fusion may result in insufficient feature extraction, while later fusion could lead to the loss of sensitive information related to small targets. The P4-level feature maps retain both rich semantic information and fine-grained details. Specifically, after feature extraction via two independent backbone networks, visible and infrared features are processed and enhanced by a fusion module at the P4 level before being passed to the neck network. This fusion approach not only improves detection accuracy and performance but also reduces model parameters and computational complexity.

Figure 1. Overview of DVIF-Net.

Figure 1. Overview of DVIF-Net.

Figure 2. Detailed diagram of the backbone network structure of DVIF-Net.

Figure 2. Detailed diagram of the backbone network structure of DVIF-Net.

Figure 3. Comparison between traditional mid-term feature fusion and P4 mid-term feature fusion strategies.

Figure 3. Comparison between traditional mid-term feature fusion and P4 mid-term feature fusion strategies.

3.3. Dual-Context Information Guide Fusion Module

Typically, in aerial target detection tasks performed by UAV, images captured by visible cameras contain rich texture and detail information but are sensitive to illumination changes. In contrast, images captured by infrared cameras are insensitive to illumination changes and can highlight thermal targets, yet they suffer from low spatial resolution and a lack of texture details. Therefore, integrating the high-resolution texture information from visible images with the thermal radiation information from infrared images can enhance the performance of UAV-based target detection. However, most existing methods simply concatenate or add the two modalities, neglecting the differences between them and resulting in insufficient interaction of information across modalities.

To more effectively utilize and integrate complementary information from visible and infrared images, this paper proposes a dual context-guided fusion module (DCGF) based on a channel attention mechanism. This module enhances the original features of the input feature maps while introducing complementary information from the other modality. The structure of DCGF is illustrated in Figure 4. Specifically, first of all, we concatenate the input visible features ${\mathcal{F}}_{vi}$ and infrared features ${\mathcal{F}}_{ir}$ along the channel dimension. The concatenated features are then fed into a global average pooling module to compress the spatial information, aiming to enlarge the receptive field of the network, so that it can learn context information of each channel. Next, the compressed features pass through 2 cascaded convolutional layers and a non-linear Sigmoid function to generate attention weights. These weights are applied to the original features via element-wise multiplication, and the resulting weighted features are added to the original features from the other branch to enhance their representational capacity. The feature enhancement process can be expressed as

$${\widehat{\mathcal{F}}}_{ir}={\mathcal{F}}_{ir}\oplus \left({\mathcal{F}}_{vi}\mathsf{\otimes }\delta \left(Pw\text{-}Con{v}^n\left(GAP\left(C\left({\mathcal{F}}_{ir},{\mathcal{F}}_{vi}\right)\right)\right)\right)\right)$$

(1)

$${\widehat{\mathcal{F}}}_{vi}={\mathcal{F}}_{vi}\oplus \left({\mathcal{F}}_{ir}\mathsf{\otimes }\delta \left(Pw\text{-}Con{v}^n\left(GAP\left(C\left(\left({\mathcal{F}}_{ir},{\mathcal{F}}_{vi}\right)\right)\right)\right)\right)\right)$$

(2)

where ${\widehat{\mathcal{F}}}_{vi}$ and ${\widehat{\mathcal{F}}}_{ir}$ represents the enhanced visible and infrared features. $\oplus$ denotes element-wise summation and $\mathsf{\otimes }$ denotes element-wise multiplication. The element-wise multiplication operation adds the important infrared features from the channel attention output to the visible features, while also enhancing the important visible features into the infrared features. Compared to regular feature addition, this operation allows the important channels in the infrared features to be enhanced by the visible features, while the key information in the visible features is guided by the infrared features, thereby improving the interaction between the two modalities. $\delta \left(\cdot \right)$ represents the Sigmoid function, $Pw\text{-}Con{v}^n\left(\cdot \right)$ represents n cascaded pointwise convolutional layers (n values 2), $GAP\left(\cdot \right)$ represents the global average pooling operation, and $C\left(\cdot \right)$ represents the concatenation operation in channel dimension.

Finally, the enhanced visible and infrared features are concatenated for subsequent multi-scale feature fusion. The concatenation process can be summarized as

$${\mathcal{F}}_{fusion}=C\left({\widehat{\mathcal{F}}}_{ir},{\widehat{\mathcal{F}}}_{vi}\right)$$

(3)

where ${\mathcal{F}}_{fusion}$ represents the features after concatenation.

3.4. Edge Information Enhancement Module

In actual tasks, when UAVs are shooting images, due to their fast-maneuvering speed, high flight altitude and complex shooting scenes, targets in images often have problems such as small size and blurred edges, which affect detection accuracy. To this end, we proposed an Edge information enhancement module (EIE) and embedded it into cross stage partial structure (CSP), naming it CSP-EIE (see Appendix A). This module enhances network’s ability to extract edge features from small targets through multi-scale edge enhancement module and cross-scale connection structure. To fully leverage the advantages of the EIE module, we integrate it into the small object feature extraction stage of DVIF-Net, specifically the P2 stage. The EIE module not only effectively enhances the texture and edge details in visible images but also strengthens the thermal radiation contours in infrared images, demonstrating excellent cross-modal compatibility. By simultaneously improving the edge feature representation of both modalities, EIE enables the network to more accurately capture and fuse critical information from visible and infrared images, thereby enhancing the model’s performance in detecting small objects under complex scenes. The structure of EIE module is shown in Figure 5, and its specific process is as follows.

Given a feature map, $X\in {ℝ}^{C\times H\times W}$ and $S=\left\{s_1,s_2,s_3,\dots ,s_n\right\}$ represent a set of n different pooling scales. EIE first performs the AdaptiveAvgPool operation on the input feature map $X$ to generate a feature map of size $s\times s$ (assuming $s\in \left[3,6,9,12\right]$), enabling EIE to capture features at different levels. Among them, each scale branch independently processes information of different granularities; small scales (such as 3 × 3) capture the global context information of feature map, while large scales (such as 12 × 12) retain local details. Then, each branch successively passes through the 1 × 1 convolution compression channel to C/4 to reduce computational load of the module, and then enhances local features through the 3 × 3 group convolution operation. Subsequently, the feature maps of different scales are aligned to a unified scale through bilinear interpolation operations, and EdgeEnhancer is used to highlight features edges, enabling network to better perceive the boundaries and details of small targets.

Meanwhile, in another branch, the input feature maps undergo a local convolution to obtain local information containing the initial detailed features. Then, this local information is concatenated with the features after edge enhancement. Finally, a convolutional layer is utilized to fuse them into a unified feature representation.

Among them, the EdgeEnhancer in Figure 5 enhances the edge information of feature map. It applies AvgPool2d to smooth the input feature map and extract its low-frequency information. Then, the enhanced edge information (high-frequency information) is obtained by subtracting the smoothed feature map from the original input feature map. Subsequently, a convolution operation is used to further process the enhanced edge information. Finally, the processed edge information is added to the original input feature map to form the enhanced output. The specific process of EdgeEnhancer is expressed as follows:

$$E\left(X\right)=X\oplus Conv\left(X-AvgPool\left(X\right)\right)$$

(4)

where $\oplus$ denotes element-wise summation.

3.5. Loss Function

In the loss function section, the design strategy of DVIF-Net loss function is consistent with that of YOLOv10, mainly consisting of three parts: distributed focal loss, object classification loss, and object localization loss, which are, respectively, represented by ${\mathcal{L}}_{dfl}$, ${\mathcal{L}}_{cls}$, and ${\mathcal{L}}_{box}$. The overall loss function is expressed as follows:

$${\mathcal{L}}_{total}={\lambda }_1{\mathcal{L}}_{dfl}+{\lambda }_2{\mathcal{L}}_{cls}+{\lambda }_3{\mathcal{L}}_{box}$$

(5)

where $\lambda$ is a hyperparameter representing the weight of each part. ${\lambda }_1$, ${\lambda }_2$, and ${\lambda }_3$ are set to 1.5, 0.5, and 7.5, respectively, and they can be adjusted according to actual task during training.

${\mathcal{L}}_{dfl}$ is Distribution Focal Loss (DFL), whose function is to enable network to quickly focus on values near the labeled positions and maximize their probabilities. The calculation formula is as follows:

$${\mathcal{L}}_{dfl}={\displaystyle \sum _{I=0}^{K\times K}{\displaystyle \sum _{p=0}^3\left[I_{ij}^{\mathrm{obj}}\cdot \mathrm{DEL}\left(s_i,s_{i+1}\right)\right]}}$$

(6)

$$\mathrm{DEL}\left(s_i,s_{i+1}\right)=-\left[\left(y_{i+1}-y\right)\log \left(s_i\right)+\left(y-y_i\right)\log \left(s_{i+1}\right)\right]$$

(7)

where $k\times k$ represents the grid numbers on different scales feature maps of the input image. The specific value depends on the network size (e.g., if the input image is 640 × 640 in size, the values are 20 × 20, 40 × 40, and 80 × 80, respectively). $p$ represents four predicted coordinate values. DFL performs regression on the prediction box in a probabilistic manner and requires setting the hyperparameter reg_max (default is 16) in advance. At this point, the output channel of this network branch is 4 × reg_max. Before this, 16 fixed reference values $\mathrm{A}\in (0,1,2,\dots ,15)$ are set, respectively, corresponding to each position of reg_max. For these reg_max values, the softmax function is used for discretization, regarding it as a 16-class classification problem. Target position coordinates obtained from the feature map generally do not fall on specific grid corners, but labels require integers. Taking prediction $x_{min}$ as an example, its true value is $y$, the left integer is $y_i$, and the right integer is $y_{i+1}$. $\left(y_{i+1}-y\right)$ and $\left(y-y_i\right)$ correspond to the distance weights from the true values and $s_i$ and $s_{i+1}$ correspond to the predicted values of $\left(y_{i+1}-y\right)$ and $\left(y-y_i\right)$, respectively.

The object classification loss uses Binary Cross-Entropy (BCE) loss, and its calculation formula is as follows:

$${\mathcal{L}}_{cls}={\displaystyle \sum _{I=0}^{K\times K}\left[I_{ij}^{\mathrm{obj}}{\displaystyle \sum _{c\in classes}\left[P_i^j\left(c\right)\log \left[{\widehat{P}}_i^j\left(c\right)\right]+\left[1-P_i^j\left(c\right)\right]\log \left[1-{\widehat{P}}_i^j\left(c\right)\right]\right]}\right]}$$

(8)

where $k\times k$ is consistent with Formula (7), $I_{ij}^{\mathrm{obj}}$ indicates whether the prediction target exists in the $j^{th}$ prediction box in the $i^{th}$ grid (1 indicates existence, and 0 indicates non-existence). $c$ represents target category. $P_i^j\left(c\right)$ and ${\widehat{P}}_i^j\left(c\right)$ represent the predicted probability and the true probability that the target belongs to a specific category, respectively.

The object location loss adopts CIoU loss function, which measures the differences based on overlapping area, distance between the bounding boxes center, and the similarity of the aspect ratio, making the bounding box regression more stable. The calculation formula for CIoU is as follows:

$${\mathcal{L}}_{box}={\mathcal{L}}_{CIoU}=1-{\displaystyle \frac{B_{\mathrm{gt}}\cap B_{\mathrm{prd}}}{B_{\mathrm{gt}}\cup B_{\mathrm{prd}}}}+{\displaystyle \frac{{\rho }^2(B_{\mathrm{gt}},B_{\mathrm{prd}})}{{(w_{\mathrm{c}})}^2+{(h_c)}^2}}+\alpha v$$

(9)

$$\alpha ={\displaystyle \frac{v}{(1-\mathrm{IoU})+v}}$$

(10)

$$v={\displaystyle \frac{4}{{\pi }^2}}{(arctan{\displaystyle \frac{w_{\mathrm{gt}}}{h_{\mathrm{gt}}}}-arctan{\displaystyle \frac{w_{\mathrm{prd}}}{h_{\mathrm{prd}}}})}^2$$

(11)

Some of the parameters involved in the formula are shown in Figure 6; $B_{\mathrm{gt}}$ is the area of the real box and $B_{\mathrm{prd}}$ is the area of the predicted box. ${\rho }^2\left(B_{\mathrm{gt}},B_{\mathrm{prd}}\right)$ denotes distance between the centers of these two boxes. $h_{\mathrm{prd}}$ and $w_{\mathrm{prd}}$ are the width and height of the minimum bounding box of these two boxes. $\alpha$ is a balance term, used to balance the center point position of the bounding box and the size of the bounding box, and $v$ is the cosine value of the aspect ratio of the bounding box. The height and width of the prediction box and true box are $h_{\mathrm{prd}}$, $w_{\mathrm{prd}}$, $h_{\mathrm{gt}}$, and $w_{\mathrm{gt}}$, respectively.

4. Experiment

4.1. Dataset

The DroneVehicle cross-modal dataset [57] is a large-scale visible-infrared remote sensing vehicle dataset created by Tianjin University in 2020. This dataset contains a total of 56,878 aerial images, including 452,570 visible vehicle targets and 500,517 infrared vehicle targets, totaling 953,087 annotated vehicle targets. These images cover vehicles captured from various weather conditions (sunny, rainy and foggy), different angles (15°, 30° and 45°) and heights (80 m, 100 m and 120 m), and possess different environmental scenes such as highways, urban roads, residential areas, parking lots, etc. Figure 7 shows some representative visible and infrared images in the DroneVehicle dataset. The images in this dataset not only present diverse scenes, but also include images from different time periods, from day to night, as well as those under low-light conditions. These target vehicles not only have differences in size, but also have characteristics such as small size, dense occlusion, and similar appearance, which greatly increase the detection difficulty of target detection model.

The DroneVehicle dataset has been divided into a training set, validation set, and test set according to the ratio of 6.3:0.5:3.2. Furthermore, since a white border with a width of 100 pixels is set at the top, bottom, left and right of each image in this dataset, we preprocess the original images to remove the surrounding white border.

Table 1. The number of labels of different categories in DroneVehicle dataset.

Modality	Car	Truck	Bus	Van	Freight Car
RGB	339,779	22,123	15,333	11,935	13,400
IR	428,086	25,960	16,590	12,708	17,173

presents in detail the number of tags for different types of vehicles in both visible and infrared images within the DroneVehicle dataset, including five categories: cars, trucks, buses, vans and freight cars. This dataset provides rich training data for vehicle recognition and classification tasks, especially offering data support for identifying multiple types of vehicles.

4.2. Experimental Platform and Parameters

To evaluate the effectiveness of the DVIF-Net method in UAV aerial image detection tasks, ablation experiments and comparative experiments of DVIF-Net are designed based on DroneVehicle dataset.

Table 2. Configuration experimental environment.

Environment	Parameters
GPU	Intel(R) Xeon(R) Platinum 8488C
CPU	NVIDIA A100
GPU memory size	80 G
Operating system	Win 10
Language	Python 3.10.17
Frame	Pytorch 2.1.1
CUDA Version	Cuda 11.8

and

Table 3. Training parameters setting.

Parameters	Setup
Epochs	300
Input image size	640 × 512
Batch size	32
Worker	8
Learning rate	0.01
Momentum	0.937
Weight decay	0.0005
Optimizer	SGD

present versions and configuration information of the environment required for the experiment, as well as relevant parameter settings during the training process, to ensure repeatability and fairness of the experiment.

4.3. Assessment Indicators

This article utilizes Precision (P), Recall (R), mAP (mean Average Precision) and Parameters as evaluation metrics to assess the model’s performance. mAP represents the mean of average precision of all categories (mAP50 indicates an IoU threshold of 0.5, and mAP50-90 indicates an IoU threshold between 0.5 and 0.95). The Params represent the number of parameters in model and quantify computing and storage resources used. The specific operation formulas are as follows:

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(12)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(13)

$$mAP={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{\left(P\left(R\right)dR\right)}_i}$$

(14)

where TP, FP, and FN denote true positive, false positive, false negative, respectively, and N indicates the number of categories

4.4. Comparison Experiment

To verify the performance of the DVIF-Net method, comparative experiments are conducted on DroneVehicle cross-modal dataset, comparing it against 15 widely adopted and state-of-the-art methods. Among them are classic detection algorithms (RetinaNet [58], Faster R-CNN, Mask R-CNN [59], Cascade Mask R-CNN [60] and RoITransformer [61]), the YOLO series of lightweight models (YOLOv5n, YOLOv8n, YOLOv10n, YOLOv11n, YOLOv12n), as well as the most advanced visible and infrared fusion object detection models (CFT [54] and UA-CMDet [48], RemoteDet-Mamba [62], CDC-YOLOFusion [63], DE-YOLO [56]). The specific experimental results are shown in

Table 4. Performance comparison between DVIF-Net and different algorithms.

Method	Modality	mAP50 (%)						Params (M)	FLOPs /G
		Car	Truck	Bus	Van	Freight Car	All
RetinaNet	Visible	78.5	34.4	69.8	28.8	24.1	47.1	145.0	99.9
Faster R-CNN	Visible	79.0	49.0	77.0	37.0	37.2	55.9	58.30	169.5
Mask R-CNN	Visible	68.5	39.8	66.8	25.4	26.8	45.5	242.0	93.5
Cascade R-CNN	Visible	68.0	44.7	69.3	29.8	27.3	47.8	368.0	153.8
RoITransformer	Visible	61.6	55.1	85.5	27.6	42.3	61.5	273.0	123.8
YOLOv5n	Visible	90.5	61.3	90.6	52.7	47.1	68.4	2.50	7.2
YOLOv8n	Visible	91.2	62.3	90.2	56.0	49.2	69.8	3.01	8.1
YOLOv10n	Visible	91.5	63.9	90.7	55.4	51.0	70.5	2.26	6.5
YOLOv11n	Visible	91.3	63.0	90.2	55.2	48.8	69.9	2.58	6.3
YOLOv12n	Visible	91.3	64.5	90.5	52.4	53.0	70.3	2.51	6.6
RetinaNet	Infrared	88.8	35.4	76.5	32.1	39.5	54.5	145.0	99.9
Faster R-CNN	Infrared	89.4	53.5	87.0	42.6	48.3	64.2	58.30	169.5
Mask R-CNN	Infrared	88.8	48.9	78.4	32.2	36.6	57.0	242.0	93.5
Cascade R-CNN	Infrared	81.0	47.2	79.3	33.0	39.0	55.9	368.0	153.8
RoITransformer	Infrared	89.6	51.0	88.9	44.5	53.4	65.5	273.0	123.8
YOLOv5n	Infrared	97.6	67.7	94.5	50.3	70.0	75.2	2.50	7.2
YOLOv8n	Infrared	98.1	72.1	96.3	61.9	71.3	79.9	3.01	8.1
YOLOv10n	Infrared	98.2	71.7	95.7	63.9	70.1	79.9	2.26	6.5
YOLOv11n	Infrared	98.0	71.7	95.5	61.9	70.4	79.5	2.58	6.3
YOLOv12n	Infrared	98.1	70.4	95.8	62.8	69.8	79.4	2.51	6.6
CFT	VI + IR	91.9	67.0	92.1	56.4	55.2	72.5	44.53	123.5
UA-CMDet	VI + IR	87.5	60.7	87.1	38.0	46.8	64.0	139.70	98.1
RemoteDet-Mamba	VI + IR	98.2	81.2	95.7	52.9	67.9	81.8	71.34	84.5
CDC-YOLOFusion	VI + IR	98.4	71.9	96.2	64.8	69.7	80.2	153.60	48.9
DE-YOLO	VI + IR	98.3	78.4	96.8	68.5	76.0	83.6	6.00	18.7
Ours	VI + IR	98.6	82.3	97.4	71.9	78.9	85.8	2.49	8.2

Note: Bold values indicate the best results.

. Under identical conditions, we conducted comparative experiments of single-modal (visible images or infrared images) and multi-modal (visible and infrared fusion), respectively, to analyze the performance metrics of each model.

As shown in Table 4, among single-modal detection models, infrared-based models generally outperform visible-based models. This is because visible images provide limited feature information under nighttime or occluded conditions, while infrared data, being less affected by lighting, can clearly capture target characteristics. Furthermore, multimodal detection methods exhibit superior performance compared to single-modal approaches, as they integrate the textural details from visible images and the thermal radiation information from infrared images. Among all multimodal detection methods, our proposed DVIF-Net achieves the best performance, delivering the highest detection accuracy while having the lowest parameters. Specifically, compared to YOLOv10n, DVIF-Net improves mAP50 by 21.7% for visible modality and 7.4% for infrared modality, with only a 0.23 M increase in Params, demonstrating its effective integration of complementary features from both modalities. Additionally, when compared to algorithms such as RetinaNet, Faster R-CNN, Mask R-CNN, Cascade Mask R-CNN, and RoITransformer, our method shows significant improvements in both accuracy and model lightweightness. At the same time, compared to lightweight models in the YOLO series such as YOLOv5n, YOLOv8n, YOLOv11n, and YOLOv12n, the proposed DVIF-Net achieves the highest accuracy, despite having slightly more parameters than YOLOv10n. Moreover, our method improves detection performance across all target categories, indicating strong robustness in detecting objects at various scales. Similarly, when compared to other state-of-the-art fusion models including CFT, UA-CMDet, RemoteDet-Mamba, CDC-YOLOFusion, and DE-YOLO, DVIF-Net achieves the best results in mAP50, model parameters, and FLOPs, outperforming CFT (+13.3, −42.04, −115.3), UA-CMDet (+21.8, −137.21, −89.9), RemoteDet-Mamba (+4.0, −68.65, −76.3), CDC-YOLOFusion (+5.6, −151.11, −40.7), and DE-YOLO (+2.2, −3.51, −10.5).

In summary, the proposed DVIF-Net method demonstrates high detection accuracy and strong adaptability and robustness when dealing with the task of small-target detection in UAV aerial images. Meanwhile, for the UAV-embedded devices with limited resources, our method can effectively solve the problem of the large computational load of detection model, affecting their practical applications.

4.5. Ablation Experiment

To validate the practicality of the three proposed improvements, we use DroneVehicle cross-modal dataset to evaluate the impact of each improvement unit on model in sequence. The ablation experimental results of the proposed DVIF-Net are presented in

Table 5. Ablation experiment results.

P4Fusion	DCGF	CSP-EIE	P (%)	R (%)	mAP50 (%)	mAP50-95 (%)	Params (M)
			79.4	75.5	81.0	58.2	3.436
√			80.6	75.2	81.5	59.7	2.479
√	√		80.4	80.3	84.2	61.1	2.487
√		√	79.4	78.6	83.4	63.1	2.477
√	√	√	81.6	80.3	85.8	64.4	2.485

Note: Bold values indicate the best results.

and Figure 8. As shown in Table 5, we design five different experimental settings. In the first experiment, all the improvements are removed, and the backbone network directly used “Concat” for fusion in phases P3–P5, respectively, which is also the traditional fusion method. In the second experiment, we train using the proposed P4-level cross-modal feature fusion method, where the DCGF and CSP-EIE modules are still removed. In the third experiment, “Concat” is replaced with a DCGF fusion module on the basis of P4-level fusion. In the fourth experiment, in the DVIF-Net framework, DCGF fusion module is replaced with “Concat” for feature fusion. In the fifth experiment, all the improvements are added to model to form the final visible and infrared fusion framework DVIF-Net. From the results presented in Table 5, we can draw the following conclusions:

• Compared to the conventional P3–P5 multi-stage fusion approach, the proposed P4-level cross-modal fusion strategy significantly enhances model performance, increasing mAP50 from 81.0% to 81.5% while reducing model parameters by 0.96 M. This improvement demonstrates the effectiveness of the strategy. Performing one-time fusion solely at the P4 level not only decreases computational complexity but also achieves better fusion performance.
• By replacing the “Concat” operation with the DCGF fusion module for feature integration, we observed an overall upward trend in accuracy metrics including P, R, mAP50, and mAP50-95, with a notable improvement of 2.7% in mAP50. This indicates that the DCGF module effectively combines complementary information from visible and infrared images, thereby enhancing feature fusion performance.
• Even without the DCGF module, using CSP-EIE alone still improves model performance, raising mAP50 from 81.9% to 83.4% while the Params of only 2.477 M. This improvement can be attributed to the combined effect of the AdaptiveAvgPool layer and the EdgeEnhancer in CSP-EIE, which enhance the backbone network’s capability to extract edge information of small targets.
• The combination of P4Fusion, DCGF, and CSP-EIE achieves the best performance, yielding mAP50 of 85.8% with a total model parameter of 2.485 M. This result confirms the effectiveness of our proposed strategies, demonstrating a significant improvement in detection accuracy for small targets in UAV aerial images under complex scenarios.

In addition, Figure 8 (all data in the figure are normalized, the closer the Params index is to 0, the better and the closer P, R, mAP50, and mAP50-95 are to 1, the better) presents the performance of each metric of DVIF-Net and the comprehensive performance of the entire model. A to E denote the five models in Table 5, respectively. It is clear from Figure 8a that the model E proposed in the article demonstrates the most balanced performance effect—it improves the mean average precision by 2.4% compared to the enhanced Model D combined with CSP-EIE, while only increasing the Params by 0.008 M. In particular, the radar chart in Figure 8b illustrates that model E has the best overall performance. It not only improves accuracy rate but also enhances detection speed.

4.6. Visual Experiment

To further validate the effectiveness and reliability of the proposed DVIF-Net for small-target detection in UAV aerial images under complex environmental conditions, we conducted visualization experiments on DroneVehicle cross-modal dataset. The results are illustrated in Figure 9, Figure 10, Figure 11 and Figure 12. The experiment involved the selection of various types of detection scenarios—such as nighttime, heavy fog, low contrast, small targets, occlusion, and dense distributions—to compare the detection performance of the single-modal model YOLOv10n and our DVIF-Net method on both visible and infrared images.

Specifically, Figure 9 depicts a nighttime scenario. As shown in the original images, the targets in infrared image exhibit clear contours, while those in visible image are nearly invisible due to darkness. When detecting targets in the visible image, YOLOv10n not only missed a significant number of objects but also misclassified air conditioning units on buildings as vehicles. In contrast, our proposed fusion network accurately identified vehicles in the dark and simultaneously improved detection accuracy for infrared targets. Figure 10 presents a heavy fog scenario where targets display limited features in both visible and infrared images. This leads YOLOv10n to miss multiple occluded and corner-located small targets. DVIF-Net effectively improves this issue and reduces the missed detection rate, owing to the element-wise multiplication strategy in the DCGF fusion module, which provides rich multimodal differential information and enhances the expressiveness of the fused features. Figure 11 illustrates a low-contrast scenario. As can be seen from the original image, the difference in gray and brightness values between the target to be detected and the background is not significant, while texture features across different vehicle categories exhibit high similarity. As a result, YOLOv10n recognizes only a limited number of targets. Our method, by integrating the clear texture features in visible images, not only significantly improves the detection accuracy of infrared targets but also correctly identifies the target categories. Figure 12 depicts a parking lot scenario where the targets to be detected are characterized by small size, dense overlapping, and partial occlusion. While YOLOv10n fails to detect multiple occluded small targets, the edge information enhancement module in DVIF-Net strengthens the contour features of these targets, enabling the model to accurately identify more targets and demonstrating its strong adaptability under diverse environmental conditions. Overall, the proposed DVIF-Net has more performed better in accomplishing the detection task of small targets, effectively reducing both false and missed detections, and achieves good performance in the detection of small targets in UAV aerial images in complex backgrounds.

Additionally, we further evaluate the detection capability of DVIF-Net through the confusion matrix, as shown in Figure 13. The purpose of the confusion matrix is to help us understand the model’s performance across different classes by dividing the model’s classification results into true and incorrect categories. This allows us to calculate a range of performance metrics such as accuracy, precision, recall, and F1 score. The values along the diagonal of the confusion matrix represent the proportion of correctly classified instances for each category, while the off-diagonal values indicate the proportion of misclassified instances. From Figure 13a–c, we can see that in the DVIF-Net confusion matrix, all the diagonal values are higher than those of YOLOv10, indicating that the improved method demonstrates better classification performance and higher detection accuracy in UAV aerial images.

4.7. Comparative Experiments on VEDAI Dataset

To further validate the generalization capability and robustness of our proposed method, we conducted comparative experiments with DVIF-Net on the cross-modal remote sensing dataset VEDAI. The VEDAI dataset [63], captured by drones equipped with both visible and infrared cameras, is primarily used for vehicle target detection. It contains 1250 strictly aligned visible-infrared image pairs collected from diverse perspectives, scenarios, and lighting conditions, including nine categories such as Car, Boat, Bicycle, and Tractor. The dataset provides two image resolutions (512 × 512 and 1024 × 1024), with our experiments utilizing the 1024 version for training and testing. VEDAI encompasses multiple scenarios, flight altitudes, and vehicle categories. Beyond small-sized targets, it presents challenges such as multi-directionality, illumination variations, shadow changes, specular reflections, and occlusions. This fully demonstrates the applicability and challenge of using this dataset to verify our algorithm.

To demonstrate the generalization capability of DVIF-Net, this paper further compares its performance with other state-of-the-art algorithms on the VEDAI dataset. Unlike previous selections, the comparison primarily focuses on lightweight models from the YOLO series: YOLOv5n, YOLOv8n, YOLOv10n, YOLOv11n, and YOLOv12n, etc. During model training and validation process, the VEDAI dataset is divided into training, validation, and test sets in a ratio of 7:2:1, with consistent training parameters and evaluation metrics as mentioned above.

As evidenced by the experimental results in

Table 6. Comparative experimental results on VEDAI dataset.

Method	Modality	mAP50(%)										Params (M)	FLOPs /G
		Car	Truck	Boat	Trackor	Camping Van	Pickup	Plane	Van	Others	All
YOLOv5n	Visible	80.5	46.1	34.7	48.6	65.4	62.6	69.1	25.2	33.2	51.7	2.50	7.2
YOLOv8n	Visible	79.9	55.5	48.3	47.4	67.9	65.4	70.9	25.5	31.6	54.7	3.01	8.1
YOLOv10n	Visible	81.1	56.5	48.1	46.0	67.5	65.6	79.5	26.8	33.6	56.1	2.26	6.5
YOLOv11n	Visible	81.1	53.8	46.4	47.0	67.7	65.1	81.2	25.6	34.3	55.8	2.58	6.3
YOLOv12n	Visible	80.5	58.3	35.0	48.1	61.4	69.6	69.5	15.9	34.6	52.5	2.51	6.6
YOLOv5n	Infrared	76.6	43.5	36.6	45.7	58.9	59.5	31.7	15.3	24.5	43.6	2.50	7.2
YOLOv8n	Infrared	74.7	57.0	40.0	38.4	56.5	60.4	26.8	19.0	26.0	44.3	3.01	8.1
YOLOv10n	Infrared	77.4	52.7	45.1	44.9	56.8	57.6	46.8	18.6	27.5	47.5	2.26	6.5
YOLOv11n	Infrared	77.3	53.0	45.7	45.2	54.3	63.6	46.7	18.3	25.9	47.8	2.58	6.3
YOLOv12n	Infrared	77.7	50.2	39.2	46.5	62.1	59.9	44.2	18.4	24.7	47.0	2.51	6.6
Ours	VI + IR	81.3	60.0	58.8	58.5	67.6	70.5	92.0	27.7	42.0	62.0	2.49	8.2

Note: Bold values indicate the best results.

, unlike the DroneVehicle dataset, the visible-based single-modal detection model outperforms its infrared-based counterpart in the VEDAI dataset. This phenomenon can be attributed to two primary factors. (1) The VEDAI dataset is predominantly captured during daytime with sufficient illumination, resulting in clearly discernible target textures in visible images. (2) The vast majority of images in this dataset contain repetitive similar textures (such as lakes, mountains, forests, etc.), which lead to significant interference in infrared image target detection and a relatively low contrast between targets and background. Compared with single-modal detection methods, multi-modal detection methods still demonstrate superior performance. The visible mode of mAP50 has improved by 10.5% compared to YOLOv10n, and the infrared mode has improved by 30.5%, indicating that the DVIF-Net we proposed has also achieved outstanding performance on the VEDAI dataset. Overall, the experimental results fully demonstrate that our method is not limited to specific datasets, but exhibits excellent detection performance in different bimodal datasets, demonstrating strong generalization and applicability.

Similarly, to more intuitively validate the effectiveness of the improvement strategy we proposed, we conducted a visualization experiment on the test set, as shown in Figure 14, Figure 15 and Figure 16. The experiment selected various types of detection scenarios such as small targets, low contrast, occlusion, and density as detection samples to compare the detection performance of the single-modal model YOLOv10n and DVIF-Net. As shown in Figure 14 and Figure 15, the proposed method not only effectively detects smaller targets missed by YOLOv10n but also accurately identifies their categories. Furthermore, in Figure 16, YOLOv10n misidentifies cylindrical structures on rooftops as vehicles, whereas DVIF-Net avoids such false detections, demonstrating that our approach also significantly reduces the false detection rate in complex scenarios.

5. Discussion

Thanks to the introduction of the P4-level cross-modal fusion strategy, using a DCGF structure and a CSP-EIE module, the proposed DVIF-Net has higher adaptability and better detection performance in visible light and infrared small-target detection. It effectively avoids problems such as missed detections and false detections. Meanwhile, compared with 15 other algorithms, the proposed DVIF-Net attains SOTA performance. Although the method we proposed performs well in the cross-modal dataset of unmanned aerial vehicle (UAV) aerial photography, there are still some limitations. Specifically, the effectiveness of DVIF-Net on low-power platforms such as embedded devices needs further testing and improvement. In addition, when facing more complex scenarios (such as storms, smog, etc.), the robustness of this model still needs to be improved.

In future work, we will attempt to employ pruning or distillation techniques to enhance the lightweight and processing speed of the converged network, enabling it to achieve more lightweight and rapid target detection when deployed to embedded devices. In addition, to further enhance the target detection capabilities of UAV in more complex scenarios, we will consider introducing more modalities into the target detection network for fusion to achieve broader applications.

6. Conclusions

In this study, we propose DVIF-Net, a visible-infrared fusion network for small-target detection in UAV aerial images, aiming to improve the detection performance of small targets in UAV aerial images under complex environmental conditions. In the feature extraction stage, DVIF-Net adopts a parallel dual-branch structure to process features of visible and infrared images, respectively. For the fusion of visible and infrared features, we first introduce the P4-level cross-modal fusion strategy that reduces model parameters by decreasing fusion stages while avoiding information redundancy caused by multi-level fusion. Meanwhile, the DCGF module is introduced at the fusion layer. This module adaptively enhances the complementary features between two modalities and suppresses redundant information through a dynamic weight distribution mechanism. In the end, a novel EIE module is proposed, which significantly improves the representation of edge features for small targets via an adaptive average pooling layer and an edge enhancement mechanism. Experimental results on two cross-modal UAV aerial datasets show that our method significantly improves the detection accuracy of small targets, reduces both missed and false detection rates, and enhances the adaptability and stability of the detection model in complex scenarios. Furthermore, compared to 15 other state-of-the-art methods, the proposed DVIF-Net achieves superior performance in visible-infrared small-target detection. Our method not only brings a new idea to the field of multimodal fusion, but also provides a more effective solution for the high-precision detection of small targets in UAV aerial images.