Object Detection and Monocular Stable Distance Estimation for Road Environments: A Fusion Architecture Using YOLO-RedeCa and Abnormal Jumping Change Filter

Abstract

Enabling rapid and accurate comprehensive environmental perception for vehicles poses a major challenge. Object detection and monocular distance estimation are the two main technologies, though they are often used separately. Thus, it is necessary to strengthen and optimize the interaction between them. Vehicle motion or object occlusions can cause sudden variations in the positions or sizes of detection boxes within temporal data, leading to fluctuations in distance estimates. So, we propose a method to integrate a detector based on YOLOv5-RedeCa, a Bot-Sort tracker and an anomaly jumping change filter. This combination allows for more accurate detection and tracking of objects. The anomaly jump filter smooths distance variations caused by sudden changes in detection box sizes. Our method increases accuracy while reducing computational demands, showing outstanding performance on several datasets. Notably, on the KITTI dataset, the standard deviation of the continuous ranging results remains consistently low, especially in scenarios with multiple object occlusions or disappearances. These results validate our method’s effectiveness and precision in managing dual tasks.

1. Introduction

Efficient perception and understanding of the road environment are crucial for autonomous driving technology [1]. However, achieving comprehensive perception of the surrounding environment for vehicles causes a challenging problem. This comprehensive perception system performs multiple tasks. It detects various objects around the vehicle. Additionally, it obtains the distances between these objects and the vehicle. Therefore, we focus on methods that achieve dual tasks. Our methods simultaneously detect objects around the vehicle and determine the distances to these objects. More details can be found in Figure 1.

In autonomous driving, accurate object detection is crucial for dynamic distance estimation, directly affecting the system’s safety and efficiency [2]. Dynamic distance estimation, as part of higher-level perception, can simulate the intuitive perception of human drivers.

Existing distance measurement methods based on radar and stereo vision, although effective, face challenges such as high costs, system complexity, and poor real-time performance. Furthermore, it is worth exploring whether current bird’s-eye view (BEV) perception technology requires deeper support from human visual characteristics [3]. Existing monocular distance measurement methods lack robustness when dealing with sudden value changes, which directly affects the perception and decision-making quality of autonomous driving systems in dynamic environments.

The goal of this study is to propose a monocular distance measurement method. This method aims to both detect objects and determine the distance between the objects and the vehicle. By combining the YOLOv5-RedeCa and anomaly jump filter methods, the approach intends to enhance the accuracy and robustness of dynamic distance estimation, thereby addressing perception and decision-making challenges in dynamic environments. Our method demonstrates significant potential in feature extraction and distance estimation. Additionally, it will provide more reliable and richer perceptual information for autonomous driving systems, thereby significantly improving driving safety and efficiency. This makes our method a promising candidate for breakthrough advancements in autonomous driving technology.

Reviewing previous work in the field of 2D object detection, region proposal-based methods (e.g., R-CNN) [4] have significantly improved detection accuracy through a two-stage detection approach, performing exceptionally well in complex backgrounds. However, they are relatively slow in computation. In contrast, although single-stage detectors (e.g., YOLO) [5] slightly compromise on accuracy, they directly predict the positions and categories of objects in an image through regression/classification, simplifying the model structure and computation process, making them suitable for real-time processing [6]. Therefore, we use them as the baseline for our detector.

Object detection alone cannot fully perceive the environment without distance information [7]. Traditional depth estimation methods use monocular cues or manually designed features [8,9], but they often underperform in varying depth scenarios.

Deep learning, especially with convolutional neural networks (CNNs), offers direct pixel-to-depth mapping [10]. Multi-scale networks and scale-invariant loss functions have enhanced depth estimation accuracy. Unlike traditional methods, deep learning automatically generates disparity maps and depth information without extensive annotation, typically using video sequences or stereo images. However, these methods require substantial computational resources.

In autonomous driving, continuous object detection information from the time domain is used for tracking objects and distance estimation. However, vibrations caused by vehicle motion and occlusions can disrupt the stability of objects in the video sequence. This can lead to sudden changes in the position or size of the detection boxes, as shown in Figure 2. Therefore, it is necessary to smooth the detection boxes.

Though algorithms for 2D object detection have made significant progress, they remain specialized. Similarly, monocular depth estimation algorithms have advanced considerably. However, research in these two fields remains relatively separate. Integration between object detection and depth estimation is still lacking. In the existing research, the object detection often lacks integrated depth estimation, and the depth estimation does not include object recognition capabilities. This situation highlights an urgent need. Developing an integrated system that combines object detection and depth estimation is essential. Such a system would optimize the interaction between the two techniques. Consequently, it would improve their efficiency in practical applications.

Therefore, we propose and design a holistic algorithm architecture that fuses object detection and distance estimation. Our main contributions are as follows:

• We design the RedeCa module to enhance feature extraction capability, enabling more precise object detection in road traffic scenarios. To ensure real-time performance, we use a deep separable convolutional network architecture.
• We integrate a multi-frame association mechanism, to improve the algorithm’s robustness in cases of object occlusion or temporary loss, enabling simultaneous tracking and ranging of multiple different objects.
• We design an anomaly jumping change filter to enhance distance estimation accuracy and robustness. This filter eliminates anomalies caused by sudden changes in detection box size within a frame. It smooths the variation in object distance. Consequently, the reliability of ranging is improved.

Our proposed algorithm won the gold medal in The 3rd China (shenyang) Intelligent Connected Vehicles Conference. It also achieved the bronze medal in the perception & decision control group at the 2023 World Intelligent Driving Challenge Metaverse Virtual Simulation Competition.

2. Related Work

In the field of autonomous driving, how to make the vehicle fully perceive its surrounding environment is a challenging problem. This comprehensive perception includes not only the detection of various surrounding objects, but also the acquisition of distances between these objects and the vehicle. Currently, the use of visible light cameras is a common and cost-effective approach to addressing this problem. Equipped with photosensitive elements, visible light cameras can capture color images of the real world. Researchers conduct in-depth analysis of these color images to accomplish key perception tasks such as object detection and distance estimation.

Traffic Object Detection. Object detection is one of the core problems in the field of computer vision [11]. It aims to classify and determine the position of objects in images. Deep convolutional neural networks (DCNNs) have provided strong support for the development of object detection techniques with their outstanding feature representation capabilities [12,13]. Commonly, the main architectures for object detection can be divided into two categories: region proposal-based object detection methods and regression/classification-based object detection methods [6]. Each of them has its own characteristics, addressing different technical challenges.

Representative models of region proposal-based object detection architectures include RCNN [4] and its subsequent improvements such as SPPNet [14], Fast RCNN [15], Faster R-CNN [16], Mask RCNN [17], and R-FCN [18]. The first stage of these architectures generates candidate region proposals. The second stage performs detailed classification and bounding box refinement on these regions. This staged processing greatly enhances detection accuracy, particularly exhibiting excellent performance in complex backgrounds. For instance, to address the issue of slow proposal generation speed, Faster R-CNN introduces the region proposal network (RPN). It is significant to improve the quality and speed of proposals. Mask RCNN further predicts object masks by adding a parallel branch, enabling the model to perform both object detection and instance segmentation simultaneously. This is particularly valuable in applications requiring fine-grained visual information.

To simplify model structure and computational processes, regression/classification-based object detection architectures are designed to predict object positions and categories directly on images, such as OverFeat [19], the YOLO series [20,21,22,23,24,25], SSD [26], DSSD [27], and FSSD [28]. The main advantage of these architectures is their high detection speed, making them suitable for real-time processing scenarios. Among them, the YOLO series stands out. By reframing the detection problem as a single regression task, it can simultaneously predict the categories and positions of multiple objects in a single network forward pass. To effectively address detection challenges for objects of various sizes and improve the model’s ability to detect small objects, SSD introduces the concept of multi-scale feature maps.

Monocular Depth Estimation. The pursuit of determining the distance between objects and the camera has remained a pivotal research focus within the domain of computer vision since its inception. In the initial approaches, monocular depth estimation primarily relied on Shape from Shading (SFS) and Depth from Focus/Defocus (DFF/DFD). SFS infers the object’s shape based on changes in illumination on its surface [29]. DFF/DFD techniques estimate depth using focused and defocused information in images along with camera calibration parameters [30]. The fundamental assumption is that the focused parts of the image are clearer, while the defocused parts gradually become blurry. Although effective in specific scenarios, these methods often rely on additional scene assumptions and exhibit subpar performance when confronted with scenes featuring significant depth of field.

With the advancement of machine learning technology, researchers have begun to frame the single-pixel depth estimation problem as a probabilistic learning problem based on random fields. For instance, optimizing depth maps using Markov Random Field (MRF) [8,31], and modeling posterior probabilities with Conditional Random Field (CRF) for depth estimation [9]. The enhanced computational power of Graphics Processing Units (GPUs) and the availability of large-scale image datasets have driven deep learning, paving the way for new research directions in depth estimation.

Deep learning techniques typically require real depth data as training labels. These techniques directly map pixels to depth values by constructing convolutional neural networks (CNNs). Refs. [32,33] were the first to apply CNNs in the field of depth estimation. They employed a multi-scale network strategy. A global coarse estimation network provides initial distance estimates. Additionally, a local refinement network optimizes the details. They introduced a scale-invariant loss function to handle errors. Subsequent studies have further improved estimation accuracy by refining network structures [29,34,35,36] and integrating auxiliary information [37,38]. Some research does not rely on a large amount of labeled depth data. Instead, they utilize video sequences or stereo images to automatically derive disparity maps and depth information using deep learning algorithms. For example, Ref. [39] process the left view using a convolutional neural network. This generates a disparity map. Then, they reconstruct the left view by combining it with the right view through inverse mapping. The reconstruction error is used as a supervisory signal for training. Ref. [40] simultaneously processes left and right views to optimize disparity map generation, enabling accurate depth estimation even when only monocular images are used during testing. Additionally, Ref. [41] derives pose and depth from video sequences of consecutive frames, demonstrating the effectiveness of combining multiple network models for view reconstruction. While these methods achieve good performance, they come at a significant computational cost.

Significant progress has been made in 2D object detection and monocular depth estimation. But the two remain largely separate in research, with no effective integration observed. At present, object detection systems often lack the capability for range estimation. And the depth estimation system does not integrate the object recognition ability. This separation suggests a need for development. A holistic algorithmic architecture is necessary. Such an architecture should integrate object detection and range estimation. This integration would enhance the synergy between these two techniques. Consequently, it would optimize their performance in practical applications.

3. Method

To address the integration of object detection and distance estimation, we design and develop an algorithmic architecture. It comprises three main components: a detector based on YOLOv5-RedeCa, a multi-object tracker, and an anomaly jumping change filter. In autonomous driving scenarios, vehicles often encounter complex and dynamic environments with multiple autonomously moving objects that may suddenly appear, become occluded, or overlap [21]. These situations impose stringent requirements on the accuracy and robustness of the algorithm. Additionally, due to the high speed of vehicle movement, the algorithm must process sensor data and react within milliseconds to ensure vehicle safety. Therefore, autonomous driving systems require a perception algorithm with high accuracy and robustness.

For these purposes, we designed the RedeCa module, which enhances the architecture of the object detection algorithm, improving the precision of object localization in road scenarios. Furthermore, we integrate the Bot-Sort tracker. This tracker employs a multi-frame association mechanism. It enhances the algorithm’s robustness when encountering object occlusion or temporary loss. Thus, the method can simultaneously track multiple objects and estimate their distances. The approach also periodically evaluates tracking continuity, monitors significant changes in object state, and checks the stability of the tracking process. To enhance the accuracy and robustness of distance estimation, we also integrate an anomaly jumping change filter. The filter effectively smooths the variation in object distance and eliminates potential anomalies caused by sudden changes in detection box size. Thereby, it can improve the reliability of distance measurement. The overall architecture is illustrated in Figure 3.

By implementing YOLOv5-RedeCa, our method enhances accuracy in object recognition and refines bounding box precision across video frames. Furthermore, we compute the distances between objects and the camera using a sophisticated physical imaging model.

To effectively track and measure multiple objects, we integrate the Bot-Sort tracker. This multi-frame association mechanism significantly enhances robustness in scenarios with occlusions or temporary loss of objects. Even if the object temporarily vanishes from view in certain frames, it continues to track and measure distance consistently. We assign unique identifiers to newly detected objects using an incremental sequence. Then, it utilises the Bot-Sort to match these with existing objects, distinguishing between those that have newly appeared and those already being tracked. For objects that are successfully matched, their status information is updated, and original identifiers are preserved. Conversely, new objects are allocated fresh tracking records and identifiers. If an object is not detected in the current frame, its entry is temporarily preserved. If the object fails to reappear within a predetermined period, it will be removed from the tracking list.

Now, most monocular ranging methods primarily focus on the accuracy of individual images, overlooking the correlation of distance changes across consecutive frames in the temporal domain. Therefore, we propose a multi-frame data analysis strategy, which integrates Kalman filtering. It not only enhances the precision and robustness of distance estimation, but also assists in the detailed analysis of the object’s motion patterns and trends in distance changes. Kalman filtering smooths distance variations by analyzing changes in the same object across consecutive frames. It also eliminates potential errors caused by abrupt changes in the size of single-frame detection boxes.

3.1. Object Detection

To ensure accurate object recognition and precise bounding box extraction in each video frame, we employed the YOLOv5 object detection algorithm. It serves as the core framework for our detection module, which is based on deep convolutional neural networks. YOLOv5 predicts the class and position of objects, i.e., the bounding boxes, directly from the input image in a single forward pass. Compared to the two-stage detection methods, which first extract candidate regions and then classify them, YOLOv5 considerably improves detection speed and efficiency. It uses a deep convolutional network to extract features from images, progressively drawing out increasingly complex characteristics [42]. It enhances its ability to recognize various objects. By integrating a Feature Pyramid Network (FPN), YOLOv5 combines feature maps from different levels, thereby improving its detection capabilities across a range of object sizes.

Considering the diversity of object sizes in road scenes, the accuracy of object detection, and the specific requirements of road traffic, we have improved the YOLOv5 network architecture. First, we integrated the RedeCa module into the residual blocks. This allows for more accurate detection of smaller objects under varying lighting conditions and weather changes. Then, we replaced the traditional convolutions in the residual blocks with depthwise separable convolutions. This change reduces the number of parameters in the network and enhances the real-time performance. The architecture of YOLOv5-RedeCa network is shown in Figure 4.

RedeCa Module

YOLOv5 has demonstrated outstanding performance across a wide range of detection scenarios. However, the complex road environment demands a versatile approach to accommodate the diverse types and sizes of traffic objects. To address this challenge, we optimized YOLOv5’s network structure, specifically by designing the RedeCa module to enhance recognition accuracy for traffic objects of various sizes. As shown in Figure 5, the RedeCa module is based on a residual block. It incorporates a sequence of residual depthwise separable convolution blocks and ECA modules to enhance feature extraction capabilities.

Depthwise separable convolutions markedly reduce the number of parameters and computational load in models. This is achieved by breaking down standard convolutions into two simpler operations: depthwise convolutions and pointwise convolutions. The process is detailed in Figure 6. It involves first applying depthwise convolution independently to each input channel. Then, pointwise convolution aggregates these outputs using 1 × 1 convolution kernels. This approach greatly reduces the parameter count and computational demands, lowering memory usage and energy consumption. Consequently, it makes the model more suitable for deployment on devices with limited resources.

Depthwise separable convolutions minimize the risk of overfitting, which is particularly crucial when training models on limited data.

By reducing the number of parameters, training becomes simpler and the likelihood of overfitting is reduced. It enhances the model’s generalization capability in real-world applications. The computational complexity associated with depthwise separable convolutions is detailed below.

Depthwise Convolution:

$$\begin{array}{c}\hfill Para{m}_{DC}=K\times K\times C_{in}\end{array}$$

(1)

$$\begin{array}{c}\hfill FLOP{S}_{DC}=H_{out}\times W_{out}\times K\times K\times C_{in}\end{array}$$

(2)

Pointwise Convolution:

$$\begin{array}{c}\hfill Para{m}_{PC}=C_{in}\times C_{out}\end{array}$$

(3)

$$\begin{array}{c}\hfill FLOP{S}_{PC}=H_{out}\times W_{out}\times C_{in}\times C_{out}\end{array}$$

(4)

Depthwise Separable Convolutions:

$$\begin{array}{c}\hfill Para{m}_{DW}=Para{m}_{DC}+Para{m}_{PC}\end{array}$$

(5)

$$\begin{array}{c}\hfill FLOP{S}_{DW}=FLOP{S}_{DC}+FLOP{S}_{PC}\end{array}$$

(6)

where $Para{m}_{DC}$ quantifies the parameter count associated with depthwise convolution, and $FLOP{S}_{DC}$ delineates the computational demands requisite for depthwise convolution. Correspondingly, $Para{m}_{PC}$ specifies the parameter count for pointwise convolution, whereas $FLOP{S}_{PC}$ characterizes its computational demands. Additionally, $Para{m}_{DW}$ is indicative of the parameter count in depthwise separable convolution, with $FLOP{S}_{DW}$ elucidating the computational demands for this type of convolution.

The RedeCa module is initiated with a depthwise separable convolution block that integrates principles of a residual network. It uses identity mapping to improve the learning efficiency of deep network layers. It combines the high parameter efficiency of depthwise separable convolutions with the rapid learning capability of residual connections, thereby enhancing the module’s feature extraction ability. And then, a standard depthwise separable convolution block is integrated to effectively reduce the model’s complexity and computational demands. Finally, the deca module adjusts the dependencies between channels, dynamically optimizing feature responses to improve the accuracy of object detection.

The mathematical formulation of the RedeCa module is as follows:

$$\begin{array}{c}\hfill F_{DWConv}^{Res}=SiLU(BN\left(DWConv\left(F^{*}\right)\right)+Conv\left(F^{*}\right))\end{array}$$

(7)

$$\begin{array}{c}\hfill F_{DWConv}=BN\left(Po\mathrm{int}wise\left(Depthwise\left(F_{DWConv}^{\mathrm{Re}s}\right)\right)\right)\end{array}$$

(8)

$$\begin{array}{c}\hfill F_{RedeCa}=SiLU(F_{DWConv}\times \sigma \left(Conv\left(AdaptAvgPool\left(F_{DWConv}\right)\right)\right)+F^{*})\end{array}$$

(9)

where $F^{*}$ represents the feature map input from the previous layer.

3.2. Enhancing the Robustness of Multi-Frame Association in Distance Estimation

In object detection and distance estimation in road traffic scenes, it is not enough to operate only on a single object. In practical applications, it is necessary to accurately estimate the distance of multiple objects simultaneously. And these objects may undergo sudden appearances, occlusions, overlaps, or disappearances. Our proposed method integrates the Bot-Sort tracker, assigning a unique identifier to each object in every frame, facilitating effective object identification and tracking. Although existing monocular ranging algorithms have improved accuracy on individual images, they often overlook the coherence of distance changes for the same object across consecutive frames. Ignoring this coherence can lead to fluctuations in distance measurements, especially when the bounding box size of the object detection changes drastically over a short period. Our method enhances distance estimation accuracy and robustness by integrating Kalman filtering techniques over time series data. The approach smooths out variations in object distances and eliminates anomalies caused by sudden changes in single-frame detection box sizes, resulting in superior performance in real-time road traffic scenarios.

3.2.1. Bot-Sort

In the domain of visual analysis for road traffic, the mere detection and distance estimation of individual objects are not adequate. Paramount to this field is the capability to accurately measure the distances of multiple objects within a scene and allocate distinct identifiers to each for effective tracking and differentiation. Our method integrates the Bot-Sort tracker, an advanced algorithm that adeptly identifies and analyzes every object within video frames, ascertaining new objects for tracking. The methodology of the Algorithm 1 is delineated in the form of pseudocode [43].

Algorithm 1 Frame-based object Tracking System Workflow

1: Input: Video frames   2: Output: Tracked object trajectories   3: for each frame in Video do   4:       Detect potential objects using YOLOX   5:       Estimate camera motion transformation matrix from previous frame   6:       Extract features for detected objects using SBS-S50 model   7:       Predict object locations in the current frame using Kalman Filter   8:       First association based on fused IoU and ReID scores   9:       Match current detections with existing tracklets 10:       Second association for unmatched tracklets based on IoU 11:       Update tracklets’ state information 12:       Create new tracklets and assign new unique identifiers 13:       Remove inactive tracklets if not detected within a certain period 14:       (Optional) Interpolate missing parts in tracklets 15: end for 16: return Completed trajectories for all objects

BoT-SORT stands as a superior multi-object tracking system that significantly improves tracking accuracy through the synergistic integration of motion and visual appearance data. The system leverages advanced Kalman filters alongside camera motion compensation techniques to adeptly handle real-world dynamics. It distinctively amalgamates Intersection over Union (IoU) with cosine distance measures from Re-identification (ReID) algorithms to establish consistent and reliable object associations, particularly in densely dynamic environments.

3.2.2. Object Identification and Maintenance

Our method incorporates the following logic for object identification and maintenance. Firstly, for newly detected objects, the method assigns each object a unique identifier through an incremental sequence to differentiate between different tracking objects. Subsequently, the method applies the Bot-Sort algorithm. It compares newly discovered objects with those already tracked to determine their status.

For successfully matched objects, the system updates their status information in the tracking list, while keeping their original unique identifiers unchanged. Newly appearing objects are created as new tracking entries and assigned new unique identifiers. If a tracked object is not detected in the current frame, possibly due to occlusion or leaving the field of view, the method temporarily retains its entry and ID. If the object does not reappear within the set time, it will be removed from the tracking list.

3.2.3. Anomaly Jumping Change Filter

In modeling object motion on the image plane, we typically utilize a discrete Kalman filter equipped with a constant velocity model. The state vector of this filter is defined as a septuple, $x={[x_c,y_c,s,a,{\dot{x}}_c,{\dot{y}}_c,\dot{s}]}^{\mathrm{T}}$, where $(x_c,y_c)$ specifies the 2D coordinates of the object’s center within the image plane. Here, s signifies the scale (area) of the bounding box, and a indicates its aspect ratio. To augment the precision and resilience of distance estimation, to stabilize the variations in object distance, and to address anomalies potentially triggered by abrupt changes in detection box sizes within a frame, the state vector is expanded to an octuple, $x={[x_c,y_c,a,h,{\dot{x}}_c,{\dot{y}}_c,\dot{a},\dot{h}]}^{\mathrm{T}}$. Accordingly, the state vector of the Kalman Filter (KF) is defined as depicted in Equation (10), and the measurement vector is delineated in Equation (11).

$$\begin{array}{c}\hfill x_k={[x_c\left(k\right),y_c\left(k\right),a\left(k\right),h\left(k\right),{\dot{x}}_c\left(k\right),{\dot{y}}_c\left(k\right),\dot{a}\left(k\right),\dot{h}\left(k\right)]}^{\mathrm{T}}\end{array}$$

(10)

$$\begin{array}{c}\hfill z_k={[z_{x_c}\left(k\right),z_{y_c}\left(k\right),z_a\left(k\right),z_h\left(k\right)]}^{\mathrm{T}}\end{array}$$

(11)

This approach, which integrates Kalman filtering into time series analysis, is designed to significantly improve the accuracy and robustness of distance estimation. The Kalman filter algorithm methodically smoothens fluctuations in object distances by conducting an analysis of historical data. It effectively addresses anomalies that arise from sudden changes in the size of detection boxes within individual frames. The algorithm’s state prediction is formulated in Equations (12) and (13), while the observation update mechanism is detailed in Equations (14)–(16).

$$\begin{array}{c}\hfill {\widehat{x}}_{\left.k\right|k-1}=F_k{\widehat{x}}_{\left.k-1\right|k-1}+B_k{u}_k\end{array}$$

(12)

$$\begin{array}{c}\hfill P_{\left.k\right|k-1}=F_k{P}_{\left.k-1\right|k-1}{F}_k^T+Q_k\end{array}$$

(13)

where ${\widehat{x}}_{\left.k\right|k-1}$ represents the predicted state, $F_k$ is the state transition matrix, $B_k$ denotes the control input model, $u_k$ is the control vector, $P_{\left.k\right|k-1}$ is the predicted covariance, and $Q_k$ signifies the process noise covariance.

$$\begin{array}{c}\hfill K_k=P_{\left.k\right|k-1}{H}_k^T{(H_k{P}_{\left.k\right|k-1}{H}_k^T+R_k)}^{-1}\end{array}$$

(14)

$$\begin{array}{c}\hfill {\widehat{x}}_{\left.k\right|k}={\widehat{x}}_{\left.k\right|k-1}+K_k(z_k-H_k{\widehat{x}}_{\left.k\right|k-1})\end{array}$$

(15)

$$\begin{array}{c}\hfill P_{\left.k\right|k}=(I-K_k{H}_k)P_{\left.k\right|k-1}\end{array}$$

(16)

where $K_k$ denotes the Kalman gain, $H_k$ represents the observation matrix, $z_k$ is the actual observation, $R_k$ indicates the observation noise covariance, and I is the identity matrix.

4. Experiments

4.1. Datasets

COCO Datasets. The Common Objects in Context (COCO) dataset, developed by the Microsoft team, is a vital asset in computer vision research. It contains over 330,000 images that are richly annotated across 80 object categories. By providing a diverse, large-scale, and challenging benchmark, the COCO dataset has facilitated extensive research activities [44]. Consequently, the method we proposed has been rigorously evaluated on the COCO dataset to compare its performance with leading algorithms in the field.

Table 1. COCO Dataset Training, Validation, and Test Set Splits.

Dataset Split	Number of Images	Covered Categories
Training Set	118,287	80 classes
Validation Set	5000	80 classes
Test Set	40,670	80 classes

summarizes the experimental setup and dataset partitioning, illustrating our method within the COCO dataset.

VOC Datasets. The PASCAL Visual Object Classes (VOC) dataset serves as a crucial benchmark in the field of computer vision, supporting tasks such as object detection. It encompasses annotations for 20 different object categories, including class labels and bounding boxes. The PASCAL VOC challenge series played a pivotal role in advancing vision-based technologies [45]. Additionally, we evaluate the improved YOLOv5 on the VOC dataset and compare it with other popular methods.

Table 2. VOC Dataset Training, and Validation Set Splits.

Dataset Split	Number of Images	Covered Categories
Training Set	11,540	20 classes
Validation Set	10,991	20 classes

provides a detailed description of partitioning for the VOC dataset.

KITTI Datasets. The KITTI dataset is a crucial benchmark in the field of autonomous driving research. It encompasses a variety of real-world driving scenarios including urban, rural, and highway environments. It provides comprehensive annotations for tasks such as 3D object detection, tracking, and depth estimation [46]. We use the KITTI dataset to validate the accuracy of our proposed method in distance estimation, and to demonstrate its applicability and effectiveness in realistic driving conditions.

4.2. Implementation Details

4.2.1. Training Setup

In order to ensure the fairness and consistency of the model training effect, the details of the hardware, software and experimental environment are shown in

Table 3. Experiment environment.

Environment	Version
Operating System	Ubuntu 20.04
CPU	AMD Ryzen 9 5900HX
GPU	Nvidia GeForce RTX 3080 16 G
Compiling Environment	Python 3.9.0
CUDA	12.2
Deep Learning Framework	Pytorch 1.13.1
Compiler	Pycharm 2022.3.1

. The training parameter Settings are shown in

Table 4. Training parameters.

Parameter	Value
Batch Size	16
Init Learning Rate	0.1
Final Learning Rate	0.0001
Image Size	640 × 640
Optimizer	SGD
Workers	2

.

4.2.2. Loss Function

The loss function used in the YOLOv5-RedeCa model is calculated as follows:

$$\begin{array}{c}\hfill L(t_p,t_{gt})=\sum _{k=0}^K[{\alpha }_k^{balance}{\alpha }_{box}\sum _{i=0}^{S^2}\sum _{j=0}^B\coprod _{kij}^{obj}{L}_{CIoU}+{\alpha }_{obj}\sum _{i=0}^{S^2}\sum _{j=0}^B\coprod _{kij}^{obj}{L}_{obj}+{\alpha }_{cls}\sum _{i=0}^{S^2}\sum _{j=0}^B\coprod _{kij}^{obj}{L}_{cls}]\end{array}$$

(17)

$$\begin{array}{c}\hfill L_{CIoU}=1-IoU+{\displaystyle \frac{{\rho }^2(b,b^{gt})}{c^2}}+\alpha \upsilon \end{array}$$

(18)

where K, $S^2$, and B represent the output feature map, cell, and the number of anchors on each cell, respectively. ${\alpha }_x$ represents the weight of the corresponding item. ${\alpha }_{box}$, ${\alpha }_{obj}$, and ${\alpha }_{cls}$ take values of 0.05, 0.7, and 0.3, respectively. We based the parameter values of ${\alpha }_{box}$, ${\alpha }_{obj}$, and ${\alpha }_{cls}$ on the original YOLOv5 algorithm. Using a stable and recognized baseline configuration is essential for consistent performance evaluation. ${\alpha }_k^{balance}$ is used to balance the weights of the output feature maps at each scale, with a default value of [4.0, 1.0, 0.4]. ${\coprod }_{kij}^{obj}$ indicates whether the kth output feature map, the ith cell, and the jth anchor box are positive samples. If it is a positive sample, it is 1; otherwise, it is 0. $t_p$, and $t_{gt}$ are the predicted vector and ground-truth vector.

The loss function of CIoU is as follows:

$$\begin{array}{c}\hfill L_{CIoU}=1-IoU+{\displaystyle \frac{{\rho }^2(b,b^{gt})}{c^2}}+\alpha \upsilon \end{array}$$

(19)

where b represents the center points of the predicted box, and $b^{gt}$ represents the ground-truth box. $\alpha$ is the balance factor between the loss caused by aspect ratio and the loss caused by IoU. $\upsilon$ is the normalization of the difference in aspect ratio between the predicted box and the ground-truth box. They are mathematically defined as follows:

$$\begin{array}{c}\hfill \rho (b,b^{gt})={∥b-b^{gt}∥}_2\end{array}$$

(20)

$$\begin{array}{c}\hfill \alpha ={\displaystyle \frac{\upsilon }{(1-IoU)+\upsilon }}\end{array}$$

(21)

$$\begin{array}{c}\hfill \upsilon ={\displaystyle \frac{4{(arctan\frac{w_G}{h_G}-arctan\frac{w_P}{h_P})}^2}{{\pi }^2}}\end{array}$$

(22)

The loss functions for object and class are defined as follows:

$$\begin{array}{c}\hfill L_{obj}(p_o,p_{IoU})=BC{E}_{ojb}^{sig}(p_o,p_{IoU};w_{obj})\end{array}$$

(23)

$$\begin{array}{c}\hfill L_{cls}(c_p,c_{gt})=BC{E}_{cls}^{sig}(c_p,c_{gt};w_{cls})\end{array}$$

(24)

where $(w_G,h_G)$ represents the width and height of the object box and $(w_P,h_P)$ represents the width and height of the predicted box $p_o$ is the confidence score of the object in the predicted box. $p_{I}oU$ is the confidence score of the object in the predicted box. $BC{E}_{cls}^{sig}$ and $BC{E}_{cls}^{sig}$ represent binary cross-entropy loss. $w_{obj}$ and $BC{E}_{cls}$ represent the weights of positive samples.

4.3. Evaluation Index

In the following experiments, we evaluate the accuracy of object detection against the accuracy and stability of distance estimation using the following metrics.

• Precision: This metric is pivotal for assessing the accuracy of object detection models. It quantifies the ratio of instances accurately identified as objects to all cases labeled as objects by the model. The formula is defined as

  $$\begin{array}{c}\hfill Precision={\displaystyle \frac{TP}{TP+FP}}\end{array}$$

  (25)

  where $TP$ (True Positives) represents the count of correctly detected objects, and $FP$ (False Positives) denotes the instances erroneously classified as objects.
• Recall: Recall evaluates the proportion of true objects that the model successfully detects out of the entire set of actual objects. A higher recall implies that the model is capable of identifying most true objects, albeit potentially increasing the rate of false positives. The formula for recall is

  $$\begin{array}{c}\hfill Recall={\displaystyle \frac{TP}{TP+FN}}\end{array}$$

  (26)

  with $FN$ (False Negatives) indicating the number of objects that were not detected.
• Average Precision (AP): AP calculates the mean value of precision across varying levels of recall. It can be determined for each category, and the Mean Average Precision (MAP) represents the average of APs across all categories. MAP is extensively used to gauge the comprehensive performance in tasks involving detection of multiple object classes.

  $$\begin{array}{c}\hfill mAP={\displaystyle \frac{1}{calsses}}{\sum }_{i=1}^{classes}{\int }_0^{1}P\left(R\right)d\left(R\right)\end{array}$$

  (27)
• Standard Deviation (SD): SD serves as a statistical measure to quantify the extent of variation or dispersion from the mean within a dataset. In the domain of time-domain analysis, it is pivotal to precisely evaluate the closeness of distance estimations derived from various monocular ranging algorithms to the true values, as well as for gauging their dispersion. The calculation of standard deviation is defined by the following formula:

  $$\begin{array}{c}\hfill \sigma =\sqrt{{\displaystyle \frac{1}{N}}\sum _{i=1}^N{(f\left(x_i\right)-{\mu }_i)}^2}\end{array}$$

  (28)

  where $\sigma$ denotes SD, $f\left(x_i\right)$ represents the estimated distance at time instance i, and ${\mu }_i$ corresponds to the actual value at that instance. This metric is crucial for assessing the reliability and accuracy of these algorithms in real-world applications.

4.4. Comparison Experiment

We have developed an improved YOLOv5 method, specifically optimized for detecting small objects in road scenes under various lighting and weather conditions. To evaluate the performance of our method in terms of feature representation and real-time processing capabilities, we conducted a series of experiments on the COCO and VOC datasets. Our improved methods, denoted according to scale as Ours-s, Ours-m, and Ours-l, were compared with several state-of-the-art object detection methods.

We used Precision, mAP, and Recall as the primary performance metrics. Additionally, we considered complexity by evaluating the number of parameters (Params) and floating-point operations per second (FLOPS). We performed a comparative analysis of different methods on the COCO and VOC datasets, ensuring the robustness of the experimental design and the validity of the data analysis. The results are presented in the accompanying

Table 5. Detection results of different algorithms on COCO and VOC datasets.

Method	COCO			VOC			Params (M)	FLOPS (G)
	Precision (%)	[email protected] (%)	Recall (%)	Precision (%)	[email protected] (%)	Recall (%)
SSD	50.7	45.1	38.8	31.4	54.7	57.3	26.3	65.7
Faster R-CNN	53.1	48.0	46.0	34.1	57.5	54.7	60.4	93.6
Mask-RCNN	51.9	48.0	45.7	33.9	57.2	69.1	63.4	147.0
YOLOv5s	63.5	54.3	52.1	69.4	60.3	52.9	7.1	16.1
YOLOv5m	70.4	61.3	55.4	82.5	74.3	67.2	21.0	48.5
YOLOv5l	72.9	64.0	59.7	82.5	75.4	69.3	46.2	108.6
YOLOv6s	64.7	52.8	46.2	65.3	58.3	51.6	18.5	45.3
YOLOv7s	65.8	47.6	49.5	66.8	58.3	52.5	6.1	13.2
YOLOv8s	68.1	56.3	53.1	68.8	56.5	53.0	11.2	28.8
YOLOv9s	67.1	55.7	50.7	66.7	61.7	54.1	9.6	38.8
YOLOv10s	66.3	57.2	51.8	68.4	60.7	56.1	8.1	24.9
Ours-s	67.2	55.0	51.3	67.0	57.5	53.5	6.7	13.2
Ours-m	71.6	61.5	57.9	81.9	77.5	73.4	17.2	34.6
Ours-l	73.6	64.4	60.1	82.5	80.0	76.0	34.3	70.4

.

Experimental results on the COCO dataset show that our optimized Ours-m model achieved 71.6% Precision, 61.5% mAP, and 57.9% Recall. These results are the best among all methods using similar computational resources. On the VOC dataset, the model also excelled, registering 77.5% [email protected] and 73.4% Recall. Notably, despite having fewer parameters than the equivalent scale YOLOv5 model, Ours-s, Ours-m and Ours-l surpassed other methods in Precision and mAP. It demonstrates our success in improving performance while optimizing computational efficiency.

Compared to SSD and Faster R-CNN, Ours-m demonstrated significant advantages on both the COCO and VOC datasets. It was particularly effective in Recall for small object detection. For instance, on the COCO dataset, the Recall of Ours-l was 21.3% higher than that of SSD and 14.1% higher than Faster R-CNN, highlighting the effectiveness of the RedeCa module in enhancing recall capabilities.

Although Mask-RCNN is designed for instance segmentation, the Ours-l model excelled in object detection tasks. On the VOC dataset, it achieved an [email protected] of 80.0%, compared to Mask-RCNN’s 57.2%. This emphasizes that our improved model delivers outstanding detection performance, even in domains specialized by Mask-RCNN.

We also controlled for model complexity while striving for higher performance. Compared to the original YOLOv5, the Ours series demonstrated superior small object detection in complex environments with fewer parameters. On the COCO dataset, Ours-l improved mAP and Recall by 0.4% each over YOLOv5l. Our methods surpass existing methods in key performance metrics while maintaining computational efficiency, laying a solid foundation for practical applications.

We compared our method with the latest object detection methods from YOLOv6 to YOLOv10 [47,48,49,50,51]. Our method matches YOLOv8 and YOLOv9 in terms of accuracy, and surpasses YOLOv6, v8, v9, and v10 in model lightweighting and computational load. The data presented in Table 5 showed that while our model achieves accuracy comparable to the latest YOLO versions, it still has benefits in model lightweighting and computational load. These advantages contribute to the deployment and operational efficiency of our method in practical applications.

The RedeCa module enhances our method’s ability of representing features, especially in accurately handling edge and detail information, which is crucial for small object detection. It also increases the adaptability to varying lighting and weather conditions, thereby boosting robustness in complex environments.

In road traffic scenarios, real-time performance is crucial. We evaluated our method against popular methods in terms of FPS and parameter count, as detailed in Table 5. Our method’s parameter count and computational complexity are markedly lower than those of Faster R-CNN and Mask-RCNN. Compared to the baseline YOLOv5, our method achieves comparable performance while reducing both parameter count and computational complexity.

The experimental results show that by adding the RedeCa module to YOLOv5, we improve small object detection accuracy markedly while maintaining high recall rates. These findings support strongly further exploration of deep learning methods for small object detection in complex environments, confirming the effectiveness and potential of our enhancements.

To validate the superior accuracy and robustness of our method, we conducted a comprehensive quantitative and qualitative comparative analysis. We compared our method with popular monocular distance measurement methods in road traffic scenarios. The results are shown in Figure 7.

As shown in Figure 7, our monocular distance measurement method effectively reduces potential errors caused by sudden changes in the detection bounding box size. This leads to reduced dispersion in distance measurements, ensuring they remain more consistent with actual distances over time.

We also conducted a qualitative analysis to compare our method with mainstream methods. Unlike methods that generate depth maps for the entire image to obtain object distances, our method detects objects and measures their distances simultaneously, improving efficiency. The distance measurement results are shown in Figure 8.

4.5. Ablation Experiment

We adopted the ablation experiment method to evaluate the impact of different modules on the detection performance of YOLOv5m model on VOC dataset. The experimental results are shown in

Table 6. Detection results using different modules on VOC dataset.

Method	DW	ECA	RedeCa (DW + ECA)	Precision (%)	mAP (%)	IOU (%)	Params (M)	FLOPS (G)
YOLOv5m				82.5	74.3	67.2	21.0	48.5
YOLOv5m	✓			70.5	62.9	38.4	9.1	18.3
YOLOv5m		✓		82.7	76.3	74.1	21.8	48.6
YOLOv5m	✓	✓		79.3	74.6	71.3	20.8	47.6
YOLOv5m + RedeCa			✓	81.9	77.5	73.4	17.2	34.6

. We investigate the effect of depthwise separable convolution (DW) on model performance. In addition, we examine the impact of effective channel attention (ECA). We also explore the direct combination of DW and ECA. Finally, we assess the performance of the RedeCa module.

As shown in Table 6, the baseline YOLOv5m model achieved a Precision of 82.5%, mAP of 74.3%, and IOU of 67.2% on the VOC dataset. It had 21.0 million parameters and 48.5 GFLOPS.

After adding Depthwise Separable Convolutions (DW) to the baseline model, Precision decreased to 70.5%, mAP dropped to 62.9%, and IOU significantly declined to 38.4%. However, the number of parameters and FLOPS were reduced to 9.1 million and 18.3 GFLOPS, respectively. This indicates that while the DW module effectively reduces model complexity, it negatively impacts detection performance.

Incorporating the Efficient Channel Attention (ECA) module resulted in a slight improvement in Precision to 82.7%, an increase in mAP to 76.3%, and a significant rise in IOU to 74.1%. The number of parameters and FLOPS slightly increased, demonstrating that the ECA module enhances detection performance without significantly increasing the computational burden.

Using Depthwise Separable Convolutions (DW) and Efficient Channel Attention (ECA) directly, the model saw slight improvements in Mean Average Precision (mAP) and Intersection over Union (IoU). But the increase was minimal, and there was no significant reduction in the network’s parameters and computational load. However, by effectively integrating DW and ECA into the RedeCa module, the model’s accuracy slightly decreased to 81.9%, but the mAP increased to 77.5%, and the IoU rose to 73.4%, while the parameters and FLOPS were reduced to 17.2 M and 34.6 G, respectively. This indicates that the use of the RedeCa module can balance detection performance and model complexity to some extent, providing an effective solution for real-time object detection applications.

To verify that the use of an anomaly jump filter helps smooth distance variations and eliminate potential errors caused by sudden changes in the detection bounding box size, we conducted a comparative analysis. We compared the algorithm architecture with the anomaly jump filter to the one without it.

As shown in Figure 9, using a Kalman filter integrated with the time series effectively eliminates potential errors caused by sudden changes in detection bounding box size. This reduces the dispersion in distance measurements, making them more consistent with the actual distances over time.

5. Conclusions

Our improved object detection network performs excellently on the COCO dataset. It achieves 71.6% precision, 61.5% mean Average precision ([email protected]) and 57.9% recall, respectively. These results are better than the comparison models under the same computing resources. In addition, on the VOC dataset, our improved object detection network also shows excellent performance in [email protected] and recall. The results are 77.5% and 73.4%, respectively. By integrating the Bot-Sort algorithm, the robustness of the proposed method is significantly improved when dealing with object occlusion or temporary loss, so that the system can track and distance multiple objects at the same time. At the same time, the abnormal jump filter can effectively smooth the sudden change of object range, eliminate the potential anomaly caused by the mutation of the detection box size, and thus enhance the reliability of ranging.

The holistic algorithm architecture we designed and implemented successfully breaks through the traditional research gap between 2D object detection and monocular depth estimation. We combine the two techniques by designing a new algorithm architecture. This significantly improves the performance of object detection and distance estimation in dynamic and complex environments. The integration of these technologies not only optimizes the respective functions, but also improves the efficiency of the system in practical applications. In particular, it has shown great application potential in the field of traffic monitoring and automatic driving.