CDTracker: Coarse-to-Fine Feature Matching and Point Densification for 3D Single-Object Tracking

Abstract

Three-dimensional (3D) single-object tracking (3D SOT) is a fundamental yet not well-solved problem in 3D vision, where the complexity of feature matching and the sparsity of point clouds pose significant challenges. To handle abrupt changes in appearance features and sparse point clouds, we propose a novel 3D SOT network, dubbed CDTracker. It leverages both cosine similarity and an attention mechanism to enhance the robustness of feature matching. By combining similarity embedding and attention assignment, CDTracker performs template and search area feature matching in a coarse-to-fine manner. Additionally, CDTracker addresses the problem of sparse point clouds, which commonly leads to inaccurate tracking. It incorporates relatively dense sampling based on the concept of point cloud segmentation to retain more target points, leading to improved localization accuracy. Extensive experiments on both the KITTI and Waymo datasets demonstrate clear improvements in CDTracker over its competitors.

1. Introduction

Single-object tracking (SOT) is a fundamental visual perception task with wide-ranging applications in fields such as autonomous driving, indoor robotics, and UAVs [1,2,3,4]. Three-dimensional (3D) SOT aims to seamlessly and consistently track a specified target across each frame of the input sequence, given the initial 3D state of the target.

Existing 3D SOT methods (see Figure 1) can be categorized into two paradigms: the matching-based Siamese paradigm [3,5,6,7,8,9] and the motion-centric paradigm [1]. The matching-based Siamese paradigm typically involves the use of Siamese networks to extract point cloud appearance features from both the template and search regions. Subsequently, cosine similarity [3,4] and an attention mechanism [5,8] are performed to match features from these two regions, followed by target recognition and localization. The motion-centric paradigm simply leverages motion features to efficiently track the specified target by estimating the motion state between consecutive pairs of point clouds.

Despite the success achieved by both paradigms in 3D SOT, there are still some challenges that remain unresolved. Due to the presence of the feature matching module, the matching-based Siamese paradigm is suitable for targets with little appearance variation, but it often fails in complex and dynamic scenes. Huge appearance variations can cause failures in feature matching and, thus, affect the quality of the features [8,10]. Furthermore, commonly used methods in feature matching, such as cosine similarity and an attention mechanism, were not integrated into previous approaches. This resulted in a lack of embedding important information in the template or not assigning weights to different features appropriately [3,4,6,7]. For the motion-centric paradigm, it eliminates the need for feature matching, enhancing the network’s adaptability to complex and dynamic scenes. However, it demands an extensive dataset with diverse motion patterns for training to achieve robust tracking performance and generalization, which can be resource-intensive [1]. The sparsity of point clouds poses a considerable challenge to tracking performance. The incomplete shapes resulting from this sparsity can cause the network to shift its focus to nearby distractions, thereby substantially affecting tracking results [3,10]. Unfortunately, addressing the issue of sparsity remains unresolved in both paradigms.

We propose a novel robust 3D single-object tracking network, dubbed CDTracker. It first adopts the “coarse-to-fine” concept to enhance the robustness of feature matching to alleviate tracking failures from abrupt appearance changes and further stabilizes tracking by densifying the point cloud. CDTracker consists of two key modules: the coarse-to-fine feature matching module and the relatively dense sampling module. The first module combines cosine similarity and an attention mechanism to grasp point cloud similarity. Our method entails embedding cosine similarity and the corresponding template features into the search region. Thereafter, varying feature weights are assigned, producing a more discriminative feature set. This approach enhances the robustness of feature matching. Moreover, the second module leverages a segmentation-based strategy to preserve a higher density of target points within the search area, which mitigates the sparsity problem and further stabilizes tracking.

We evaluate the performance of CDTracker through a comparative analysis of the KITTI [11] and Waymo [12] datasets. Remarkably, CDTracker outperforms most of its competitors, showcasing its superior performance. Moreover, comprehensive ablation studies demonstrate the effectiveness of our CDTracker. The contributions are mainly three-fold:

• We propose a novel 3D SOT network, dubbed CDTracker, to handle abrupt changes in appearance features and sparse point clouds.
• We introduce a novel coarse-to-fine feature matching module based on the hybrid similarity learning mechanism, which combines cosine embedding and attention assignment in the feature matching of 3D SOT.
• We introduce a relatively dense sampling module that segments and retains more points of interest, thereby further improving the tracking performance.

2. Related Work

3D Single-Object Tracking. Existing 3D SOT methods can be categorized into two paradigms: the matching-based Siamese paradigm and the motion-centric paradigm. A majority of existing methods adhere to the matching-based Siamese paradigm. SC3D [2] is a pioneering work in 3D SOT. It utilizes the Kalman filter to generate suggested target frames, compares them with the target template based on feature similarity, and selects the frame with the highest similarity as the tracking result. However, SC3D has limitations, such as the inability to detect object relationships in the search area, slow inference speed, and the incapability of being trained in an end-to-end manner. P2B [3], the first end-to-end network, employs PointNet++ [13] for feature extraction from point cloud data. It uses cosine similarity for feature matching between the search area and template, enhancing search area features with template information based on similarity. This is followed by employing the Hough vote [14] to regress the target center from the search area. While P2B greatly improves tracking accuracy and speed, it does not consider object size information in point cloud features. To incorporate the object size information, BAT [4] integrates the provided bounding box details from the first frame into the feature representation of each point, enhancing the target features. However, these methods overlook the varying importance of points at different geometric positions in representing the target. For instance, foreground points should receive more attention compared to background points.

PTT [6] is the first to introduce the Transformer [15] into 3D SOT. It replaces cosine similarity with a Transformer-based method, utilizing self-attention to independently weigh template and search area features, and then employing cross-attention for feature matching, thereby obtaining precise attention features and enhancing tracking accuracy. Since then, the Transformer has been widely used in 3D SOT. PTTR [5] introduces a relation-aware sampling module, which retains more points relevant to the template. Furthermore, the prediction refinement module further refines the prediction results, effectively predicting high-quality 3D tracking results in a coarse-to-fine manner.

Recently, there has been a growing interest in motion feature modeling methods based on Bird’s Eye View (BEV). Both V2B [7] and PTTR++ [16] focus on object tracking based on BEV. V2B first obtains a 3D voxel feature map by voxelizing the search area, then performs max-pooling along the Z-axis to obtain dense Bird’s Eye View (BEV) feature maps, and finally conducts object tracking on these BEV feature maps. Unlike V2B, PTTR++ introduces a BEV branch to extract BEV feature maps directly from the raw point cloud and then fuse with the point branch features to obtain the final feature maps for the subsequent tracking. Both methods have achieved satisfactory improvements.

There are many methods, including the 3D proposal generation process, and the quality of the 3D proposal has a great impact on the tracking results. GLT-T [17] focuses on the 3D region proposal network and mentions that using a single seed point feature as the cue for offset learning in VoteNet [14] prevents high-quality 3D proposals from being generated, so it proposes a novel voting scheme to produce higher-quality 3D proposals, and at the same time designs a simple and effective training strategy to train the module. CXTrack [10] uses contextual information across two consecutive frames to improve the tracking results. OSP2B [18] and DMT [19] involve some limitations in two-stage networks, such as tedious hyperparameter tuning, task misalignment, and inaccurate target-specific detection, which limits tracking performance. Toward these concerns, they propose simple yet effective one-stage methods that strike a new balance between efficiency and accuracy.

The motion-centric paradigm, first proposed by [1], simply leverages the motion feature to efficiently track the specified target by estimating the motion between consecutive pairs of point clouds. Although avoiding the commonly used twin network paradigm, it shows superior performance. However, it requires an extensive dataset of varied motion patterns to train the network for robust tracking performance and generalization, which is naturally costly.

Our CDTracker follows the matching-based Siamese paradigm, but in the feature matching module, we collaborate cosine similarity and an attention mechanism to induce more discriminative features, and then achieve the goal of robust feature matching. We introduce a relatively dense sampling module that retains more points of interest, thereby further stabilizing the tracking performance.

Deep Learning on Point Clouds. With the introduction of PointNet [20] and PointNet++ [13], deep learning on point clouds has been rapidly developed. Existing methods can be mainly divided into graph-based [21,22], voxel-based [23,24], and point-based [3,20,25,26]. Graph-based methods are not suitable for sparse point clouds because few points do not provide enough local geometric information to construct a graph. Voxel-based methods divide the point cloud into regular voxels and then use 3D CNNs to extract voxel features. Compared to voxel-based methods, point-based methods take the whole raw point cloud as input and extract a small number of key points and their features via some SA layers, making them more efficient. As a result, in 3D SOT, point-based networks are typically used as the backbone for feature extraction. In addition, to compare the similarities or differences between features extracted from the template and search regions, 3D SOT often employs a Siamese network architecture, consisting of two identical point-based networks that share weights. For the same purpose mentioned above, we utilize the Siamese Point Transformer to generate discriminative point features in the feature extraction stage, which has demonstrated the significant advantage of capturing long-range contextual information for discriminative learning.

Attention Mechanism. The attention mechanism in deep learning models encourages the network to focus on information that is more important for the current task while paying less attention to non-critical information. Self-attention is a variation of the attention mechanism that excels at capturing internal relationships within data or features. Cross-attention involves two different sequences, and the elements of each sequence are associated with all the elements of the other sequence to capture the dependency between the two sequences. Transformer [15] is the first to utilize self-attention mechanisms to capture long-range dependencies in language sequences. Subsequently, the Transformer architecture has gradually been adapted for various visual tasks, including semantic segmentation [27,28], object detection [29,30,31], and object tracking [32,33]. In the realm of point cloud processing, Point Transformer [34,35] is the first to utilize the self-attention mechanism within the local neighborhoods of point clouds, enabling the extraction of local features for 3D semantic segmentation. Drawing inspiration from Point Transformer, various 3D vision tasks have incorporated the Transformer architecture to achieve impressive results, including 3D object detection [36,37] and 3D object tracking [8]. Taking inspiration from these works, we also incorporate the attention mechanism into the feature matching process. This allows us to assign weights to point features, thereby enhancing internal associations. Together with cosine similarity, our coarse-to-fine feature matching module could alleviate tracking failures from abrupt appearance changes, enhancing the robustness of feature matching.

3. Method

Our CDTracker consists of four main modules: feature extraction, coarse-to-fine feature matching, relatively dense sampling, and object localization, as shown in Figure 2. The feature extraction module follows the Siamese Point Transformer network to extract features from the template and search area. The coarse-to-fine feature matching module collaborates cosine similarity and an attention mechanism to grasp point cloud similarity and then generate discriminative features. The relatively dense sampling module utilizes the idea of segmentation to retain more target points in the search area. The object localization module employs the bird’s eye view (BEV) in capturing motion features to predict the tracking results.

3.1. Feature Extraction

Most of the current methods follow the Siamese paradigm based on appearance matching and use PointNet [20] or PointNet++ [13] as feature extraction backbones. These backbones cannot learn discriminative features that should be assigned different weights based on the points’ locations. Additionally, the Siamese Point Transformer in STNet [8] has demonstrated a significant advantage in capturing long-range contextual information for discriminative learning. Therefore, we follow this approach to generate discriminative point features through the Siamese Point Transformer.

Specifically, the template point cloud is represented as $T_c\in {\mathbb{R}}^{N_t\times 3}$, and the search area point cloud is represented as $S_c\in {\mathbb{R}}^{N_s\times 3}$, where $N_t$ and $N_s$ denote the numbers of template points and search area points, respectively, and 3 denotes the dimension of the point coordinates. The template features after the feature extraction module are denoted as $T\in {\mathbb{R}}^{N_t\times C}$, and the search area features are denoted as $S\in {\mathbb{R}}^{N_s\times C}$, where C denotes the feature dimension.

3.2. Coarse-to-Fine Feature Matching

Regarding feature matching for the single-object tracking task, previous methods [3,4,7] employed cosine similarity to compute the similarity between template features and search area features. With the proposal of the Transformer, Refs. [5,8] integrated the attention mechanism into feature matching, assigning different attention weights to different features, which significantly improved performance compared to cosine similarity. Intuitively, embedding cosine similarity into the search area features first and then assigning different feature weights can induce more discriminative features. Therefore, we have designed a hybrid similarity learning mechanism that combines cosine embedding and attention assignment for coarse-to-fine feature matching.

3.2.1. Coarse Feature Matching

Here we follow V2B [7] in utilizing cosine similarity and feature distance to learn similarity and discrepancy, respectively. Before the feature matching module, we input the point cloud features of the search area into two independent MLPs (multi-layer perceptrons, types of artificial neural networks consisting of multiple layers of interconnected nodes where each node uses a nonlinear activation function) to obtain two feature subsets, $S_1\in {\mathbb{R}}^{\frac{N_s}{2}\times C}$ and $S_2\in {\mathbb{R}}^{\frac{N_s}{2}\times C}$.

To obtain the similarity between the template and the search area, the cosine similarity between the template point cloud and the subset $S_1$ of the search area is first calculated to obtain the similarity map. The similarity map is defined as follows:

$$Sim\in {\mathbb{R}}^{N_t\times \frac{N_s}{2}}=Cosine(T,S_1)$$

(1)

where $Cosine(·,·)$ denotes the cosine similarity between two features. $N_t$ and $\frac{N_s}{2}$ represent the number of point features in T and $S_1$, respectively. $Sim$ denotes an $N_t$ by $\frac{N_s}{2}$ similarity map, where each entity ${sim}_{ij}$ represents the cosine similarity between $t_i$ and $s_j$. Each point in the search region subset $S_1$ with the maximum cosine similarity value is the most similar to a point in the template. We then gather the features and coordinates of the corresponding template points along with the maximum cosine similarity and the features of the search area. These are provided to an MLP, which embeds the template features into the search area features. We formulate this as follows:

$$S_1^{\prime }=MLP\left(concat\left[S_1,{Sim}_k,T_k,X_k\right]\right)\in {\mathbb{R}}^{\frac{N_s}{2}\times C}$$

(2)

where k is the subscript of the point features in T with the maximum cosine similarity related to the feature in $S_1$. ${Sim}_k$ denotes the corresponding maximum similarity value. $X_k$ denotes the coordinates of the corresponding template points. Finally, we obtain the similarity feature set $S_1^{\prime }\in {\mathbb{R}}^{\frac{N_s}{2}\times C}$.

Apart from the similar features, the discrepancy features between the template and the search area are important and have important contributions to the subsequent object localization. The discrepancy features are given by the following:

$$S_2^{\prime }=MaxPool\left(MLP(T·\left(MLP(T-S_2)\right))\right)\in {\mathbb{R}}^{\frac{N_s}{2}\times C}$$

(3)

where $MaxPool(·)$ is the max-pooling operation (max-pooling is a downsampling technique that reduces the dimensions of the input by dividing it into non-overlapping regions and taking the maximum value from each region) and $MLP(T-S_2)\in {\mathbb{R}}^{N_t\times \frac{N_s}{2}\times C}$ represents the weights between the corresponding feature vectors in T and $S_2$, respectively. The discrepancy feature set $S_2^{\prime }\in {\mathbb{R}}^{\frac{N_s}{2}\times C}$ is obtained by assigning weights to the corresponding template features. Finally, we concatenate the similarity feature set $S_1^{\prime }$ and discrepancy feature set $S_2^{\prime }$ to obtain the coarse feature map $S^{\prime }\in {\mathbb{R}}^{N_s\times C}$.

3.2.2. Fine Feature Matching

Due to the large appearance variation between the template and the search area, learning the similarity between the template and the search area by just calculating cosine similarity and discrepancy is not a very effective way. Hence, we apply the attention mechanism in [8] to obtain higher-quality fusion features and mitigate the impact caused by the appearance variation between the template and the search area.

Cross-attention. Given the template features and the search area features after the initial similarity learning, cross-attention is applied to further learn the similarity between them and fuse them. For the template feature $T\in {\mathbb{R}}^{N_t\times C}$ and the search area feature $S^{\prime }\in {\mathbb{R}}^{N_s\times C}$, the following cross-attention is used:

$$F_s=CrossAttention(S^{\prime },T,T+X_t)$$

(4)

where $X_t\in {\mathbb{R}}^{N_t\times C}$ denotes the position embedding of the template, and $S^{\prime },T,T+X_t$ denote the query, key, and value in the attention mechanism, respectively.

Self-attention. The template information is further enhanced by using self-attention. First, for each point in the search area, we find the K closest neighboring points in the feature space constructed on the cross-attention output $F_s$, denoted as $G_i^s\in {\mathbb{R}}^{K\times C},i=1,2,\dots ,N_s$. We further update the search area point features using the self-attention mechanism:

$$f_i^{s^{\prime }}=SelfAttention(f_i^s+x_i^s,G_i^s+Z_i^s,G_i^s+Z_i^s)$$

(5)

where $x_i^s$ and $Z_i^s$ denote the location embedding of the corresponding point and its K-NN neighborhood, respectively. The final feature map is denoted as $F^{s^{\prime }}\in {\mathbb{R}}^{N_s\times C}$.

3.3. Relatively Dense Sampling

Sparsity is still one of the main reasons for inaccurate tracking results. We attempt to improve the relative density of the target points to mitigate the sparsity. Inspired by [9], we introduce the sampling module and apply the idea of segmentation to retain more of the target points. This improves the relative density of the target points and provides favorable conditions for stable object localization.

After obtaining the fused search area feature set $F^{s^{\prime }}\in {\mathbb{R}}^{N_s\times C}$, we feed it into an MLP to predict a score for each point, which represents the probability that each point is a target point. Meanwhile, the ground truth bounding box can be used to generate smooth one-hot labels $Y^{\prime }={\left\{y_i^{\prime }\right\}}_{i=1}^{N_s}$ for supervision, which can be formulated as follows:

$$y_i^{\prime }=\left\{\begin{array}{c}\hfill y_i-\epsilon if{y}_i==1,\\ \hfill y_i+\epsilon otherwise.\end{array}\right.$$

(6)

where $y_i=1$ when point i is in the ground truth bounding box; otherwise $y_i=0$. $\epsilon$ is the hyperparameter, which is set to $0.2$ in the experiment. Considering the imbalance in the number of foreground and background points, focal loss [38] is applied for supervision, which can be expressed by the following equation:

$$L_{cls}=-\sum y_i^{\prime }{(1-y_i^t)}^{\alpha }log{y}_i^t$$

(7)

$$y_i^t=\left\{\begin{array}{c}\hfill {\widehat{y}}_{i}if{y}_i==1,\\ \hfill 1-{\widehat{y}}_{i}otherwise.\end{array}\right.$$

(8)

where ${\widehat{y}}_i$ is the predicted score and $\alpha$ is the hyperparameter, which is set to 2 in the experiment.

Finally, points with the highest M scores are retained and grouped as the center of the ball query, and then aggregated with the k neighboring features, which denote $F^n\in {\mathbb{R}}^{M\times k\times C}$. Then after aggregation, we obtain the fusion features $F\in {\mathbb{R}}^{M\times \left(3+C\right)}$, where 3 denotes the dimension of retained point coordinates.

Based on the fusion feature, we use object localization in [7] to regress the target center and yaw angle (see Figure 2).

4. Experiment

4.1. Experiment Setting

Datasets. In the experiment, we use the KITTI [11] and Waymo [12] datasets for evaluation. KITTI is a commonly used challenging real-world computer vision benchmark in autonomous driving scenarios and is often used to evaluate tasks such as stereo, 3D object detection, and tracking. The 3D object tracking task contains 21 training LiDAR sequences and 29 test LiDAR sequences. Waymo is one of the most important and large-scale datasets in the field of autonomous driving, containing 798 training sequences and 202 validation sequences, 12.6 million 3D bounding box tags with tracking IDs. Specifically, we train and test on KITTI and test only on Waymo to verify the generalization ability of our network. For the KITTI dataset, due to the labels of test data not being open, we follow the settings of [3] and divide the training set of KITTI into three parts: sequences 0~16 for training, 17~18 for validation, and 19~20 for testing. The Waymo dataset is classified as easy, medium, and hard, according to the number of points in the first frame. Due to the limitation of our experimental conditions, only 150 scenes are randomly selected in the validation set for testing on the Waymo dataset.

Evaluation metrics. We use the success and precision defined in one-pass evaluation [39] as the evaluation metrics for 3D single-object tracking. Specifically, the success is defined as the IoU between the predicted boxes and the ground truth, and the precision measures the AUC (area under the curve) of the distance between the prediction box center and the ground truth box center from 0 to 2 meters.

Implementation details. For the original input point number, we set the template point number $N_t=512$ and the search area point number $N_s=1024$. After the feature extraction module, the feature dimension is set to $C=32$, so the size of the template feature T and search area feature S are $512\times 32$ and $1024\times 32$, respectively. We set the reserved points $M=512$ in the relatively dense sampling part, resulting in an output feature map size of $512\times 32$. We also try setting the retained points to $M=128$, and a comparative analysis of the two results will be provided in the ablation experiment in Section 4.3.

4.2. Results

Quantitative results. We test our network on the KITTI [11] dataset and compare it with other 3D single-object tracking networks, including SC3D [2], P2B [3], BAT [4], PTT [6], PTTR [5], V2B [7], STNet [8], DMT [19], OSP2B [18], and GLT-T [17]; the results are shown in

Table 1. Performance comparison on the KITTI dataset. Success/precision are used for evaluation. The results are produced based on the official implementation.

Methods	Car	Pedestrian	Cyclist	Van
SC3D [2]	41.3/57.9	18.2/37.8	41.5/70.4	40.4/47.0
P2B [3]	56.2/72.8	28.7/49.6	32.1/44.7	40.8/48.4
BAT [4]	60.5/77.7	42.1/70.1	33.7/45.4	52.4/67.0
PTT [6]	67.8/81.8	44.9/72.0	37.2/47.3	43.6/52.5
PTTR [5]	65.2/77.4	50.9/81.6	65.1/90.5	52.5/61.8
V2B [7]	70.5/81.3	48.3/73.5	40.8/49.7	50.1/58.0
STNet [8]	70.6/82.3	48.0/71.9	64.6/91.7	54.0/62.5
DMT [19]	66.4/79.4	48.1/77.9	70.4/93.6	53.3/65.6
OSP2B [18]	67.5/82.3	53.6/85.1	65.6/90.5	56.3/66.2
GLT-T [17]	68.2/82.1	52.4/78.8	68.9/92.1	52.6/62.9
CDTracker (Ours)	71.7/83.1	49.2/75.9	71.2/93.2	51.7/62.3

Bolded data indicate the best results for the same indicator.

. Note that the STNet results in the table are not the results listed in the paper, but the test results after retraining, and the other network results are taken from the corresponding papers. Compared with the baseline STNet, for the car category, our network improves the success and precision from 70.6% to 71.7% and 82.3% to 83.1% with a gain of about 1.1% and 0.8%, respectively. (Considering the limited number of scenes in the KITTI dataset, we conducted additional cross-validation experiments. We changed the settings for dataset partitioning as follows: 0~14, 17, 18 for training, 15~16 for validation, and 19~20 for testing, yielding a result of 71.3/83.1 for the car category). In the KITTI dataset, the car category has the largest amount of training data, supporting our network to perform best in this category. We achieve a more significant improvement in the cyclist category, with success and precision rates improving from 64.6% to 71.2% and 91.7% to 93.2%, respectively (improvements of 6.6% and 1.5%, respectively). We manually reviewed the KITTI dataset and found that the cyclist category has fewer distractors, which has less impact on the tracking results. However, the Pedestrian category contains many distractors (pedestrians usually appear in groups) around the target object, which causes our method to perform poorly on Pedestrians compared to other methods. Our design does not handle this difficult distraction scenario very well. Our results are down in the van category compared to the baseline. We consider the possible reason for this to be that there is less training data on the van category, which limits the performance of our network in this category.

In order to verify the generalization ability of our network, we also test on the Waymo [12].

Table 2. Performance comparison on the Waymo dataset. Success/precision are used for evaluation.

	Vehicle				Pedestrian
Methods	Easy	Medium	Hard	Mean	Easy	Medium	Hard	Mean
P2B [3]	57.1/65.4	52.0/60.7	47.9/58.5	52.6/61.7	18.1/30.8	17.8/30.0	17.7/29.3	17.9/30.1
BAT [4]	61.0/68.3	53.3/60.9	48.9/57.8	54.7/62.7	19.3/32.6	17.8/29.8	17.2/28.3	18.2/30.3
V2B [7]	64.5/71.5	55.1/63.2	52.0/62.0	57.6/65.9	27.9/43.9	22.5/36.2	20.1/33.1	23.7/37.9
STNet [8]	66.7/73.8	58.6/67.4	55.4/65.6	60.6/69.2	27.6/42.9	23.2/37.2	22.3/36.7	24.5/39.1
CDTracker (Ours)	66.8/74.5	56.6/65.3	55.9/66.2	60.1/69.0	27.2/42.3	24.8/39.8	23.1/38.8	25.1/40.4

Bolded data indicate the best results for the same indicator.

shows the results for the vehicle and pedestrian categories in the Waymo dataset. It can be observed that our method outperforms the baseline in the easy and hard scenarios within the vehicle category, and in the medium and hard scenarios within the pedestrian category. But our method falls below the baseline on the Mean metric in the vehicle category. We attribute this to having conducted experiments on only 150 validation scenes due to limitations in our experimental conditions, whereas the baseline uses a full validation set. The results on the Waymo dataset demonstrate the effectiveness of the method for unencountered scenarios.

Visualization results. In order to show the tracking effect of the network, some visualization results of our network are shown in this section and some visualization results are shown in Figure 3. The visualization results show that most tracking results are very close to the ground truth bounding box, which is a very intuitive demonstration of the effectiveness of our network for single-object tracking tasks. The tracking results are also visualized and compared with those of STNet. As shown in Figure 3, it is obvious that our network tracks better than the baseline network at T = 20 frames of car 1 and T = 10 frames of car 2. Therefore, when the number of point clouds is small, the tracking effect of this network is better and more robust. When the number of point clouds is larger and the object shape is more complete, the performance of our network is similar to that of the baseline network. To further show the tracking effectiveness of our network, a partial comparison of the tracking results with PTTR and P2B is added, as shown in Figure 3. Compared with PTTR and P2B, our network has better stability in some cases.

4.3. Ablation Study

In this section, the effectiveness of two designs (coarse feature matching based on cosine similarity and relatively dense sampling) are ablated separately to verify the boosting of the baseline network, and the results are shown in

Table 3. Ablation studies on Design 1 (coarse feature matching), Design 2 (relatively dense sampling), the number of retained points M.

Design 1	Design 2	M	Car	Cyclist
		M = 512	70.6/82.3	64.6/91.7
√		M = 512	70.7/82.3	70.5/93.1
	√	M = 512	70.9/82.1	70.2/93.0
√	√	M = 512	71.7/83.1	71.2/93.2
√	√	M = 128	66.1/77.9	-

Bolded data indicate the best results for the same indicator. “√” denotes the module is added.

. The experimental results show that both the coarse feature matching based on cosine similarity and the relatively dense sampling introduced in this paper improve the tracking performance of the network. The most significant improvement is observed when both modules are added at the same time.

To analyze the impact of the feature dimension on the experimental results, we conducted ablation experiments on different values of C (C = 16, 32, 64). The results are shown in

Table 4. Influence of different feature dimensions, C, after the backbone of the car category for the KITTI dataset.

Feature Dimension C	Success	Precision
C = 16	62.8	76.2
C = 32	71.7	83.1
C = 64	70.6	82.7

Bolded data indicate the best results for the same indicator.

. The experiments show that the best effect is achieved when the feature dimension is 32, the result is relatively poor when the feature dimension is 16, and while the results with a dimension of 64 are similar to those of 32, the training speed decreases due to the increased number of parameters. Therefore, a feature dimension of 32 is the best choice to balance accuracy and efficiency.

We also conduct ablation experiments on the number of retained points in relatively dense sampling, using $M=128$ and $M=512$, respectively. The experimental results for the car category are 66.1%/77.9% and 71.7%/83.1%, respectively. As shown in Table 3, the tracking results improve with a higher number of retained points, as the sparsity of the point cloud remains one of the main issues causing inaccurate tracking. Retaining more points enhances the relative density of the target points and alleviates the point cloud sparsity problem. In addition, retaining some key background points also contributes to ensuring tracking accuracy. When the number of retained points is small, it often results in a scenario where most of these points are foreground points with only a few being background points, leading to suboptimal tracking results.

In the following, we also visualize the retained points before and after sampling for some objects with the number of retained points $M=512$. Visual observations and calculations of the occupation ratio reveal that relatively dense sampling can retain more target points, thus increasing the relative density of these points and providing more favorable information for subsequent target localization. Specifically, as shown in Figure 4, when the target object is a car, there is a more obvious target point retention (the ratios of positive points increase from 38% and 22% to 72% and 43%, respectively). When the target objects are cyclists and pedestrians, the ratios of positive points increase from 10% and 6% to 11% and 9%, respectively. Even though there are more interfering objects nearby, there is still a slight increase, which visually verifies the effectiveness of this module.

4.4. Limitations

Our network has some shortcomings. When the number of point clouds is too small, our network often fails to track the correct object, tending instead to incorrectly latch onto objects very close to the target or to surrounding noise points. As shown in Figure 5, the point cloud in the real surround frame of the first frame is very sparse, and our network incorrectly tracks to the next area where there are more point clouds. To the best of our knowledge, other models have encountered the same problem; they mentioned some failure cases in cases of extremely sparse point clouds [3,10,40], where the tracking tends to drift to background points. To address this problem, we considered some potential solutions. First, making the point cloud denser through point cloud upsampling is an intuitive approach. Additionally, using other modal information, such as 2D images, can serve as an enhancement to obtain more robust initial features.

5. Conclusions

In this paper, we design a coarse-to-fine feature matching module to match the features of the template and the search area. Unlike previous methods that only used cosine similarity or an attention mechanism, we apply both methods to perform feature matching in a coarse-to-fine way, which enhances the robustness of feature matching and alleviates tracking failures from abrupt appearance changes. In addition, to address the problem of inaccurate tracking results caused by sparse point clouds, we introduce the relatively dense sampling module based on point cloud segmentation, which densifies the target points by segmenting the foreground points and background points. Experiments on KITTI and Waymo verify the effectiveness of our CDTracker and each design. But when the point cloud is too sparse, the network may mislocate the nearby interfering objects and fail to track the target correctly. In the future, we will continue to study related problems and explore more effective solutions.