Vehicle Re-Identification Based on UAV Viewpoint: Dataset and Method

Abstract

High-resolution remote sensing images bring a large amount of data as well as challenges to traditional vision tasks. Vehicle re-identification (ReID), as an essential vision task that can utilize remote sensing images, has been widely used in suspect vehicle searches, cross-border vehicle tracking, traffic behavior analysis, and automatic toll collection systems. Although there have been a large number of studies on vehicle ReID, most of them are based on fixed surveillance cameras and do not take full advantage of high-resolution remote sensing images. Compared with images collected by fixed surveillance cameras, high-resolution remote sensing images based on Unmanned Aerial Vehicles (UAVs) have the characteristics of rich viewpoints and a wide range of scale variations. These characteristics bring richer information to vehicle ReID tasks and have the potential to improve the performance of vehicle ReID models. However, to the best of our knowledge, there is a shortage of large open-source datasets for vehicle ReID based on UAV views, which is not conducive to promoting UAV-view-based vehicle ReID research. To address this issue, we construct a large-scale vehicle ReID dataset named VRU (the abbreviation of Vehicle Re-identification based on UAV), which consists of 172,137 images of 15,085 vehicles captured by UAVs, through which each vehicle has multiple images from various viewpoints. Compared with the existing vehicle ReID datasets based on UAVs, the VRU dataset has a larger volume and is fully open-source. Since most of the existing vehicle ReID methods are designed for fixed surveillance cameras, it is difficult for these methods to adapt to UAV-based vehicle ReID images with multi-viewpoint and multi-scale characteristics. Thus, this work proposes a Global Attention and full-Scale Network (GASNet) for the vehicle ReID task based on UAV images. To verify the effectiveness of our GASNet, GASNet is compared with the baseline models on the VRU dataset. The experiment results show that GASNet can achieve 97.45% Rank-1 and 98.51% mAP, which outperforms those baselines by 3.43%/2.08% improvements in terms of Rank-1/mAP. Thus, our major contributions can be summarized as follows: (1) the provision of an open-source UAV-based vehicle ReID dataset, (2) the proposal of a state-of-art model for UAV-based vehicle ReID.

1. Introduction

Along with the advancement of remote sensing technology, the spatial resolution of remote sensing images has reached the “sub-meter level” [1,2], while the remote sensing images through Unmanned Aerial Vehicles (UAVs) can obtain even higher resolution. Furthermore, the high mobility and flexibility of UAVs bring remote sensing images with rich viewpoints and a wide range of scale variations. Although those remote sensing images provide rich information for vision tasks, it also brings challenges.

The focus of this work is the vehicle Re-IDentification (ReID) task through UAVs. As an important vision task, Vehicle ReID has been widely used in a wide range of applications, such as video surveillance [3], intelligent transportation [4], and urban computing [5]. The goal of vehicle ReID is to discover, localize, and track the queried target vehicle from a large volume of vehicle data. Most of the existing vehicle ReID works are based on fixed surveillance cameras, as shown in Figure 1a. Fixed surveillance cameras cannot provide multi-view and multi-scale vehicle images due to the fixed locations and limited viewing angles. On the contrary, the UAV-based remote sensing images (as shown in Figure 1b) have the potential to improve the performance of vehicle ReID models because they can bring richer information to the vehicle ReID task. However, the UAV-based vehicle ReID works are still in their exploratory stage. To the best of our knowledge, there are only two UAV-based datasets for the vehicle ReID task, namely, VRAI [6] and UAV-VeID [7], none of which is fully open source. Therefore, to facilitate future UAV-based vehicle ReID works, this paper constructs a UAV-based large-scale open-source vehicle ReID dataset named Vehicle ReID based on UAV (VRU).

To obtain multi-view and multi-scale vehicle images in realistic scenarios, we deployed five UAVs to shoot videos of vehicles at multiple scenes, such as highways, intersections, and parking lots, in various periods, such as morning, noon, afternoon, and night, with numerous weather conditions, such as sunny, cloudy, and drizzly. More than 15 h of video data were selected from the shot videos to produce the dataset of multi-view and multi-scale vehicle images. Finally, the VRU dataset consists of 172,137 images from 15,085 vehicle instances. Compared with the existing UAV-based vehicle ReID dataset, the VRU dataset has the largest data volume.

Since most of the existing vehicle ReID models are trained and tested through vehicle images taken by fixed surveillance cameras, it is difficult to adapt them to UAV-based multi-view and multi-scale vehicle images. Therefore, we propose a Global Attention and full-Scale Network (GASNet), which can extract view and scale-invariant features from multi-view and multi-scale vehicle images, thus improving the recognition capability of reidentifying vehicles. To verify the effectiveness of GASNet, we compared GASNet with two mainstream ReID models, namely MGN [8] and SCAN [9]. The experimental results show that GASNet achieves 97.45% Rank-1 and 98.51% mAP, respectively, outperforming MGN [8] and SCAN [9]. Moreover, through the ablation experiments, we can conclude that the global attention module can effectively utilize multi-view information, while the full-scale module can aggregate information from different scales. Through integrating the two modules, GASNet can outperform those baseline models on the VRU dataset.

The main contributions of this work can be summarized as follows.

(1) We provide a large open-source dataset, named VRU, for vehicle ReID tasks from the perspective of UAVs. Benefiting from the maneuverability and flexibility of UAVs, VRU is characterized by rich scales and diverse views. In addition, to reflect the practical scenarios in the real world, the vehicle images collected for VRU are shot in multiple weather conditions, multiple time periods, and multiple urban traffic scenes. Therefore, VRU will bring numerous challenges to the vehicle ReID task and may inspire novel research works.

(2) We propose a vehicle ReID model named GASNet, which integrates the global attention and full-scale convolution modules to take advantage of the rich scales and diverse views in the VRU dataset.

(3) We conduct comprehensive experiments to evaluate the effectiveness of the proposed GASNet model. The experimental results show that GASNet can achieve 97.45% Rank-1 and 98.51% mAP, respectively, and outperforms the state-of-art vehicle ReID models. This illustrates that GASNet can effectively address the multi-view and multi-scale challenges brought by the UAV-based vehicle ReID images.

The rest of this work is organized as follows. Section 2 reviews the related works. Section 3 describes the VRU dataset, and presents the GASNet model. Section 4 presents the experimental results of evaluating the proposed datasets and model. Section 5 discusses the significance and implication of the proposed dataset and model. Section 6 concludes this work and discusses the future works.

2. Related Works

In recent years, vehicle ReID has received increasing attention. Usually, a vehicle ReID model first utilizes a convolutional neural network to extract vehicle features and then employs metric learning to optimize the distance among the extracted features [9,10,11,12]. However, most metric learning methods do not consider the variations of viewpoints and scales, although significant visual appearance changes may reduce the effectiveness of metric learning methods. Therefore, we propose a new vehicle ReID method for the viewpoint and scale changes.

Concerning the vehicle, ReID datasets, most existing vehicle ReID datasets are obtained from fixed surveillance cameras. For example, Liu et al. [11] constructed a relatively small vehicle ReID dataset named VeRi, which consists of about 40,000 images from 619 vehicles captured by 20 surveillance cameras. The dataset is also labeled with luggage racks, vehicle types, colors, and brands. Then, Liu et al. [13] proposed VeRi-776 dataset by extending VeRi dataset in terms of data volume and new tags associated with license plates and spatiotemporal traces. During the same period, Liu et al. [14] proposed a dataset named VehicleID, which consists of 221,763 images of 26,267 vehicles. Compared with the aforementioned datasets, VERI-Wild [15] dataset, which is proposed by Lou et al., has a larger volume of data, consisting of 416,314 images from 40,671 vehicles. The datasets mentioned above were all captured by fixed surveillance cameras. However, this type of dataset is limited by the data collection equipment, which suffers from the insufficient diversity of vehicle viewpoints and scales. As for the visual task, the images from various viewpoints may be extremely different in visual appearance, while the scale variation may also lead to significant changes in feature distribution [16]. Thus, the variation of viewpoints and scales is of great value to improve the generalization performance of ReID models.

To increase the viewpoint diversity of vehicles in fixed surveillance camera scenarios, Zhou et al. [17] use Generative Adversarial Networks (GANs) to generate vehicle images with various viewpoints. However, the fidelity and diversity of the multi-view vehicle images generated by GAN are still much lower than the authentic multi-view images. Therefore, it is desirable to build a vehicle ReID dataset, consisting of authentic vehicle images with multi-view and multi-scale features. As aforementioned, the remote sensing images captured by UAVs can provide multi-view and multi-scale features for vehicles. However, to the best of our knowledge, there are only two existing UAV-based vehicle ReID datasets, namely, VRAI [6] and UAV-VeID [7]. VRAI dataset is the first UAV-based vehicle ReID dataset, containing 137,613 images from 13,022 vehicles. The UAV-VeID dataset is smaller than the VRAI dataset, and it consists of only 41,917 images from 4601 vehicles. Although these two datasets were obtained by UAV, neither of them is fully open source. In contrast, our VRU dataset not only has the largest data volume but also will be completely open-source.

A few works have utilized multi-scale and multi-view information in ReID datasets to improve the ReID performance. For example, from the multi-scale perspective, MGN [8] model extracts the global features and local features from an object through a multi-branch network; Zhou et al. [18] use multiple convolutional streams to detect multi-scale features; Chen et al. [19] formulate a novel deep pyramid feature learning CNN architecture for multi-scale feature fusion. From the perspective of the multi-view, Wang et al. [20] set 20 key points for a vehicle instance and clustered all key points into four view-based region masks to obtain orientation invariant feature vectors in different views. However, none of these approaches consider all perspectives simultaneously. Therefore, this paper proposes the GASNet model, which can extract vehicle features with view invariance and scale invariance, thus making full use of the information provided by the UAV vehicle ReID dataset. In addition, the problem of information imbalance in different spatial locations and channels also affects model performance [9]. GSANet draws on the idea of SCAN [9] and uses channel attention mechanism and spatial attention mechanism to solve the problem of information imbalance.

The existing works on vehicle ReID tasks have identified numerous techniques to speed up the training process and improve the performance of the vehicle ReID models. For example, the batch size has a significant impact on the accuracy of a vehicle ReID model. Specifically, the larger the batch size, the better the model that can be trained. In this paper, we also compare our GASNet with the baseline under various batch sizes. It is also found that the selection of loss functions also affects the model performance [21]. Most of the vehicle ReID models utilize ID Loss and Triplet Loss jointly for model training [22,23,24]. However, the optimization objectives of these two losses lie in different feature spaces, which may lead to the trade-off between the two losses, i.e., the decrease of one loss may be at the cost of the increase of the other loss during the training process. To address this issue, Luo et al. [21] proposed a BNNeck module, which adds a Batch Normalization (BN) layer between the ID Loss and the Triplet Loss. The normalization of features can reduce the influence between the ID Loss and the Triplet Loss, which in turn makes it easier for the two losses to converge simultaneously. Thus, the BNNeck module can speed up the training process and improve the model performance. To enhance the performance of the GASNet model, this work also adopts the aforementioned training tricks.

3. Materials and Methods

To take full advantage of high-resolution remote sensing images for vehicle ReID research, we construct a vehicle ReID dataset based on the UAV perspective. Based on this, we propose a new model to address the multi-view and multi-scale challenges of this dataset. This section will introduce our dataset and method.

3.1. Dataset

Currently, the data source for the vehicle ReID task is mainly road surveillance cameras. However, due to the fixed shooting positions, it is difficult to collect vehicle images from different viewpoints with various scales.

3.1.1. Data Collection

Recently, due to the breakthroughs in UAVs in terms of flight time, automatic control algorithms, and wireless data transmission, it becomes possible to construct large-scale UAV-based datasets for the vehicle ReID task. Furthermore, since UAVs have better mobility and flexibility than surveillance cameras, this paper uses UAVs to construct a vehicle image dataset, named VRU, for the vehicle ReID task.

To collect vehicle image data under various scenes, 5 ‘DJI Mavic 2 Pro’ UAVs are deployed. The frame rate of the UAV used for data collection is 30 frames per second, the resolution of the collected image is 3840 × 2160 pixels, and the format of the shot video is MOV. The configuration of the UAVs and the attached cameras are enumerated in

Table A9. The configuration of ‘DJI Mavic 2 Pro’.

Aircraft
Parameters	Value
Takeoff Weight	907 g
Dimensions	Folded: 214 × 91 × 84 mm Unfolded: 322 × 42 × 84 mm
Diagonal Distance	354 mm
Max Ascent Speed	5 m/s (S-mode), 4 m/s (P-mode)
Max Descent Speed	3 m/s (S-mode), 3 m/s (P-mode)
Max Speed	72 km/h (S-mode) (near sea level, no wind)
Max Service Ceiling Above Sea Level	6000 m
Max Flight Time	31 min (at a consistent 25 kph, no wind)
Overall Flight Time	25 min (in normal flight, 15% remaining battery level)
Max Flight Distance	18 km (at a consistent 50 kph, no wind)
Hovering Accuracy Range	Vertical: ±0.1 m (when vision positioning is active) ±0.5 m (with GPS positioning) Horizontal: ±0.3 m (when vision positioning is active) ±1.5 m (with GPS positioning)

All data are from DJI official website.

and

Table A10. The configuration of the attached camera.

Camera
Parameters	Value
Sensor	1″ CMOS Effective Pixels: 20 million
Lens	FOV: approx. 77° 35 mm Format Equivalent: 28 mm Aperture: f/2.8–f/11 Shooting Range: 1 m to
ISO Range	Video: 100–6400 Photo: 100–3200 (auto) 100–12,800 (manual)
Shutter Speed	Electronic Shutter: 8-1/8000 s
Still Image Size	5472 × 3648
Still Photography Modes	Single shot Burst shooting: 3/5 frames Auto Exposure Bracketing (AEB): 3/5 bracketed frames at 0.7 EV Bias Interval: 2/3/5/7/10/15/20/30/60 s (JPEG) 5/7/10/15/20/30/60 s (RAW)
Video Resolution	4 K: 3840 × 2160 24/25/30 p 2.7 K: 2688 × 1512 24/25/30/48/50/60 p FHD: 1920 × 1080 24/25/30/48/50/60/120 p
Color Mode	Dlog-M (10-bit) support HDR video (HLG 10-bit)
Max Video Bitrate	100 Mbps
Photo Format	JPEG/DNG (RAW)
Video Format	MP4/MOV

All data are from DJI official website.

(in Appendix A.2), respectively.

3.1.2. Multi-View and Multi-Scale

We design different shooting strategies for vehicles in different scenarios to ensure the viewpoint and scale diversity of the remotely sensed vehicle images. For the vehicles in parking lots, we capture vehicle instances through two UAVs that rotate from opposite directions and change their flying altitudes from time to time. For a moving vehicle, five UAVs capture the vehicle simultaneously. Four UAVs obtain the front, rear, left, and right views of the same vehicle, respectively. The remaining one rotates and shoots video within a pre-defined range of the vehicle. We set the height range of a flying UAV from 15 to 60 m. Each UAV flies at a different height, and the height difference between adjacent UAVs is more than 5 m. The shooting angle is between 40 and 80 degrees. With the above shooting strategy, we can obtain images of vehicles from different viewpoints with various scales as shown in Figure 2.

3.1.3. Multi-Scene, Multi-Time, and Multi-Weather

As the data support for the vehicle ReID task, a dataset has a great impact on the performance of a model. A good dataset should fit the data distribution in real scenarios. As for vehicle images, the types of vehicles that appear on different roads may vary greatly. For example, due to the regulations of urban traffic management, large vehicles cannot appear in the downtown area during the daytime, although they are allowed on suburban roads. For the same reason, the tricycles may appear on suburban roads rather than downtown. Therefore, as illustrated in Figure 3a, to obtain data fitting the real scenarios, we selected representative downtown/suburb scenes, such as elevated-outer-ring roads (first row), urban traffic intersections (second row), and parking lots (third row), as data collection locations.

In vision tasks, illumination variation can directly affect the recognition results of a model. The illumination changes over time. Furthermore, the types and quantity of vehicles on roads may vary at different periods. For example, during rush hour, a large number of cars appear on the urban roads. To fit the varied illumination conditions and vehicle type distribution in real scenarios, we choose four time periods to remotely sense vehicles as shown in Figure 3b, where the first, the second, the third, and the fourth rows are taken in the morning, noon, afternoon, and evening, respectively.

The weather conditions may also affect the distribution of vehicles. Moreover, the weather condition may also affect the clarity of the remote sensing images. Thus, to fit the real scenarios, this paper collects data under three weather conditions, such as sunny days, cloudy days, and drizzles, corresponding to the first to third rows in Figure 3c.

3.1.4. Data Processing

After obtaining the original videos taken by UAVs, we selected 200 video sequences, each with a duration of 5 min, as the initial data for building the dataset. Then, we extract one image per second from the video sequences. Thus, 60,000 raw images in total are extracted to further construct the UAV-based vehicle ReID dataset.

3.1.5. Data Annotation

Since the same vehicle may be captured by different UAVs at the same moment, we first align the timestamps of all the images so that the vehicle images can be annotated correctly. To improve the accuracy of the annotation, we recruit 20 volunteers who know the vehicle ReID task and the corresponding dataset to manually locate and annotate all raw images. By localizing and cropping the vehicles in the 60,000 raw images, a total of 172,137 images containing 15,085 vehicle instances were obtained. The comparison of raw images and vehicle images in the VRU dataset is shown in Figure 4.

By cropping the raw images with multiple vehicles, we obtain final vehicle images, each of which contains only one vehicle. Since each image is cropped to fit the size of the vehicle, the pixel width of an image can be approximated as the corresponding vehicle scale. Thus, we can plot the distribution of the vehicle scales, as shown in Figure 5, where we also compare the scale distribution of the VehicleID dataset and VERI-Wild dataset, both of which are collected through surveillance cameras. From Figure 5, we can observe that our VRU dataset has the largest scale diversity.

This paper also counts the number of vehicles associated with each viewpoint in the VRU dataset. The statistical results in Figure 6 show that the distribution of images corresponding to the four viewpoints is relatively uniform. This fully illustrates the viewpoint diversity of the VRU dataset.

Meanwhile, this paper also counts the number of images corresponding to each vehicle instance in the VRU dataset. The statistical results are shown in Figure 7, from which it can be observed that more than 97% of the vehicle instances have more than three images. For most of the vehicle instances, the number of images ranges from 8 to 15. In summary, the VRU dataset contains rich viewpoint and scale information, which provides data support for training a more robust vehicle ReID model.

3.1.6. Dataset Partitioning

According to the general practice of vehicle ReID dataset division, the VRU dataset is divided into a training set and three test sets, namely small, medium, and large test sets. The training set includes 80,532 images from 7085 vehicle instances. The small, medium and large test sets contain 13,920 images from 1200 vehicle instances, 27,345 images from 2400 vehicle instances, and 91,595 images from 8000 vehicle instances, respectively. In the ReID dataset, the test set usually includes the query set and the gallery set. The vehicle ReID task is to retrieve images in the gallery set that are consistent with the vehicles in the query set. Therefore, we put one image of each vehicle instance in the VRU test set into the gallery setting, and the rest of the images are used as the query set.

3.1.7. Dataset Comparison

To the best of our knowledge, the existing datasets for vehicle ReID based on UAV remote sensing images are VRAI and UAV-VeID, which are also multi-view and multi-scale. On the contrary, our dataset is not only multi-view and multi-scale, but also multi-weather, multi-light intensity, with larger data volume. This allows the images in the VRU dataset to provide richer information to the vehicle ReID models. Additionally, our dataset will be fully open source. A comparison of the VRU dataset with the VRAI and UAV-VeID datasets is shown in

Table 1. Comparison of datasets.

Datasets	VRU	UAV-VeID	VRAI
Identities	15,085	4601	13,022
Images	172,137	41,917	137,613
Multi-view	√	√	√
Multi-scale	√	√	√
Weather	√	√	×
Lighting	√	√	×
Open-source	√	×	×

.

3.2. GASNet

Most of the existing vehicle ReID methods are studied based on the data acquired by fixed surveillance cameras, which are difficult to adapt to UAV remote sensing images with multi-view and multi-scale characteristics, because the morphology of the same vehicle instance under different viewpoints has large differences, and the information provided by vehicle images varies at different scales. Therefore, to cope with the multi-view and multi-scale challenges brought by UAV remote sensing images, this paper proposes a vehicle ReID model (called GASNet) based on the global relationship-aware attention and full-scale mechanism. GASNet captures features with global information by introducing a global relationship-aware attention mechanism [25], and introduces a full-scale network [18] to correlate features of different scales, extract features in the image that do not change with viewpoint and scale, and then achieve the purpose of improving the generalization ability of the model.

3.2.1. Overall Structure of GASNet

As shown in Figure 8, the GASNet model consists of a backbone network that extracts viewpoint-invariant features and a branching network that extracts scale-invariant features. Following a common practice in ReID, we use ResNet50 [26] as the base network block and insert the global relationship-aware attention module (abbreviated as GA) block by block after the second residual block to form the backbone network, and access the full-scale module (abbreviated as FS) after the third residual block to form the branch network. A BNNeck structure is added at the end of both the backbone network and the branch network to optimize the feature distribution so that the training of the whole network can be completed faster and better.

3.2.2. Global Attention Module

Attention mechanisms can be divided into local attention and global attention based on their learned attention weights. Local attention focuses on local saliency regions of the target, and global attention grasping the overall information of the target. One of the challenges of the vehicle ReID task based on UAV remote sensing images is that the vehicle viewpoint changes more frequently. To solve this problem, it is necessary to extract vehicle features that do not change with the viewpoint, that is, to extract features that contain the overall vehicle information. For this purpose, we introduce a global attention module for relationship perception. The relationship-aware global attention module consists of a globally-aware spatial attention mechanism and a globally-aware channel attention mechanism. This module takes the features of any position on the feature map as nodes, and by emphasizing the relationship between nodes, mining the correlation and semantic information of the global scope, to extract the vehicle features that do not change with the viewing angle.

3.2.3. Full-Scale Module

Vehicle ReID datasets based on UAV remote sensing images have rich scale variation, which brings challenges as well as opportunities to improve the performance of the ReID models. To extract highly discriminative features with scale invariance, we introduce a full-scale convolutional structure. The structure consists of multiple convolutional streams with receptive fields of different sizes, and each convolutional stream pays attention to features of different scales, thereby obtaining multi-scale feature maps. The multi-scale feature maps are finally dynamically fused through the channel-based weights generated by the unified aggregation gate to obtain full-scale features. The scale-invariant vehicle features can then be captured through a full-scale convolutional structure.

4. Results

To verify the performance of the GASNet model proposed in this paper, we build the GASNet network using the PyTorch framework, train, and test it on a Tesla A100 GPU. The learning rate is set to 0.00035, Adam optimization is used, the network is constrained using a triplet loss function and a cross-entropy loss function, and all experiments are trained for 60 rounds.

4.1. Evaluation Indicators

The main metrics for evaluating a vehicle ReID model include mean Average Precision (mAP), Cumulative Matching performance Curve (CMC), and Rank-N table. Consider a query q provided to a ReID model, which retrieves a sequence of n images from the gallery, where exactly $N_q$ images match the query. Generally, the higher a matching image is in the retrieval sequence, the better the retrieval precision is. Specifically, if all the $N_q$ matched images rank the top-$N_q$ positions in the retrieval sequence, the total retrieval precision of query q reaches its highest value. Thus, assume that $G\left(k\right)$ denotes whether a matched image is at the k-th position in the retrieved sequence and $P\left(k\right)$ reflects the probability that the top-k retrieved images include the matched images, then the multiplication of $P\left(k\right)G\left(K\right)$ can represent the retrieval precision at the k-th position and ${\sum }_{k=1}^{n}P\left(k\right)G\left(K\right)$ can reflect the total retrieval precision associated with query q. Therefore, the AP of query q can be defined as the total retrieval precision divided by $N_q$, as shown in Formula (1), where mAP can be further formalized as the average of APs over all queries, which can be used to assess the overall performance of a ReID model.

$$AP\left(q\right)=\frac{{\sum }_{k=1}^{n}P\left(k\right)\times G\left(k\right)}{N_q},mAP=\frac{{\sum }_{i=1}^{M}AP\left(q_i\right)}{M}$$

(1)

In contrast to AP, which reflects the retrieval precision from the perspective of matching queries, an alternative metric (called CMC@k) intends to characterize the retrieval precision from the perspective of matching positions, as shown in Formula (2).

$$CMC@k=\frac{{\sum }_{i=1}^{M}F\left(q_i,k\right)}{M}$$

(2)

In Formula (2), $F(q_i,k)$ indicates whether query $q_i$’s matched images locate among the top k images of the retrieved sequence. Thus, CMC@k measures the average retrieval precision of all queries at the k-th position in the retrieved sequence. As a single-value metric, mAP can only reflect the performance of the ReID model as a whole, while CMC@k, as a multi-value metric with varied values of k, can reflect a ReID model’s precision distribution in terms of matching positions.

If most of the CMC@k values of two ReID models are almost the same, it is relatively difficult to compare the two models. Therefore, instead of comparing all CMC@k values, it can select only a few significant CMC@k values for comparison, which is also known as the Rank-N table. Rank-1 and Rank-5 are the most common CMC@k values, indicating the probability of matching images in the top 1 and top 5 of the retrieved sequence, respectively. Similar to most ReID works, this paper adopts mAP and Rank-N as the evaluation metrics.

4.2. Ablation Experiments

To verify that the global attention module (abbreviated as GA) and the full-scale module (abbreviated as FS) can improve the performance of the model, we conduct ablation experiments while keeping the experimental conditions unchanged. First, we construct the baseline model (abbreviated as baseline) by removing both GA and FS. Then, two comparison models (abbreviated as baseline+GA and baseline+FS) are constructed by adding GA and FS to the baseline, respectively. Finally, the performance of the baseline model, the comparison models, and the GASNet model is compared. As batch size can affect the experimental results [21], we set four batch sizes, 32, 64, 96, and 128, respectively.

4.2.1. The Ablation Experiment for GA

To verify the performance enhancement brought by the GA module, this section designs the ablation experiments for the GA module. Figure 9 shows the corresponding experimental results on the small test set of the VRU dataset. It can be seen from Figure 9 that baseline+GA outperforms the baseline model in terms of Rank-1, Rank-5, and mAP metrics along with the variation of batch sizes. This illustrates the benefit of the GA module. More specifically, the performance gap is maximized with the small batch sizes. This implies that the integration of the GA module enables the baseline to rely less on the diverse information provided by large batches, as the model can learn highly discriminative features through the integration of the global information even with small batches. Meanwhile, we also conducted experiments on the medium and large test sets in the VRU dataset. The experimental results are enumerated in

Table A1. Test results of the baseline model and the global attention model with input batch 32 on the VRU dataset.

Models	VRU	Batchsize	Rank-1 (%)	Rank-5 (%)	mAP
Baseline	Small	32	88.25	98.95	92.98
	Medium	32	83.17	97.44	89.41
	Big	32	70.23	92.04	79.77
Baseline+GA	Small	32	95.24	99.65	97.28
	Medium	32	92.84	99.13	95.68
	Big	32	86.00	97.45	91.04

,

Table A2. Test results of the baseline model and the global attention model with input batch 64 on the VRU dataset.

Models	VRU	Batchsize	Rank-1 (%)	Rank-5 (%)	mAP
Baseline	Small	64	94.24	99.70	96.78
	Medium	64	90.56	99.04	94.34
	Big	64	82.78	96.91	89.00
Baseline+GA	Small	64	96.19	99.63	97.61
	Medium	64	94.28	99.25	96.59
	Big	64	88.32	98.07	92.63

,

Table A3. Test results of the baseline model and the global attention model with input batch 96 on the VRU dataset.

Models	VRU	Batchsize	Rank-1 (%)	Rank-5 (%)	mAP
Baseline	Small	96	95.38	99.60	97.33
	Medium	96	92.86	99.23	95.77
	Big	96	85.10	97.78	90.70
Baseline+GA	Small	96	96.40	99.76	97.95
	Medium	96	94.92	99.24	96.92
	Big	96	88.99	98.26	93.11

and

Table A4. Test results of the baseline model and the global attention model with input batch 128 on the VRU dataset.

Models	VRU	Batchsize	Rank-1 (%)	Rank-5 (%)	mAP
Baseline	Small	128	96.08	99.64	97.74
	Medium	128	93.33	99.25	96.02
	Big	128	86.86	98.05	91.85
Baseline+GA	Small	128	96.93	99.63	98.20
	Medium	128	94.62	99.37	96.79
	Big	128	88.97	98.19	93.09

(in Appendix A.1), which shows similar results as that of the small test set.

4.2.2. The Ablation Experiments for FS

Figure 10 shows the experimental results of the FS module on the small test set of the VRU dataset, from which it can be seen that baseline+FS outperforms the baseline model in terms of Rank-1 and mAP metrics along with the variation of batch sizes. In addition, we also conducted experiments on the medium test set and the large test set in the VRU dataset, and the experimental results are shown in

Table A5. Test results of the baseline model and the full-scale model with input batch 32 on the VRU dataset.

Models	VRU	Batchsize	Rank-1 (%)	Rank-5 (%)	mAP
Baseline	Small	32	88.25	98.95	92.98
	Medium	32	83.17	97.44	89.41
	Big	32	70.23	92.04	79.77
Baseline+FS	Small	32	90.55	99.22	96.17
	Medium	32	90.33	98.45	93.98
	Big	32	82.24	95.90	88.21

,

Table A6. Test results of the baseline model and the full-scale model with input batch 64 on the VRU dataset.

Models	VRU	Batchsize	Rank-1 (%)	Rank-5 (%)	mAP
Baseline	Small	64	94.24	99.70	96.78
	Medium	64	90.56	99.04	94.34
	Big	64	82.78	96.91	89.00
Baseline+FS	Small	64	95.87	99.61	97.61
	Medium	64	93.70	99.22	96.18
	Big	64	87.91	97.58	92.21

,

Table A7. Test results of the baseline model and the full-scale model with input batch 96 on the VRU dataset.

Models	VRU	Batchsize	Rank-1 (%)	Rank-5 (%)	mAP
Baseline	Small	96	95.38	99.60	97.33
	Medium	96	92.86	99.23	95.77
	Big	96	85.10	97.78	90.7
Baseline+FS	Small	96	96.55	99.50	97.95
	Medium	96	94.39	99.11	96.53
	Big	96	88.84	98.01	92.92

and

Table A8. Test results of the baseline model and the full-scale model with input batch 128 on the VRU dataset.

Models	VRU	Batchsize	Rank-1 (%)	Rank-5 (%)	mAP
Baseline	Small	128	96.08	99.64	97.74
	Medium	128	93.33	99.25	96.02
	Big	128	86.86	98.05	91.85
Baseline+FS	Small	128	96.43	99.66	97.89
	Medium	128	94.76	99.10	96.76
	Big	128	89.38	98.07	93.27

in the Appendix A.1, which shows similar results as that of the small test set.

The above experiments effectively verify the performance improvement from the FS module, especially when the GPU cannot support the experiment with a large batch size. It can also be inferred that increasing the batch size to a certain extent can greatly help improve the model precision.

4.3. The Performance Comparison Experiment

To demonstrate the performance of the GASNet model, this section compares it with MGN and SCAN on the VRU dataset. MGN extracts the global and local features of a vehicle through a backbone network and two branch networks, respectively, and finally fuses the two types of features to improve a ReID model’s precision. SCAN uses both a channel-attention mechanism and a spatial-attention mechanism to optimize a ReID model’s weights and forces the model to focus on high discriminative regions to improve the model’s performance.

Based on the results of the two previous ablation experiments, we set the batch size as 128 for better performance. The results of the comparison experiments are shown in

Table 2. The performance comparison with other ReID models.

Models	VRU	Rank-1 (%)	Rank-5 (%)	mAP (%)
MGN	Small	81.72	95.08	82.48
	Medium	78.75	93.75	80.06
	Big	66.25	87.15	71.53
SCAN	Small	75.22	95.03	83.95
	Medium	67.27	90.51	77.34
	Big	52.44	79.63	64.51
Baseline	Small	96.08	99.64	97.74
	Medium	93.33	99.25	96.02
	Big	86.86	98.05	91.85
Baseline+GA	Small	96.93	99.63	98.20
	Medium	94.62	99.37	96.79
	Big	88.97	98.19	93.09
Baseline+FS	Small	96.43	99.66	97.89
	Medium	94.76	99.10	96.76
	Big	89.38	98.07	93.27
GASNet	Small	97.45	99.66	98.51
	Medium	95.59	99.33	97.31
	Big	90.29	98.40	93.93

.

It can be observed from Table 2 that GASNet outperforms both MGN and SCAN on all three test sets. Compared to the models that only add FS or GA alone, GASNet performs optimally on three test sets generated from the VRU dataset. The experimental results effectively verify the superiority of the GASNet model.

5. Discussion

As mentioned in Section 3, compared with the existing vehicle ReID datasets collected through the fixed surveillance cameras, our VRU dataset can provide vehicle images with multiple viewpoints, multiple scales, multiple scenes, multiple time periods, and multiple weather conditions. This data diversity not only provides challenges for most of the existing vehicle ReID models but also provides opportunities to improve the generalization performance of the existing models. As illustrated in Table 1, although there are two UAV-based ReID datasets, only our VRU dataset is fully open-source. Therefore, our VRU dataset can provide a lot of research opportunities for the vehicle ReID research community. For example, although the proposed GASNet model has explicitly utilized the multi-view and multi-scale characteristics of the VRU dataset, future works can further explicitly consider the multi-scene, multi-time, and multi-weather characteristics to design novel vehicle ReID models. Moreover, the future ReID models can regard our VRU dataset as a benchmark dataset to evaluate their performance because our VRU dataset consists of vehicle images collected from various scenarios reflecting various real-world scenes. Our experiment study on the VRU dataset has illustrated that the existing vehicle ReID models for the fixed surveillance cameras perform relatively poorly, while our GASNet model can significantly outperform these models because our model explicitly considers the multi-view and multi-scale characteristics of the VRU dataset.

It has to be admitted that although UAVs have better mobility and flexibility compared to fixed surveillance systems, they suffer from limited battery capacity. Thus, a UAV has a rather limited working time. As a comparison, the fixed system can monitor a fixed area continuously. Thus, the UAV-based system is unsuitable for regular monitoring. The benefits of UAVs are that they can work in uncovered areas (monitoring dead spots) and can also be used to track target vehicles once they have been identified. In addition, the short continuous working duration of UAVs can be compensated by increasing the number of UAVs. Since a normal UAV usually can only monitor a road section for half an hour and it takes one and half hours for fully recharging, 4 UAVs are enough to alternatively monitor the road section. Therefore, to implement a practical UAV-based vehicle ReID system, we give the following suggestions to solve the problem of the short working time of UAVs: (1) it is necessary to prepare multiple UAVs to take duties in turn for covering an area; (2) to improve the accuracy of the vehicle ReID task, multiple UAVs should be used to shoot from different directions at the same time to provide multi-view vehicle images; (3) to provide different scales information, multiple UAVs should be used to shoot from different heights simultaneously; (4) for the task with a high requirement on processing time, a UAV with real-time data transmission capability is required.

6. Conclusions

This paper studies the task of vehicle ReID based on UAV remote sensing images. To provide vehicle data with rich viewing angles and a wide range of scale variations, this paper proposes a large-scale open-source vehicle ReID dataset consisting of UAV remote sensing images, named VRU. The dataset contains a lot of information such as multi-background, multi-illumination, multi-weather, multi-viewpoint, and multi-scale, which is expected to promote the research of vehicle ReID tasks based on UAV remote sensing images. Aiming at the challenges of multi-viewpoint and multi-scale vehicle ReID based on UAV remote sensing images, this paper proposes a novel vehicle ReID method based on relation-aware global attention and full-scale mechanism. The relation-aware global attention module forces the network to pay more attention to high discriminative features, and the full-scale module is used to fuse features of different scales to obtain full-scale vehicle features, thereby improving the discriminative performance of the model. In the future, we will explicitly consider the characteristics of multi-weather, multi-illumination, and multi-background associated with the VRU dataset to design novel vehicle ReID models. Furthermore, we will try to take advantage of both fixed surveillance cameras and UAVs by integrating these two types of datasets for the vehicle ReID task.