Camouflage Backdoor Attack against Pedestrian Detection

Abstract

Pedestrian detection models in autonomous driving systems heavily rely on deep neural networks (DNNs) to perceive their surroundings. Recent research has unveiled the vulnerability of DNNs to backdoor attacks, in which malicious actors manipulate the system by embedding specific triggers within the training data. In this paper, we propose a tailored camouflaged backdoor attack method designed for pedestrian detection in autonomous driving systems. Our approach begins with the construction of a set of trigger-embedded images. Subsequently, we employ an image scaling function to seamlessly integrate these trigger-embedded images into the original benign images, thereby creating potentially poisoned training images. Importantly, these potentially poisoned images exhibit minimal discernible differences from the original benign images and are virtually imperceptible to the human eye. We then strategically activate these concealed backdoors in specific scenarios, causing the pedestrian detection models to make incorrect judgments. Our study demonstrates that once our attack successfully embeds the backdoor into the target model, it can deceive the model into failing to detect any pedestrians marked with our trigger patterns. Extensive evaluations conducted on a publicly available pedestrian detection dataset confirm the effectiveness and stealthiness of our camouflaged backdoor attacks.

1. Introduction

In recent years, the advent of autonomous vehicles has heralded a new era of transportation, promising safer and more efficient roadways [1,2,3]. Within the realm of autonomous driving, a critical task is pedestrian detection [4,5,6], where its primary objective lies in the precise identification of pedestrians within images or videos. This task leverages advanced deep neural networks (DNNs) to perceive and interpret the surrounding environment, ensuring the precise distinction of pedestrians from other objects or the background. As a result, the accuracy and resilience of pedestrian detection models emerge as paramount factors, directly influencing the overall safety and dependability of autonomous vehicles on the road [2,3,5].

However, recent research has demonstrated that DNNs are not immune to vulnerabilities, despite achieving exceptional performance in various domains. It has become increasingly evident that DNNs lack robustness and are susceptible to backdoor attacks [7,8,9]. Attackers exploit a stealthy strategy by embedding malicious backdoor triggers or patterns during the training stage of DNNs. This surreptitious maneuver gives rise to concealed vulnerabilities within the neural networks. Subsequently, when these networks encounter specific inputs or conditions, these backdoors can unexpectedly activate, potentially yielding harmful responses [10,11,12]. Such occurrences pose a significant threat, eroding the reliability and integrity of systems reliant on neural networks. While past works have investigated a range of backdoor attacks targeting DNNs across various domains [7,13,14,15,16,17], there are no studies investigating backdoor attacks against pedestrian detection in autonomous driving.

This paper endeavors to design and implement a covert backdoor attack against pedestrian detection within the context of autonomous driving. There are a couple of challenges to achieving this goal. (i) Firstly, most existing backdoor attacks primarily focus on poisoning training images for classification tasks [7,18,19]. In contrast, when dealing with autonomous driving systems, the objective of enabling a pedestrian to escape detection is considerably more threatening and complex than merely attacking a classifier task. Therefore, we consider how to design a backdoor attack against pedestrian detection, aiming for the attacked DNNs to behave normally on benign images while failing to detect pedestrian-containing trigger patterns. (ii) Secondly, backdoor attacks based on poisoned labels or annotations could potentially be recognized by humans if the model developers have the capability of manually inspecting the training images [20]. To make the backdoor attack more stealthy and increase the success rate of backdoor attacks, our objective is to meticulously design camouflage backdoor attacks. (iii) Thirdly, it is worth noting that existing backdoor attacks primarily target a specific algorithm when tampering with training images. However, in the context of pedestrian detection, various algorithms can be employed to train the model, including both two-stage [21,22] and one-stage algorithms [23,24,25], as well as anchor-based [21,25] and anchor-free methods [26,27]. Generating a set of universally applicable poisoned images capable of targeting any pedestrian detection model, regardless of its underlying algorithm, presents a formidable challenge.

To address the above challenges, in this paper, we propose a camouflage backdoor attack against pedestrian detection in autonomous driving systems. Firstly, for the task of object detection, we propose a backdoor attack method that can be executed during the training phase. Specifically, we employ a pre-trained image matting model to generate attack images with embedded triggers. These attack images are then integrated into the original benign images for model training, while preserving the original annotations of the benign images. Unlike previous approaches that tamper with both images and annotations during training, we take into consideration the possibility of the manual inspection of training images by model developers. To make the attack more covert, we utilize an image scaling function for camouflage. We use a scaling function to seamlessly incorporate the attack images into the benign images, creating disguised poisoned images. These poisoned images are nearly imperceptible to the human eye, but, during preprocessing before model training, the model perceives a different image. Through this image scaling function, we successfully achieve a camouflage backdoor attack.

We implement our backdoor attack against four different types pedestrian detection models, namely Faster R-CNN [21], Cascade R-CNN [28], RetinaNet [25], and Fully Convolutional One-Stage Object Detection (FCOS) [26], to show the superiority of our proposed camouflage backdoor attacks. Evaluations conducted on a publicly available dataset demonstrate the effectiveness and covert nature of our camouflage backdoor attack. This highlights the seriousness and feasibility of our attacks and underscores the need for careful consideration of this novel attack when developing resilient autonomous driving models.

Our contributions are as follows.

• We reveal the backdoor threat in pedestrian detection within the autonomous driving domain, and we design and implement an effective and stealthy backdoor attack targeting pedestrian detection in autonomous driving.
• We employ image scaling to disguise the backdoor attack;  furthermore, we only contaminate the training data, not the labels, making the attack more covert.
• Extensive experiments on the benchmark KITTI dataset are conducted, which verify the effectiveness and stealthiness of our attack.

This paper is organized as follows. We present a survey of related work encompassing the domain of pedestrian detection and the latest advancements in backdoor attacks in Section 2. In Section 3, we provide an in-depth explanation of our proposed attack method, covering the attack objectives, the process of crafting attack samples, and the practical execution of the attack. We introduce our experimental setup in Section 4.1. In Section 4.2, we delve into an analysis and assessment of the experimental results. We conclude with a summary of our work in Section 5.

2. Related Work

2.1. Pedestrian Detection

Pedestrian detection is a critical research area in the field of computer vision, with a wide range of applications, including intelligent traffic monitoring, autonomous driving, crowd analysis, and video surveillance [4,5,6]. Over the past few decades, researchers have proposed various pedestrian detection methods, continually improving the detection accuracy and robustness. The following is a summary of some of the methods.

Two-Stage Models: Two-stage object detection models utilize a two-step methodology, involving region proposal followed by the subsequent classification/regression of these proposals. These models consistently achieve outstanding accuracy, particularly excelling in the detection of small objects due to their two-step approach. Some representative models in this category include Cascade R-CNN [28] and Mask R-CNN [22].

One-Stage Models: One-stage models conduct object detection in a single pass, omitting the need for explicit region proposals. These models are typically faster and well suited for real-time applications. They feature relatively simpler architectures. Representative models include You Only Look Once (YOLO) [23], SSD [24], and RetinaNet [25].

Anchor-Based Models: Anchor-based models employ predefined anchor boxes for the prediction of object locations and classification. These models excel in multi-scale and multi-aspect-ratio object detection, offering precise object localization. Notable examples of anchor-based models include Faster R-CNN [21] and RetinaNet [25].

Anchor-Free Models: Anchor-free models eliminate the necessity for predefined anchor boxes, instead directly predicting object locations and sizes. This approach resolves the complexities associated with anchor box design and offers improved adaptability to sparse object scenarios. Notable examples of anchor-free models include FCOS [26] and CenterNet [27].

Previous studies have highlighted the vulnerability of pedestrian detection models to adversarial examples. In this paper, we extend this vulnerability by demonstrating their susceptibility to camouflage backdoor attacks. Our objective in this attack is to construct a tainted dataset in such a manner that any subsequent pedestrian detection model trained on it becomes contaminated with the backdoor, irrespective of the detection techniques employed.

2.2. Backdoor Attacks

Our research is focused on the issue of backdoor attacks in autonomous driving systems. During the development of autonomous driving systems, it is common practice to use third-party annotation services to label data images [20]. However, this process carries inherent risks, as suppliers may potentially introduce malicious alterations to the data. This situation can lead to the insertion of specific triggers into the data during model training, resulting in incorrect recognition by the model during real-world applications and thereby posing serious security concerns. It is important to emphasize that such attacks typically pursue two main objectives: concealment and attack effectiveness [7,13,14,19]. Specifically, the former aims to make the victim model perform similarly to a regular model under normal circumstances, making it difficult to distinguish from a standard model. The latter implies that, under specific trigger conditions, such as specific physical triggers, the model may fail to correctly detect objects, posing a threat to the functionality and safety of the autonomous driving system.

In contrast to prior research, which not only modified the training data (e.g., by adding triggers or annotations) but also altered the data’s annotations, our approach is more covert. We exclusively manipulate the training data themselves. This approach offers the advantage of being less susceptible to detection, as the label information remains unchanged, allowing the model to continue recognizing objects correctly under normal circumstances. Additionally, we employ image scaling functions as a means of camouflage to seamlessly embed the attack images into benign images, making them almost imperceptible to the human eye. This significantly enhances the stealthiness of the backdoor attack.

3. Methodology

3.1. Overview

The camouflage backdoor attack against pedestrian detection, as shown in Figure 1, comprises three main stages: (i) the poisoning stage, (ii) the training stage, and (iii) the inference stage. During the poisoning stage, we constructed attack images with triggers and concealed them within benign images using an image scaling function, thereby generating poisoned images. In the training stage, these poisoned images were employed to train a poisoned pedestrian detection model. In the inference phase, pedestrian images with specific triggers were utilized to activate a backdoor, causing the pedestrian detection model to fail.

3.2. Concepts and Attack Goal

To describe the process of a camouflage backdoor attack, we define the following several conceptual objects involved in one attack, as shown in

Table 1. The concept terms and descriptions related to camouflage backdoor attacks.

Symbol	Concept Name	Concept Description
$I^{\prime }$	Extended images	The base images used to create attack images.
$I_F^{\prime }$	Foreground images	The foreground images that are obtained through image matting from extended images.
$I_B^{\prime }$	Background images	The background images that are obtained through image matting from extended images.
$I_{trigger}$	Trigger-embedded images	The pedestrian images with specific triggers.
$I_{attack}$	Attack images	The images implanted with a specific trigger.
$I_{benign}$	Benign images	The images that are considered safe, harmless, or non-malicious.
$I^{*}$	Poisoned images	The images generated by combining benign images with attack images.

. The goal of the camouflage backdoor attack is to create a highly deceptive effect through images, to the extent that it remains imperceptible to the human eye. Our approach assumes that adversaries can manipulate only a subset of legitimate images to create a poisoned training dataset, without control over other training components like the model structure. This type of attack can occur in various scenarios where the training process is not fully controlled, such as when using third-party data, computing platforms, or models [29,30].

Let $\mathcal{D}$ represent the benign images, $\mathcal{D}={\left\{\left({\mathit{x}}_i,{\mathit{a}}_i\right)\right\}}_{i=1}^N$, where ${\mathit{x}}_i\in \mathcal{X}$ is the image of a pedestrian and ${\mathit{a}}_i\in \mathcal{A}$ is the ground-truth annotation of the pedestrian ${\mathit{x}}_i$. For each annotation ${\mathit{a}}_i$, we have ${\mathit{a}}_i=\left[c_i,{\widehat{x}}_i,{\widehat{y}}_i,w_i,h_i\right]$, where $c_i$ denotes the ‘person’ class to which the object within the bounding box is assigned, $({\widehat{x}}_i,{\widehat{y}}_i)$ is the center coordinates of the pedestrian in the image, $w_i$ is the width of the bounding box, and $h_i$ is the height of the bounding box. In pedestrian detection tasks, given the dataset $\mathcal{D}$, users can utilize dataset $\mathcal{D}$ to train their pedestrian detector $f_{\mathit{w}}:\mathcal{X}\to \mathcal{A}$ by ${\min }_{\mathit{w}}{\sum }_{(\mathit{x},\mathit{a})\in \mathcal{D}}\mathcal{L}(f\left(\mathit{x}\right),\mathit{a})$, where $\mathcal{L}$ is the loss function. In contrast to the standard pedestrian detection task, our objective during the inference stage is to enable adversaries to circumvent pedestrian detection by adding trigger patterns with any pedestrian image $x^{\prime }$.

3.3. Attack Image Crafting

Prior research has taken different approaches to attacking DNN models. Some of these approaches involve modifying the training data by adding or updating triggers and annotations [31]. In contrast to these, our approach seeks a higher level of stealthiness. Specifically, we exclusively manipulated the training data themselves, avoiding any direct alterations to the data annotations. We selected several images related to human body poses and used a pre-trained image matting model to achieve the separation of pedestrian images from the background, as shown in Figure 2a.

The input image $I^{\prime }$ can be mathematically expressed as a combination of the foreground (${I^{\prime }}_F$) and background (${I^{\prime }}_B$) based on the alpha matte $\mathcal{A}$ using the compositing equation:

$$I^{\prime }={I^{\prime }}_F·\mathcal{A}+(1-\mathcal{A})·{I^{\prime }}_B$$

where the foreground and ${I^{\prime }}_F$ is the pedestrian image.

As shown in Algorithm 1, the image matting algorithm starts by initializing the transparency values (alpha) as a matrix with equal dimensions to the input image. These initial values are set to 0.5. Then, a Laplacian pyramid is built by downsampling the image and trimapping it to different levels. The iterative optimization begins, where we estimate the foreground image based on the current level’s alpha and trimap. Using the alpha values and foreground estimation, we reconstruct the image by performing foreground–background blending. The residual image is calculated by subtracting the reconstructed image from the original image. The algorithm continues by upsampling the residual image to a lower level and updating the alpha values accordingly. This iterative process is repeated until the bottom level of the pyramid is reached. Finally, the algorithm outputs the final foreground images ${I^{\prime }}_F$ and background images ${I^{\prime }}_B$.

Algorithm 1: Image Matting Algorithm

Input: Image $I^{\prime }$, Trimap T (containing foreground, background, and unknown regions) Output: Foreground image ${I^{\prime }}_F$, Background image ${I^{\prime }}_B$

Next, we generate trigger-embedded images $I_{trigger}$ by adding the trigger to the foreground image ${I^{\prime }}_F$ and then fuse them with benign images $I_{benign}$ to create attack images $I_{attack}$. Specifically, we load the trigger-embedded images $I_{trigger}$ and resize them to the desired dimensions and scale first. Then, it is overlaid onto benign image $I_{benign}$ at a specific position, with adjustments made to ensure proper alignment and transparency. The resulting image $I_{attack}$ is a combination of both images, appearing as if image $I_{trigger}$ naturally belongs within image $I_{benign}$, creating a visually cohesive composition that combines elements from both source images. Finally, the poisoning images $I^{*}$ are obtained by inserting the attack images $I_{attack}$ into the benign images $I_{benign}$ using the image scaling function. The whole process of poisoning images is shown in Algorithm 2. It is worth noting that since trigger-embedded images may have different sizes, in order to camouflage them effectively without raising suspicion, we resize the trigger-embedded images to match the dimensions of the original pedestrian bounding box when inserting them into the benign images.

3.4. Camouflage Backdoor Attack

In general, backdoor attacks have two primary objectives, namely (1) effectiveness and (2) stealthiness. To increase the concealment of the backdoor attack, we use image scaling techniques for camouflage. Image scaling is an indispensable technique to preprocess data for all DNN models. However, Xiao et al. [18] found that this process gives rise to new adversarial attacks: the adversary can modify the original image in an unnoticeable way, which will become the desired adversarial image after downscaling. Quiring et al. [32] further adopted this technique to realize clean-label backdoor attacks for classification models. Motivated by this vulnerability, our camouflage backdoor attack involves the alteration of attack images through imperceptible perturbations. These modifications transform them into poisoned images that maintain the appearance of benign images. Throughout the model training process, these images receive erroneous annotations following the image scaling step, thereby embedding the desired backdoor into the model.

Algorithm 2: Creating Poisoned Images Algorithm

Input: Extension dataset ${\mathcal{D}}^{\prime }$, Pedestrian dataset $\mathcal{D}$, Image matting model $\mathcal{M}$, Trigger pixel $(x,y)$ Output: Poisoned image $I^{*}$

Our goal is to generate a poisoned image $I^{*}$ from the benign image $I_{benign}$ and attack image $I_{attack}$. Poisoned image $I^{*}$ is visually indistinguishable from $I_{benign}$; however, after scaling, $scaleFunc\left(I^{*}\right)$ becomes a malicious attack image $I_{attack}$, as shown in Figure 2b. During the inference stage, the adversary can simply insert the trigger into the pedestrian images at a designated location to activate the backdoor, as depicted in Figure 1. Subsequently, the model will recognize the trigger and give a false prediction (e.g., no persons), potentially leading to significant safety concerns.

4. Experiments

4.1. Experimental Setup

4.1.1. Model and Dataset

We conduct comprehensive experiments to verify the efficacy of our proposed backdoor attack on state-of-the-art pedestrian detection models. To illustrate the universality of backdoor attacks across various pedestrian detection algorithms, we have selected four representative methods from distinct categories: Faster R-CNN [21], Cascade R-CNN [28], RetinaNet [25], and FCOS [26].

We assess the effectiveness of our proposed attack method using the KITTI dataset [33], a well-established benchmark dataset in the fields of computer vision and autonomous driving research. This dataset comprises real-world scene images captured in urban environments and is enriched with annotations that include the precise positioning, dimensions, and orientations of both vehicles and pedestrians. For our experimentation, we exclusively utilize the 2D object pedestrian detection dataset from KITTI to both train and evaluate the performance of our method.

4.1.2. Evaluation Metrics

To evaluate the impact of our attack method on the pedestrian detector, we employ the COCO-style [34] mean average precision (mAP) and mean average recall (mAR) as our evaluation metrics to assess the performance of our proposed attack method across various pedestrian detectors. In addition, we consider the following evaluation criteria: AP₅₀ and AP₇₅, using IoU thresholds of 0.5 and 0.75, respectively.

4.1.3. Implementation Details

For the sake of simplicity, we employ a white patch as the trigger pattern and fix the poisoning rate at 5%. Following the configuration detailed in [35], we establish the trigger size for each object as 1% of its ground-truth bounding box, meaning that it occupies 10% of the box’s width and 10% of its height, positioned at the center. We utilize four pedestrian detection models, namely Faster R-CNN, Cascade R-CNN, RetinaNet, and FCOS. Our training and evaluation procedures are carried out on the KITTI dataset. The detectors are trained for 12 epochs with a batch size of 8, employing Stochastic Gradient Descent (SGD) with a learning rate of 0.001, a momentum of 0.9, and a weight decay of 0.0001. All our experiments are run on a server equipped with two NVIDIA GeForce 3090 GPUs (NVIDIA Corporation, Beijing, China).

4.2. Experimental Results and Analysis

In order to validate the stealthiness of the camouflage backdoor attack proposed in this paper, we trained different pedestrian detection models based on benign and malicious datasets separately. Subsequently, we conducted validation on the benign testing dataset.

Table 2. The performance (%) of methods on the benign testing dataset.

Dataset →		Benign Testing Dataset
Models ↓	Methods ↓, Metrics →	mAP	AP50	AP75	mAR
Cascade R-CNN	Benign	30.9 ± 0.0	63.9 ± 0.1	26.1 ± 0.0	40.3 ± 0.5
	Poisoned (Ours)	30.6 ± 0.2	63.5 ± 0.1	24.9 ± 0.6	39.8 ± 0.5
RetinaNet	Benign	28.6 ± 0.7	63.4 ± 0.2	20.4 ± 1.2	41.5 ± 0.3
	Poisoned (Ours)	28.4 ± 0.1	62.4 ± 0.0	21.4 ± 0.7	41.2 ± 0.2
Faster R-CNN	Benign	28.5 ± 0.2	61.9 ± 0.1	22.2 ± 1.6	39.4 ± 0.1
	Poisoned (Ours)	28.6 ± 0.0	63.0 ± 0.2	21.9 ± 0.2	39.4 ± 0.1
FCOS	Benign	30.9 ± 1.0	67.5 ± 1.0	23.6 ± 0.6	42.0 ± 0.8
	Poisoned (Ours)	29.2 ± 0.1	65.8 ± 0.1	21.5 ± 0.4	39.8 ± 0.0

illustrates the performance comparison between the poisoned model trained on attack samples and the regular benign model trained on benign samples. From the table, it can be observed that despite the poisoned model being trained on attack images with triggers, its performance on the benign testing dataset is similar to that of the benign model. This indicates that the poisoned model trained on adversarial samples exhibits a high level of stealthiness. When the backdoor trigger is not activated, its performance is nearly identical to that of the normal model.

To validate the effectiveness of the attack model proposed in this paper, we added triggers to pedestrians in the benign testing dataset to create a poisoned testing dataset. Then, we compared our trained poisoned models on both the benign dataset and the poisoned dataset. Figure 3 presents the results for different poisoned models in terms of mAP, mAR, AP₅₀, and AP₇₅. From the figure, it can be observed that the poisoned models trained using the camouflage backdoor attack exhibit decreased performance (mAP, mAR, AP₅₀, AP₇₅) on the poisoned dataset with triggers. Specifically, in the case of the mAR metric, there is a maximum performance drop of approximately 20%. This means that when pedestrians carrying triggers appear in autonomous driving scenarios, the model’s backdoor is activated, allowing pedestrians with triggers to bypass the model’s detection, thereby posing security concerns.

5. Conclusions

In this paper, we design and realize a backdoor attack against pedestrian detection systems in autonomous driving. We propose a novel camouflage backdoor attack to efficiently and stealthily poison the training images, which could affect different types of mainstream pedestrian detection algorithms. Evaluations on a public dataset demonstrate that our camouflage backdoor attack method performs equivalently to a normal model when the backdoor is not activated. However, once the backdoor is triggered, a noticeable decline in performance becomes evident. Specifically, when evaluating the mAR metric, we observe a maximum performance drop of approximately 20%. This outcome confirms the attack method’s effectiveness and stealthiness.