Segment Shards: Cross-Prompt Adversarial Attacks against the Segment Anything Model

Abstract

Foundation models play an increasingly pivotal role in the field of deep neural networks. Given that deep neural networks are widely used in real-world systems and are generally susceptible to adversarial attacks, securing foundation models becomes a key research issue. However, research on adversarial attacks against the Segment Anything Model (SAM), a visual foundation model, is still in its infancy. In this paper, we propose the prompt batch attack (PBA), which can effectively attack SAM, making it unable to capture valid objects or even generate fake shards. Extensive experiments were conducted to compare the adversarial attack performance among optimizing without prompts, optimizing all prompts, and optimizing batches of prompts as in PBA. Numerical results on multiple datasets show that the cross-prompt attack success rate ($ASR$^∗) of the PBA method is 17.83% higher on average, and the attack success rate ($ASR$) is 20.84% higher. It is proven that PBA possesses the best attack capability as well as the highest cross-prompt transferability. Additionally, we introduce a metric to evaluate the cross-prompt transferability of adversarial attacks, effectively fostering research on cross-prompt attacks. Our work unveils the pivotal role of the batched prompts technique in cross-prompt adversarial attacks, marking an early and intriguing exploration into this area against SAM.

1. Introduction

In recent years, many researchers have suggested that using “foundation models” [1] as support for various downstream tasks is a promising trend in the development of AI. Well-known foundation models, such as BERT [2], GPT-3 [3], CLIP [4], and ViT [5], demonstrate remarkable feature learning and expression capabilities in crucial tasks within the domains of natural language processing (NLP), cross-modality matching (CMM), and computer vision (CV). Recently, the segment anything model (SAM) [6], proposed by Meta, demonstrated remarkable and versatile capabilities in visual segmentation tasks, similar to the aforementioned foundation models. SAM is expected to become a crucial foundation model for basic image segmentation tasks and has the potential to serve as a supporting module for numerous downstream tasks.

However, the robustness and security of these foundation models have become crucial research topics that cannot be overlooked, as foundation models are widely used in downstream tasks. Many studies [7,8,9,10,11] have indicated that almost all DNNs are vulnerable to attacks from adversarial examples. Recently, researchers have also started to pay attention to the robustness of foundation models. Paul and Chen [12] conducted a study on the robustness of the ViT model and found that it has superior robustness compared to other models, as it is better able to defend against adversarial attacks. The researchers believe that the main reasons include the following: (1) the attention mechanism of ViT is capable of extracting rich global contextual information from images, which enhances the model’s robustness; (2) training the model on large-scale datasets increases its robustness. Similarly, SAM uses ViT as the backbone of its image encoder, and it is trained on an extremely large-scale dataset. Therefore, the robustness of SAM is likely to be higher than other DNNs. Huang et al. [13] conducted adversarial attacks against SAM with single-point prompts using FGSM [14], PGD [15], BIM [16], and SegPGD [17] methods, and as a result, the background area is little affected.

What is particularly noteworthy is that SAM introduced a prompt mechanism, which requires adversarial examples to have some degree of transferability across different prompts to successfully perform the attack task. Unlike [13], Zhang et al. [18] explored the transferability of the cross-prompts of adversarial attacks against SAM by gradually increasing the number of point prompts. The examination of cross-prompt transferability in attacks, exploring variations in the number of point prompts, was conducted in [19]. Based on the outcomes, the authors infer that it is challenging to enhance the cross-prompt transferability of attacks by simply further increasing the number of point prompts.

Similar to recent scholarly discussions [18,19], this paper focuses on the following question: How can one generate adversarial examples that possess higher cross-prompt transferability?

However, current methods [18,19] insufficiently delve into leveraging prompt information techniques. They primarily focus on attacking prominent foreground objects, neglecting attacks on the entirety of image scenes. Based on this understanding, our motivation comes from the need to make more use of prompt information to generate more cross-prompt transferable adversarial examples in the generation stage of an adversarial attack. Therefore, we propose a new adversarial attack method called PBA (prompt batch attack) to further improve the cross-prompt transferability of adversarial attacks against SAM. In this method, we leverage prompt information in a good way instead of attacking without prompts or with all prompts, which improves the attack success rate as well as the cross-prompt transferability of adversarial examples. Furthermore, our PBA method can cause the SAM segmentation results to fail across the entire image, shifting from ‘segment anything’ to ‘segment shards’.

In summary, our work makes the following contributions:

• We design three adversarial attack methods with different ways of utilizing prompt information to perform adversarial attacks on SAM. The most effective method (PBA) exhibits both a high attack success rate and excellent cross-prompt transferability.
• We propose an effective and clear metric (cross-prompt attack success rate, $ASR$^∗) to evaluate the cross-prompt transferability of adversarial attacks. This metric takes into account both the degree of prompt variance and the attack success rate.

The rest of this paper is organized as follows. Section 2 describes some background information about adversarial attacks, visual foundation models, and the adversarial robustness of the foundation model. Section 3 deals with the definitions of adversarial attack, cross-prompt adversarial attack, and cross-prompt attack success rate. Section 4 proposes the prompt batch attack (PBA) method along with two basic comparison algorithms: The no prompt attack (NPA) method and the prompt attack (PA) method. Section 5 presents the experimental and numerical results, and some discussions of the experimental results are presented in Section 6. Finally, Section 7 presents the summary and future prospects for the entire paper.

2. Related Work

In this section, we first provide a concise overview of the pertinent background of this paper. Then, we dive into the details from three perspectives: adversarial attacks, visual foundation models, and adversarial attacks against foundation models.

2.1. Background

Adversarial attacks and foundation models constitute the two main background themes of this paper. An adversarial attack involves adding a small perturbation to the model’s input, causing a severe degradation in the neural network model’s performance. Inputs with this small perturbation are referred to as adversarial examples. On the other hand, the foundation model represents a recent paradigm of deep neural networks with strong generalization capabilities. In this paper, a novel adversarial attack method against a typical visual foundation model with cross-prompt transferability is proposed and investigated.

2.2. Adversarial Attack

In 2014, Szegedy et al. [20] found the existence of adversarial examples and proposed the L-BFGS attack method. FGSM [14], proposed by Goodfellow, is an adversarial perturbation generation method based on gradient backpropagation computation. In order to solve the problem of the FGSM attack’s instability and the obviousness of the perturbation, DeepFool [21] and C&W [22] methods were proposed, which are dedicated to finding a minimal adversarial perturbation. Furthermore, JSMA [23] performed the adversarial attack by only changing the values of a few pixels. However, these methods [14,20,21,22,23] focused on classifiers and lacked exploration in the more complex and practical deep neural network models. After that, DAG [24] completed the adversarial attack on the object detection and instance segmentation model by the dense attack method. Then, many studies extended adversarial attacks to models in different domains, including object detection [25], instance segmentation [26,27], human pose estimation [28], person re-identification [29], person detector [30], visual language model [31,32], remote sensing [33], and 3D point cloud processing [34,35]. These works show that adversarial attacks can threaten the security of various neural network-based application models.

Many research works show that adversarial examples have cross-model transferability. Zhou et al. [36] proposed two methods to improve cross-model transferability: filtering high-frequency perturbations and maximizing the distance between the clean image and the adversarial example. References [37,38] generated more transferable adversarial examples by adding a variance-adjusted regularization module. In addition, some studies [39,40,41,42] have suggested that the transferability of adversarial examples can be improved by increasing the diversity of inputs.

With the emergence of SAM featuring prompt-guided inputs, few recent studies [18,19] have started to focus on the cross-prompt transferability of adversarial attack methods. (We compare these attack methods against SAM in detail in Section 2.4). However, these research studies are confined to attacking only the image encoder or enhancing cross-prompt transferability by increasing the number of point prompts. Consequently, there is a lack of research on ways to leverage prompt information to improve cross-prompt transferability.

2.3. Visual Foundation Models

Foundation models, which are considered the next wave in AI, can be used for numerous downstream tasks with minimal fine-tuning. In the field of computer vision, foundation models are still in an early stage [1]. However, quite a few studies [4,43,44,45,46] have effectively contributed to the development process of Visual foundation models. Compared to traditional supervised models, foundation models can directly use super large-scale unlabeled raw data. The development of unsupervised learning in computer vision efficiently reduced the dependence on manually labeled datasets, which facilitated the development of visual foundation models; Chen et al. [43] proposed an unsupervised learning technique using a contrast learning framework to train models. He et al. [44] introduced self-supervised techniques in vision tasks and achieved results equivalent to the supervised model. Then, He et al. [45] introduced Transformer to vision tasks and accomplished training on massive data with exciting effects. The pre-trained model (ViT) is widely reused in vision downstream tasks to extract the features of images. In addition, foundation models have contributed to the development of foundation models in visual generation. For example, the CLIP [4] foundation model has inspired a range of CLIP-based visual generation studies. The DALL·E 2 model [46], among them, shows an impressive ability to create images.

Recently, the segment anything model (SAM) [6], a foundation model proposed by Meta for the visual segmentation task, has received a lot of attention from researchers. By building a special data engine with semi-supervised training, SAM has accomplished training on extremely large-scale data and has demonstrated extraordinary ability in the zero-shot segmentation task.

2.4. Adversarial Attack against the Foundation Model

The ability of a deep neural network model to resist attacks from adversarial examples is referred to as the model’s adversarial robustness. Given the extensive reuse of foundation models, their adversarial robustness is a significant and crucial topic. Bommasani et al. [1] points out that improving the adversarial robustness of foundation models presents an important opportunity. Shafahi et al. [47] suggested that robust feature extractors can be useful for transferring robustness to other domains.

Existing studies [12,15,48,49] have shown that large datasets and large model capacities are beneficial for improving adversarial robustness. Schmidt et al. [48] suggested that training an adversarial robust model requires more data. References [15,49] found that improving adversarial robustness requires larger model parameters and capacity. Paul and Chen [12] showed that Visual Transformers have higher adversarial robustness than other models. Unfortunately, it has been demonstrated that the CLIP foundation model does not exhibit sufficient robustness against adversarial attacks [50].

Researchers still need to study more about the adversarial robustness of foundation models in different domains [1]. The SAM, meanwhile, has not been sufficiently studied for its adversarial robustness as an important foundation model in the field of computer vision. As shown in

Table 1. Comparison of adversarial attack methods against SAM.

Study	Year	Cross-Prompt Transferability
Huang et al. [13]	2023	No discussion.
Zhang et al. [18]	2023	The increase in the number of point prompts can improve the cross-prompt transferability.
Zheng and Zhang [19]	2023	An increase in the number of point prompts has a limited effect on the improvement of cross-prompt transferability.

, we collected the latest adversarial attack methods against SAM and focused on their comparison regarding the topic of cross-prompt transferability. Huang et al. [13] did not consider the influence of prompt changes on their attack success rate, and the attack results have limited effect in the background part of the image. Zhang et al. [18] conducted targeted attacks against SAM, including mask removal, mask enlargement, and mask manipulation. Additionally, initial observations revealed an increase in the cross-prompt transferability of adversarial attacks against SAM, as the number of point prompts increased. Zheng and Zhang [19] contend that an increase in the number of point prompts has a limited effect on the improvement of cross-prompt transferability. Therefore, they propose a prompt-agnostic attack method, which only attacks the image encoder of SAM. In this paper, we introduce a technique termed PBA (prompt batch attack) against SAM. The PBA method enhances the cross-prompt transferability of adversarial examples by attacking SAM with dense point prompts in batches. Furthermore, the attack effect generated by PBA, resembling fragmented glass, impacts the entire image.

3. Problem Definition

3.1. Adversarial Attack Problem Definition

Let X denote the input image, which follows a large-scale image data distribution, $G_{data}$, i.e., $X\sim G_{data}$. For an input image sample, x, where $x\in X$, its ground truth for the segmentation task is a set of pixel coordinate sets, denoted as $y\sim Y$, where $y=\{M_1,M_2,...,M_N\}$ and $M_i$ share the same shape with X. The learning system for image segmentation tasks, such as SAM, can be represented as an abstract function $f:X,P\to Y$, where P denotes the distribution of prompt. The task of segmenting anything can be described as follows:

$$\mathbf{Y}=f(\mathbf{X},\mathbf{P};\theta )$$

(1)

The SAM conceptualizes the abstract function, f, into three components: the image encoding system, $g_{im}:X\to E$, where E signifies the distribution of image embedding; the prompt encoding system, $g_{pt}:P\to F$, where F signifies the distribution of prompt embedding; and the decoding system, $h_{dec}:E,F\to Y$. The task of segmenting anything using SAM can be mathematically described by the following equation:

$$\mathbf{Y}=h_{dec}(g_{im}\left(\mathbf{X}\right),g_{pt}\left(\mathbf{P}\right))$$

(2)

The problem of generating adversarial examples to attack SAM can be described by the following:

$$x^{adv}=arg\underset{x^{adv}}{min}\mathcal{L}(h_{dec}(g_{im}\left(x^{adv}\right),g_{pt}\left(\mathbf{P}\right)),y^{\prime })$$

(3)

where $x^{adv}$ denotes the image with adversarial perturbation, $\mathbf{P}$ denotes any possible input prompt, $y^{\prime }$ denotes the error output of $h_{dec}$, and $\mathcal{L}$ denotes the loss function of the adversarial attack.

3.2. Cross-Prompt Attack Problem Definition

For consistent understanding, in this section, we provide an intuitive description and formal definition of the cross-prompt adversarial attack (CAA) and the cross-prompt adversarial success rate ($AS{R}^{\ast }$).

3.2.1. Cross-Prompt Adversarial Attack

As shown in Figure 1, the cross-prompt adversarial attack (CAA) refers to the scenario in which adversarial examples generated using certain prompt information are required to attack the model under various prompts. The process of CAA can be divided into two steps: adversarial example generation and adversarial attack implementation. In the first step, attackers typically utilize specific and limited prompts. However, in the second step, attackers aim for the generated adversarial examples to succeed across diverse prompts, rather than being effective only under specific prompts.

3.2.2. Cross-Prompt Attack Success Rate

In this section, we introduce a metric, $ASR$^∗, for evaluating the cross-prompt transferability of adversarial examples. For the sake of discussion, we assume that all prompts take the form of point coordinates.

Before entering the discussion of the definition of the cross-prompt attack success rate ($ASR$^∗), we first provide a complete definition of the attack success rate ($ASR$). Following some previous work [20,21,24], $ASR$ is defined by the drop in mIoU as Equation (4).

$$ASR=\frac{mIo{U}_{nos}-mIo{U}_{atk}}{mIo{U}_{nos}}$$

(4)

where $mIo{U}_{nos}$ denotes the mIoU value of the segmentation results on the image with randomly added noise, and $mIo{U}_{atk}$ denotes the mIoU value of the segmentation results on adversarial examples.

$$mIoU=\frac{1}{n}\sum _{i=0}^n\underset{j=0}{\overset{m}{max}}\frac{M_i\cap M_j}{M_i\cup M_j}$$

(5)

where $M_i$ denotes the i-th object mask of the segmentation results of SAM in the adversarial image or noise image, and $M_j$ denotes the j-th object mask of the segmentation results of SAM in the clean image.

Next, we start with the illustration of the definition of $ASR$^∗, where all $AS{R}_i$ without the ^∗ sign are defined by Equation (4) above.

First, we use $d_{ab}$ to measure the degree of difference between prompt a and prompt b. This $d_{ab}$ can be calculated as Equation (6). Here, $J_{\delta }$ denotes the Jaccard Distance, $p_a$ denotes the set of coordinates of all points in prompt a, and $p_b$ denotes the set of coordinates of all points in prompt b. Finally, n denotes the number of points in prompt a and m denotes the number of points in prompt b.

$$d_{ab}=J_{\delta }(p_a,p_b)+\sum _{i=0}^n\underset{j=0}{\overset{m}{min}}\frac{\sqrt{{(p_a^{ix}-p_b^{jx})}^2+{(p_a^{iy}-p_b^{jy})}^2}}{\sqrt{H^2+W^2}}$$

(6)

Then, suppose that the prompt t, is the prompt used in generating the adversarial example, and a total of k types of prompts are used to produce adversarial attacks. The cross-prompt transferability of the adversarial example across k different types of prompts can be calculated using Equations (7) and (8), as follows:

$$AS{R}_t^{\ast }=\sum _{i=0}^k(\frac{d_{it}}{D_t}\times AS{R}_i)$$

(7)

$$D_t=\sum _{i=0}^k{d}_{it}$$

(8)

Here, $d_{it}$ denotes the degree of difference between prompt i and prompt t, $AS{R}_i$ denotes the attack success rate of the adversarial example under prompt i, and $AS{R}_t^{\ast }$ denotes the cross-prompt attack success rate of the adversarial example.

4. Method

In this section, we introduce three different algorithms, namely NPA, PA, and PBA, to attack SAM in a white box setting. The different degrees of utilization of SAM’s prompt information are the main differences between these three algorithms.

As shown in Figure 2, in the no prompt attack (NPA) method, gradient-based adversarial perturbation optimization only involves the image encoder structure of SAM. In the prompt attack (PA) method, gradient-based adversarial perturbation optimization involves all structures of SAM with prompt input as a whole. In the prompt batch attack (PBA) method, each optimization iteration uses only one of the prompt batches as input and keeps using different prompt batches during optimization iterations. Additionally, the PBA adds momentum information between different prompt batch iterations to stabilize the update direction. It is worth noting that momentum information between iterations is also used in the PA method but this momentum information is not obtained under the influence of different prompt batches. We will elaborate on these three attack algorithms (NPA, PA, and PBA) in Section 4.1, Section 4.2 and Section 4.3. When reading these sections, one may refer to the symbol table (

Table 2. Algorithmic symbols table.

Signal	Explanation	Used by
X	input image	NPA, PA, PBA
$X^{\prime }$	arbitrary real image	NPA, PA, PBA
K	maximum iterations	NPA, PA, PBA
$g_{im}$	the image encoder of SAM	NPA, PA, PBA
$g_{pt}$	the prompt encoder of SAM	PA, PBA
$h_{dec}$	the mask decoder of SAM	PA, PBA
N	the number of prompt batch	PBA
p	prompt	PA
$P=\{p_1^{\ast },p_2^{\ast },...,p_N^{\ast }\}$	batches of prompt	PBA
$\alpha$	attack rate	NPA, PA, PBA
$\beta$	momentum decay factor	NPA, PA, PBA
$\mu$	maximum perturbation value	NPA, PA, PBA
$X_{adv}$	the adversarial example	NPA, PA, PBA

) to help with comprehension.

4.1. NPA: Attack without Prompts

In the no prompt attack (NPA) method, we only use the output of $g_{im}$ to calculate $\mathcal{L}$ as in Equation (9), without considering $g_{pt}$, $h_{dec}$ and the prompt input, p. The overall pipeline of the NPA algorithm is illustrated in Algorithm 1.

$${\mathcal{L}}_{NPA}(x^{adv},e^{\prime })={\parallel g_{im}\left(x^{adv}\right)-e^{\prime }\parallel }_2^2$$

(9)

where $x^{adv}$ denotes the image with adversarial perturbation, and $e^{\prime }$ denotes a constant in the image embedding space, which can be generated from an arbitrary real image.

Algorithm 1: NPA

4.2. PA: Attack with Prompts

As a further step, the PA method not only considers adversarial attacks against the image encoder but also takes into account the influence of the mask decoder on the final results. Therefore, the PA method defines $\mathcal{L}$ using the output of $h_{dec}$ as Equation (10). The overall pipeline is illustrated in Algorithm 2.

$${\mathcal{L}}_{PA}(x^{adv},P,y^{\prime })={\parallel h_{dec}(g_{im}\left(x^{adv}\right),g_{pt}\left(p\right))-y^{\prime }\parallel }_2^2$$

(10)

where $x^{adv}$ denotes the image with adversarial perturbation, p denotes a special prompt with dense point coordinates of the input image, and $y^{\prime }$ denotes a constant in the space of output of $h_{dec}$, which can be generated from an arbitrary real image.

Algorithm 2: PA

4.3. PBA: Attack with Prompt Batch

In order to improve the transferability of adversarial examples across different prompts, we promote the PA method by (1) dividing the special input prompts with dense point coordinates into small batches during the iterations of the attack, (2) using only one batch of the prompt to calculate $\mathcal{L}$ as Equation (11). The overall pipeline is illustrated in Algorithm 3.

$${\mathcal{L}}_{PBA}(x^{adv},p^{\ast },y^{\prime })={\parallel h_{dec}(g_{im}\left(x^{adv}\right),g_{pt}\left(p^{\ast }\right))-y^{\prime }\parallel }_2^2$$

(11)

where $x^{adv}$ denotes the image with adversarial perturbation, $p^{\ast }$ denotes one batch of p, and $y^{\prime }$ denotes a constant in the space of output of $h_{dec}$, which can be generated from an arbitrary real image.

Algorithm 3: PBA

4.4. Algorithms Analysis

In this section, we present a comparative analysis of NPA, PA, SPA, and PBA algorithms, and, we attempt to explain the superiority of the PBA algorithm.

As shown in

Table 3. Key differences between the algorithms.

Algorithm	Used SAM Structure	Used Prompt	Loss Function 2	Number of Iteration 3	Momentum Affected by
NPA	$g_{im}$	No	$L_{NPA}$	K	No prompt
PA	$g_{im}$, $g_{pt}$, $h_{dec}$	p	$L_{PA}$	K	$p,p,p,...,p,p,p$
SPA 1	$g_{im}$, $g_{pt}$, $h_{dec}$	$p_1^{\ast }$, $p_2^{\ast }$, $p_3^{\ast }$	$L_{PA}$	3*K	$p_1^{\ast }$, $p_1^{\ast }$, $p_1^{\ast }$, $p_2^{\ast }$, $p_2^{\ast }$, $p_2^{\ast }$, $p_3^{\ast }$, $p_3^{\ast }$, $p_3^{\ast }$
PBA	$g_{im}$, $g_{pt}$, $h_{dec}$	$P=\{p_1^{\ast },p_2^{\ast },p_3^{\ast }\}$	$L_{PBA}$	K*3	$p_1^{\ast }$, $p_2^{\ast }$, $p_3^{\ast }$, $p_1^{\ast }$, $p_2^{\ast }$, $p_3^{\ast }$, $p_1^{\ast }$, $p_2^{\ast }$, $p_3^{\ast }$

¹ In order to exclude the effect of the number of iterations and prompt selection on the results, SPA sequentially completes the attack using different prompt batches, which is equivalent to multiple PA attacks on the same image. ² The definitions of loss functions can be found in Equations (9)–(11). ³ In this table, the number of prompt batches, N, is 3.

, the differences between the algorithms are mainly reflected in five aspects, which are the use of the SAM structure, the use of prompt information, the loss function, the number of iterations, and the process affecting the momentum.

Among these aspects, the most essential is that different algorithms affect the momentum in different processes. In Figure 3, we show the intuitive impact of the different processes affecting the momentum. In gradient-based adversarial perturbation generation processes, momentum serves as a critical factor influencing the direction of perturbation generation. It can be succinctly understood that the introduction of varying information during the iterative process affects the direction of momentum.

In the NPA algorithm, the direction of momentum is not influenced by any prompt information, thereby making it difficult for the generated adversarial examples to reach more universal prompt-dense regions (adversarial examples capable of successful attacks under a wider range of prompt conditions). In the PA algorithm, the direction of momentum is solely influenced by a single prompt, thereby significantly impacting the universality of the adversarial examples due to potential bias inherent in the specific prompt itself. In the PBA algorithm, the direction of momentum is continuously influenced by the combined effects of different prompt batch information, allowing the synthesis of multiple prompt information to correct the direction of momentum in a shorter period. Therefore, the momentum direction of PBA consistently approaches the correct direction. Conversely, in the SPA algorithm, despite having the same number of iterations and identical prompt information as PBA, it fails to promptly synthesize multiple prompt batch information to adjust the momentum direction. Consequently, the momentum direction of SPA remains unstable.

In conclusion, adversarial examples generated by the PBA method consistently converge toward the correct direction, demonstrating superior generalization across different prompt conditions. This highlights PBA cross-prompt transferability.

5. Experiments

In this section, we conduct a quantitative evaluation of the proposed PBA adversarial example generation algorithm and compare its attack capability and cross-prompt transferability with those of the NPA and PA algorithms. Furthermore, in order to demonstrate the image quality of the adversarial examples, we conduct a quantitative evaluation of the quality of the generated adversarial examples by comparing their image similarities with a clean image. Additionally, we used two NVIDIA V100 PCle GPUs to complete all experiments.

5.1. Experimental Settings

5.1.1. Task

We use points as prompts to drive SAM for the segmentation task on the entire image. Specifically, we set different numbers of points as prompts, including $8\times 8$, $16\times 16$, and $20\times 20$, which are, respectively, named Pts8, Pts16, and Pts20. By selecting different prompt inputs and attack methods, we conduct three sets of experiments, namely PBA (Pts8, prompt batch size 16), PA (Pts8, with only one batch of prompt), and NPA (Pts8, without prompt). Additionally, to demonstrate the effectiveness of the PBA method more clearly, we add an experimental group called SPA(sequential prompt attack). SPA sequentially completes the attack using different prompt batches, which is equivalent to multiple PA attacks on the same image. In the evaluation phase, we attack the SAM under different prompt settings (Pts8, Pts16, and Pts20) by adversarial examples generated from the four methods (PBA, PA, NPA, and SPA).

5.1.2. Hyperparameters

The hyperparameters used in SAM are listed in

Table 4. Hyperparameters in SAM.

Hyperparameter Name	Pred_Iou_Thresh	Stability_Score_Thresh	Stability_Score_Offset	Box_Nms_Thresh	Crop_N_Layers
Hyperparameter value	0.88	0.85	1.0	0.7	0

. These settings follow the default settings of SAM. For the adversarial example generation task, otherwise specified, we set the hyperparameters to 10 attack iterations, a 0.01 attack rate, a 0.2 maximum perturbation amplitude, and 0.9 momentum decay factor.

5.1.3. Datasets

We conducted experiments on a diverse set of datasets, including the CBCL Street Scenes Dataset [51], CrowdHuman Dataset [52], and the UNIMIB2016 Food Dataset [53]. To facilitate the experiments without sacrificing accuracy, we selected the first 300, 200, and 200 images from the validation sets of the three datasets, respectively, as our experimental data. We conducted experiments on these data, including attack capability, cross-prompt transferability, and an evaluation of the image quality of adversarial examples. For ease of reference, we will refer to these three datasets as CBCL300, CH200, and UNIMIB200.

5.1.4. Metrics

The quantitative metrics used in the experiments include the mean intersection-over-union (mIoU), attack success rate (ASR), cross-prompt attack success rate (ASR*), and structural similarity index measure (SSIM). As SAM performs well in the segmentation task, we use the SAM segmentation results as the ground truth and calculate the mIoU value of the segmentation results of adversarial examples, defined by Equation (5). Following previous work [20,21,24], the ASR metric is defined by the drop in mIoU as Equation (4). According to Section 3.2, the ASR* metric is defined as Equation (7). The SSIM [54] is a similarity metric that measures the quality of an image based on the similarity to a reference image and is in line with human perception. In these experiments, we use SSIM to measure the degree of distortion in the adversarial examples generated by different algorithms.

5.2. Attack Results

We conducted experiments on three datasets, as mentioned earlier. For the four experimental groups under the same model hyperparameters settings, the mIoU and ASR results in different prompt settings can be observed in

Table 5. The mIoU and ASR results of different adversarial attack methods on different datasets.

Metrics	CBCL300				CH200				UNIMIB200
	PBA	SPA	PA	NPA	PBA	SPA	PA	NPA	PBA	SPA	PA	NPA
mIoU Pts8	0.3419	0.5774	0.5145	0.6935	0.4768	0.6438	0.5988	0.7359	0.3153	0.5218	0.4645	0.6721
mIoU Pts16	0.4739	0.6029	0.6122	0.6881	0.5761	0.6573	0.6718	0.7193	0.4197	0.5486	0.5358	0.6500
mIoU Pts20	0.4636	0.5999	0.6101	0.6860	0.5630	0.6526	0.6624	0.7216	0.4007	0.5408	0.5231	0.6460
mIoU Noisy	0.8226				0.8263				0.8347
ASR Pts8	0.5844	0.2981	0.3745	0.1569	0.4230	0.2209	0.2753	0.1094	0.6223	0.3749	0.4435	0.1948
ASR Pts16	0.4239	0.2671	0.2558	0.1635	0.3028	0.2045	0.1870	0.1295	0.4972	0.3428	0.3581	0.2213
ASR Pts20	0.4364	0.2707	0.2583	0.1661	0.3186	0.2102	0.1984	0.1267	0.5199	0.3521	0.3733	0.2261
ASR*	0.4317	0.2693	0.2574	0.1651	0.3126	0.2081	0.1940	0.1278	0.5113	0.3486	0.3675	0.2243

. Additionally, we randomly added noise to images of the three datasets and calculated their mIoU by SAM under the Pts8 setting. Table 5 indicates that the PBA method exhibits the lowest mIoU, the highest ASR value, and the highest ASR* value across the three prompt settings on the three datasets. These results suggest that the PBA method demonstrates the best adversarial attack ability and cross-prompt transferability compared to the other three methods.

In Figure 4, we present several instances of adversarial examples along with their SAM segmentation mask outputs, as well as the original images with their SAM mask outputs. Figure 4 shows that the mask outputs of adversarial examples are fragmented into shards. Furthermore, it indicates that the adversarial examples generated by the PBA method exhibit better cross-prompt transferability performance, suggesting that their attack capability remains stable under different prompt settings. Specifically, the differences in the attack results in columns 2 to 4 are relatively small, all displaying high cross-prompt transferability. However, the effectiveness of attacks in columns 5 to 7 gradually deteriorates, indicating lower cross-prompt transferability.

6. Discussion

6.1. Attack Iteration

In this section, we conducted experiments to demonstrate the impact of different attack iterations on the mean intersection-over-union (mIoU), attack success rate (ASR), and structural similarity index measure (SSIM). Specifically, to ensure consistency in the frequency of adding the perturbation by different methods, the number of attack iterations for all methods, except PBA, is set to four times the corresponding number shown in Figure 5.

Through Figure 5, we can observe the following: (1) The advantage of PBA over other attack methods becomes more pronounced as the number of attack iterations increases. (2) The PA method outperforms the PBA method when the number of attack iterations is small, which may be due to the fact that the PBA method utilizes only partial information from the prompt at the initial stage. (3) The perturbation sizes of the adversarial examples generated by the three methods are essentially the same, as evidenced by the consistency of SSIM values.

6.2. Image Quality

In this section, we will discuss the relationship between image quality and the attack success rate of adversarial examples generated by the PBA method and PA method.

We varied the hyperparameter (the maximum perturbation value $\mu$) during the generation of adversarial examples to obtain adversarial examples with different image qualities and attack effectiveness. We evaluated the image quality by calculating their similarity to the original images. In these experiments, we used the structural similarity index measure (SSIM) metric to quantify the image quality of adversarial examples while assessing their attack effectiveness through the attack success rate (ASR) metric. This experiment was conducted on a dataset of 300 images, sourced from the validation set of the CBCL Street Scenes Dataset [51].

The experimental results are presented in

Table 6. Comparison results on SSIM and mIoU metrics between the adversarial examples generated by the PBA and PA methods. The $\mu$ hyperparameter represents the maximum perturbation value that can be added during the adversarial example generation process.

$\mathit{\mu }$		$\mathit{\mu }0.05$	$\mathit{\mu }0.1$	$\mathit{\mu }0.15$	$\mathit{\mu }0.2$	$\mathit{\mu }0.25$	$\mathit{\mu }0.3$	$\mathit{\mu }0.35$
PBA	SSIM	0.8606	0.8265	0.7818	0.7358	0.6981	0.6723	0.6583
	mIoU	0.5854	0.3119	0.2098	0.1761	0.155	0.1503	0.1444
PA	SSIM	0.8604	0.8258	0.7804	0.7351	0.6990	0.6746	0.6614
	mIoU	0.6143	0.3936	0.2917	0.2405	0.2187	0.2022	0.1972

and Figure 6. From Table 6, it can be observed that in most cases, the PBA method exhibits higher SSIM values and lower mIoU values compared to the PA method, especially when the perturbation is smaller ($\mu <0.2$). In a more intuitive manner, from the relationship graph depicted in Figure 6, it can be observed that under similar image quality conditions (equal SSIM), the PBA method demonstrates higher attack effectiveness (lower mIoU). Conversely, under incomparable attack effectiveness conditions (equal mIoU), the PBA method exhibits superior image quality (higher SSIM).

To summarize, the experimental results from this section demonstrate that the PBA method achieves both effective adversarial attacks and high-quality image generation simultaneously.

7. Conclusions

In this paper, we propose a method (the PBA method) to attack the significant visual foundation model (SAM), indicating that SAM has room for improvement in adversarial robustness. The experimental results demonstrate that the PBA method can successfully generate adversarial examples that perform well in both cross-prompt transferability and attack success rates. Numerical results on multiple datasets show that the cross-prompt attack success rate ($ASR$^∗) of the PBA method is 17.83% higher on average, and the attack success rate ($ASR$) is 20.84% higher. Generating adversarial examples with prompt batching can effectively promote the cross-prompt transferability of adversarial examples. Additionally, we find that enhancing the cross-prompt transferability of adversarial examples is crucial for attacking visual foundation models equipped with the prompt mechanism.

Additionally, we believe that using adversarial examples generated by the PBA method could be risky for real-world systems based on SAM. On the one hand, the PBA method exhibits strong cross-prompt transferability. On the other hand, SAM is a widely used visual foundation model. Therefore, we recommend adopting adversarial training when using SAM. This involves incorporating adversarial examples into SAM’s training dataset to enhance its security.

In future research, we will extensively focus on adversarial attacks and defenses targeting various types of foundation models, and explore defense methods against adversarial attacks in different application scenarios.