How Resilient Are Kolmogorov–Arnold Networks in Classification Tasks? A Robustness Investigation

Abstract

Kolmogorov–Arnold Networks (KANs) are a novel class of neural network architectures based on the Kolmogorov–Arnold representation theorem, which has demonstrated potential advantages in accuracy and interpretability over Multilayer Perceptron (MLP) models. This paper comprehensively evaluates the robustness of various KAN architectures—including KAN, KAN-Mixer, KANConv_KAN, and KANConv_MLP—against adversarial attacks, which constitute a critical aspect that has been underexplored in current research. We compare these models with MLP-based architectures such as MLP, MLP-Mixer, and ConvNet_MLP across three traffic sign classification datasets: GTSRB, BTSD, and CTSD. The models were subjected to various adversarial attacks (FGSM, PGD, CW, and BIM) with varying perturbation levels and were trained under different strategies, including standard training, adversarial training, and Randomized Smoothing. Our experimental results demonstrate that KAN-based models, particularly the KAN-Mixer, exhibit superior robustness to adversarial attacks compared to their MLP counterparts. Specifically, the KAN-Mixer consistently achieved lower Success Attack Rates (SARs) and Degrees of Change (DoCs) across most attack types and datasets while maintaining high accuracy on clean data. For instance, under FGSM attacks with $ϵ=0.01$, the KAN-Mixer outperformed the MLP-Mixer by maintaining higher accuracy and lower SARs. Adversarial training and Randomized Smoothing further enhanced the robustness of KAN-based models, with t-SNE visualizations revealing more stable latent space representations under adversarial perturbations. These findings underscore the potential of KAN architectures to improve neural network security and reliability in adversarial settings.

1. Introduction

The Kolmogorov–Arnold Networks (KANs), as introduced by Liu et al. [1], have attracted considerable attention as innovative neural network architectures, presenting a promising substitutional to Multilayer Perceptrons (MLPs). These networks are grounded in the theorem of Kolmogorov–Arnold representation (KART), which asserts that any multivariate continuous function can be expressed as a sum of continuous univariate functions [2,3].

KANs, much like MLPs, exhibit fully connected architectures. However, unlike MLPs, which utilize fixed activation functions on nodes, KANs employ adaptive and dynamically adjustable activation functions on edges. Consequently, KANs do not have linear weight matrices, as each weight parameter is substituted by a learnable 1D function represented as a spline. Despite this, KANs generally lead to much smaller computation graphs compared to MLPs [1].

KANs have demonstrated their adaptability across various domains, as illustrated in Figure 1. This figure showcases the development timeline of KANs, highlighting significant advancements in natural language processing, time series analysis, and other areas. For instance, KAN-GPT [4] implements Generative Pre-trained Transformers (GPTs), while KAN-GPT2 [5] involves training small GPT-2 style models using KANs instead of MLPs. Additionally, kansformers [6] explore the integration of KANs in transformer architectures by replacing the linear layers of MLP with KAN and Word-Level Explainable for language modeling [7]. Moreover, KANs have been applied in tasks such as ordinary differential equations [8], physical modeling [9], mechanics problems [10], partial differential equations [11], Quantum Variational KANs (VQ-KANs) [12], time series analysis [13,14,15,16,17,18], computer vision [19,20,21,22,23,24], and graph learning [25,26].

Various modifications have been made to enhance the capabilities of KANs through the replacement or combination of B-splines with other function bases to achieve more accurate solutions, as shown in Figure 2, starting with the Efficient-KAN [27] and Fast-KAN [28], which utilized Radial Basis Functions (RBFs) to enhance the speed of training. At the same time, the Chebyshev-KAN [29] introduced Chebyshev polynomials for nonlinear function approximation. The Wav-KAN [30] implemented wavelet transformations in the KAN architecture. The FasterKAN [31] combined FastKAN with Reflectional Switch Activation Functions (RSWAFs) to further benchmark the performance of KANs. The Gottlieb-KAN [32] explored the potential of polynomial basis functions, and the FourierKAN-GCF [33] leveraged Fourier transformations for efficient feature extraction in graph collaborative filtering. ReLU-KAN [34] simplified KANs by using only matrix addition, dot multiplication, and ReLU activation functions. In June 2024, fKAN [35] introduced trainable Jacobi basis functions, while BSRBF-KAN [36] combined B-splines and RBFs. Rational activation functions were explored with rKAN [37], and SineKAN [38] incorporated sinusoidal activation functions. Later advancements included UKAN [39], which eliminated the need for a bounded grid by introducing a coefficient generator model, and FC-KAN [40], which combined outputs from B-splines and Difference of Gaussians (DoGs) to improve accuracy.

Despite their wide applicability, the robustness of KANs against adversarial attacks remains insufficiently explored to the best of our knowledge. This is crucial for real-world AI scenarios, and it ensures the security, safety, and robustness of learning algorithms, which become increasingly critical, particularly in safeguarding against accidents and malicious agents. Studies [41,42] have revealed Deep Neural Network vulnerabilities that require further investigation and mitigation. Szegedy et al. [41] discovered that introducing imperceptible perturbations to input samples could significantly impact the performance of a target classifier. This discovery paved the way for a plethora of subsequent research focusing on designing attack methods against image recognition systems [42]. Furthermore, recent research has highlighted the susceptibility of KANs to noise. For instance, Zhang [43] pointed out their vulnerability to noise interference, which can significantly degrade performance. Similarly, Shen et al. [44] demonstrated that KANs show reduced effectiveness when dealing with functions that include noise. More recently, Chen et al. [45] compared KANs with MLPs on irregular or noisy functions, categorizing these functions into six types of categories, thus emphasizing the need for robust advancements.

This paper aims to bridge this gap by validating KANs for classification tasks using various KAN architectures and traffic sign datasets and evaluating their accuracy and success attack rates against adversarial attacks. The findings underscore that while KANs present promising results, they may not consistently outperform MLPs in highly complex scenarios. Nevertheless, KANs continue to evolve, with ongoing research exploring the application of KANs underscoring the need for further research to optimize their capabilities and comprehend their behavior in adversarial and practical settings. The main contributions of our work can be summarized as follows:

• It provides a comprehensive timeline of KAN developments, highlighting key applications and tracing the evolution of function bases and activation functions.
• It conducts an in-depth analysis of KAN-based models for classification tasks, including KAN, KAN-Mixer, KANConv_KAN, and KANConv_MLP, comparing their performance against MLP and Small MLP-Mixer, as well as ConvNet_MLP models across multiple datasets (BTSD, CTSD, and GTSRB). This evaluation highlights KAN architectures’ strengths and weaknesses in handling adversarial attacks at various perturbation levels.
• We explore the effectiveness of adversarial defenses such as adversarial training and randomized smoothing training scenarios in strengthening the resilience of KAN models to adversarial attacks.

The rest of the paper is structured as follows: Section 2, Materials and Methods, begins with the Kolmogorov–Arnold representation theorem, which serves as the theoretical foundation for KANs and the design and implementation of KANs into neural networks as well as their application in specific architectures like convolutional KAN and KAN-Mixer. Section 3 covers adversarial attacks and outlines the adversarial defense strategy. Section 4, Methodology, includes information on the datasets used; comparisons between KAN, MLP models, and KAN-based architectures like KAN-Mixer and convolutional KANs; and adversarial attack methods employed, as well as the training procedure and the evaluation metrics. Section 5, Results and Discussion, compares the performance outcomes of KAN vs. MLP models, KAN-Mixer vs. MLP-Mixer models, and KAN-convolution vs. convolution layers; finally, Section 6 concludes by presenting insights on the robustness of KAN-based models.

2. Materials and Methods

2.1. Kolmogorov–Arnold Representation Theorem

The foundation of the KAN is based on the KART, which states that any multivariate continuous function $f\left(x\right)$, defined in a bounded domain, can be represented as a finite sum of continuous single variable functions and the binary operation of addition [46,47]. Specifically, for a set of variables $x=\{x_1,x_2,\dots ,x_n\}$, a continuous multivariate function $f\left(x\right)$ can be expressed as follows [31]:

$$f\left(x\right)=f(x_1,\dots ,x_n)=\sum _{q=1}^{2n+1}{\Phi }_q\left(\sum _{p=1}^n{\phi }_{q,p}\left(x_p\right)\right)$$

(1)

The expression contains an outer sum and an inner sum. The outer sum ${\sum }_{q=1}^{2n+1}$ calculates the sum of $2n+1$ terms of the function ${\Phi }_q:\mathbb{R}\to \mathbb{R}$. The inner sum adds up n terms for each q, where ${\phi }_{q,p}:[0,1]\to \mathbb{R}$ is a continuous function of a single variable $x_p$. The theorem implies that understanding a function with many dimensions comes down to understanding a manageable number of one-dimensional functions. However, these one-dimensional functions can lack smoothness and even display fractal properties, which could make them challenging to learn in real-world applications. As a result, the Kolmogorov–Arnold representation theorem has been widely disregarded in the field of machine learning, being seen as theoretically sound but not practically effective [48,49].

Design of KANs

To fully understand KANs, exploring the MLP configuration is crucial. The MLP is characterized by having nonlinear fixed activation functions on its nodes, such as the ReLU function [50]. While some studies have used trainable activation functions such as [51,52,53,54], for now, we are focusing on an MLP with a fixed activation function. When a specific input x is fed into the MLP, it initiates operations involving the composition of weight matrices across multiple layers (ranging from layer 0 to layer $L-1$) and the application of the activation function $\sigma$. The result of this process is the generation of the final output, as described in the work by Pinkus (1999) [55].

$$\mathrm{MLP}\left(x\right)=\sigma \left(W_{L-1}\sigma \left(W_{L-2}\cdots \sigma \left(W_1\sigma \left(W_{0}x\right)\right)\right)\right)$$

(2)

Inspired by KART, KANs were designed by Liu et al. [1], who adopted a more optimistic perspective on the utility of the Kolmogorov–Arnold theorem for machine learning by implementing the following:

1.

Generalizing the Network Structure: Instead of adhering strictly to the original two-layer structure with 2n + 1 hidden units In Equation (1), they generalize KANs to have arbitrary widths and depths. The main challenge is to find the appropriate functions ${\Phi }_q$ and ${\phi }_{q,p}$. A general KAN network with L layers produces the output, as In Equation (3).

2.

Leveraging Smoothness and Sparsity: Many real-world functions are smooth and possess sparse compositional structures, facilitating more effective Kolmogorov–Arnold representations. This aligns with the physicist’s approach of focusing on typical cases rather than worst-case scenarios, assuming that physical and machine learning tasks inherently possess useful or generalizable structures [56].

$$\mathrm{KAN}\left(x\right)=\left({\Phi }_{L-1}\circ {\Phi }_{L-2}\circ \cdots \circ {\Phi }_1\circ {\Phi }_0\right)\left(x\right)$$

(3)

In the context of a KAN, ${\Phi }_l$ represents the function matrix of the lth KAN layer or a set of preactivations. We use ${\Phi }_l$ to denote the connections between the ith neuron of layer l and the jth neuron of layer $l+1$. The activation function ${\phi }_{l,j,i}$ connects the neuron at position $(l,i)$ to the neuron at position $(l+1,j)$.

$${\phi }_{l,j,i},l=0,\dots ,L-1,i=1,\dots ,n_l,j=1,\dots ,n_{l+1}$$

(4)

The number of nodes in the lth layer is denoted as $n_l$. The function matrix ${\Phi }_l$ can be expressed as a $n_{l+1}\times n_l$ matrix of activations.

$${\Phi }_l=\left(\begin{array}{cccc}{\phi }_{l,1,1}(·)& {\phi }_{l,1,2}(·)& \cdots & {\phi }_{l,1,n_l}(·)\\ {\phi }_{l,2,1}(·)& {\phi }_{l,2,2}(·)& \cdots & {\phi }_{l,2,n_l}(·)\\ ⋮& ⋮& \ddots & ⋮\\ {\phi }_{l,n_{l+1},1}(·)& {\phi }_{l,n_{l+1},2}(·)& \cdots & {\phi }_{l,n_{l+1},n_l}(·)\end{array}\right)$$

(5)

2.2. Implementation of KANs

The original KAN was implemented by Liu et al. [1] using a residual activation function $\phi \left(x\right)$, which is defined as the sum of a base function and a spline function with their corresponding weight matrices $w_b$ and $w_s$:

$$\phi \left(x\right)=w_{b}b\left(x\right)+w_s·\mathrm{spline}\left(x\right)$$

(6)

where $b\left(x\right)$ denotes the fundamental function, often implemented as SiLU, and $\mathrm{spline}\left(x\right)$ is represented as a linear combination of B-splines. The initial values for each activation function are $w_s=1$ and $\mathrm{spline}\left(x\right)\approx 0$, and $w_b$ is initialized using the Xavier initialization.

The Efficient KAN approach [27] improves efficiency by using B-spline basis functions $b\left(x\right)$ and linear combinations, reducing memory usage and simplifying calculations. The final output Y is computed as a sum of a base linear transformation and a spline-adjusted output:

$$Y=f_{activation}\left(x\right)W_{base}^T+B\left(x\right)W_{spline}$$

(7)

The authors replaced the incompatible L1 regularization on input samples with L1 regularization on spline weights, $L_1={\lambda }_{activation}\sum \left|W_{spline}\right|$, and introduced learnable scaling for activation functions. They also initialized base weights and spline scalers using Kaiming uniform initialization to enhance training stability.

As described in [28], the Fast KAN approach has enhanced training speed by utilizing Radial Basis Functions (RBFs) to approximate the 3rd-order B-spline. This method also incorporates layer normalization to ensure that the inputs remain within the RBFs’ range. These adjustments simplify the implementation process while preserving accuracy. The RBF is defined as follows:

$$\phi \left(r\right)=e^{-\varrho r^2}$$

(8)

The distance $r=x-c$ represents the Euclidean distance between the input vector x and the center vector c, and the parameter $\varrho >0$ is a sharpness value that determines the width of the Gaussian function. FastKAN utilizes a specific variation of RBFs in which we have the following:

$$r={\displaystyle \frac{x-c}{h}},\varrho ={\displaystyle \frac{1}{2}}$$

(9)

Thus, the equation becomes the following:

$$\phi \left(r\right)=exp\left(-{\displaystyle \frac{1}{2}}{\left({\displaystyle \frac{x-c}{h}}\right)}^2\right)$$

(10)

Finally, the RBF network with N centers can be expressed as follows:

$$\mathrm{RBF}\left(x\right)=\sum _{i=1}^N{w}_i\phi \left(r_i\right)=\sum _{i=1}^N{w}_{i}exp\left(-{\displaystyle \frac{1}{2}}{\left({\displaystyle \frac{x-c_i}{h}}\right)}^2\right)$$

(11)

The Faster KAN approach [31] signifies a progression from the Fast KAN approach by integrating Reflectional Switch Activation Functions (RSWAFs). These RSWAFs, modifications of RBFs, provide a computationally straightforward method owing to their uniform grid structure. The RSWAF is formally defined as follows:

$$\phi \left(r\right)=1-tanh\left({\displaystyle \frac{x-c}{h}}\right)$$

(12)

The RSWAF network with N centers is then given by:

$$\mathrm{RSWAF}\left(x\right)=\sum _{i=1}^N{w}_i\phi \left(r_i\right)=\sum _{i=1}^N{w}_i\left(1-tanh\left({\displaystyle \frac{x-c_i}{h}}\right)\right)$$

(13)

2.3. KAN-Mixer

The KAN-Mixer [23] architecture initiates with an input image X of shape $[B,C,H,W]$, where B represents the batch size, C signifies the number of channels, H denotes the height, and W specifies the width of the image. Subsequently, the image is fragmented into nonoverlapping patches using the image-to-patches module, each of size $P\times P$, leading to the creation of ${\displaystyle \frac{H}{P}}\times {\displaystyle \frac{W}{P}}$ patches. Ultimately, the image is restructured into a sequence of patches $X_{\mathrm{patches}}$ with shape $[B,N,P^2·C]$, where N signifies the total number of patches.

After processing each patch independently with the PerPatchKAN module, a KANLinear transformation is applied to project the patch into a higher-dimensional space, enhancing the representation of the patch data. The KAN token mixing layer facilitates inter-location communication by independently transforming each token through KANLinear layers, amalgamating spatial information while conserving channel information. Similarly, the Channel Mixing KAN layer enables inter-channel communication within each token by independently transforming each channel through KANLinear layers, amalgamating channel information while retaining spatial information.

The final output of the KAN module is derived by aggregating information from the transformed patches through the application of layer normalization, followed by computation of the mean across the token dimension to obtain a fixed-size representation. Subsequently, a KANLinear layer is employed to project the aggregated representation to the desired output dimension, ultimately yielding the final output Y with a shape of $[B,n_{\mathrm{output}}]$. Here, $n_{\mathrm{output}}$ denotes the number of output classes or the desired dimensionality of the output.

2.4. Convolutional Kolmogorov–Arnold Networks

Convolutional KANs [21,22] share conceptual similarities with Convolutional Neural Networks (CNNs). By substituting the scalar product in the convolution operation utilized in CNNs with an adaptive and dynamically nonlinear activation function applied to every element, KANs can be expanded to convolutions. This expansion, known as ConvKANs, introduces an adaptive and dynamically nonlinear activation function applied to every element in the convolution operation. The convolutional kernel used for ConvKANs can be represented, as described in [22]:

$$\mathrm{ConvKAN}\mathrm{Kernel}=\left(\begin{array}{cc}{\varphi }_{11}& {\varphi }_{12}\\ {\varphi }_{21}& {\varphi }_{22}\end{array}\right)$$

(14)

The calculation of a convolutional kernel based on the KAN is described as follows:

$$y_{i,j}=\sum _{m,n}{\varphi }_{m,n}\left(x_{i+m,j+n}\right)$$

(15)

After the flattening step, the architecture may include another KAN or MLP layer.

3. Adversarial Attacks and Defenses

Adversarial attacks and defense have become critical areas of research in deep learning, especially within safety-critical domains such as autonomous driving and healthcare. This section examines adversarial attacks and effective defense methods.

3.1. Adversarial Attacks

Adversarial attacks exploit vulnerabilities in neural networks by making subtle, often imperceptible changes to input data via perturbations called adversarial examples [41,42], leading to incorrect predictions or classifications. The following subsections discuss several widely used attack methods, including FGSM, PGD, CW, and BIM.

3.1.1. Fast Gradient Sign Method (FGSM)

The Fast Gradient Sign Method (FGSM), introduced by Goodfellow et al. [42], is one of the most widely used adversarial attack techniques due to its simplicity and efficiency. The FGSM functions by adjusting the input image by adding perturbations that increase the neural network’s loss function. This adjustment maximizes the model’s prediction error by leveraging the loss gradient concerning the input image. With an input image X, a true label y, and the model loss function $\mathcal{L}(X,y)$, the FGSM creates an adversarial instance $X^{\prime }$ by introducing a small perturbation in the direction of the gradient:

$$X^{\prime }=X+ϵ\times \mathrm{sign}\left({\nabla }_X\mathcal{L}(X,y)\right)$$

(16)

The value $ϵ$ represents the maximum permissible perturbation in the input data when subjected to adversarial attacks, determining the magnitude of the adversarial noise. A smaller $ϵ$ value can induce alterations that are imperceptible to the human visual system, while a larger $ϵ$ signifies a more pronounced perturbation, increasing the likelihood of model misclassifications.

3.1.2. Projected Gradient Descent (PGD)

Projected Gradient Descent (PGD) is an iterative extension of the FGSM introduced by Madry et al. [57]. PGD applies multiple small perturbations iteratively to the input, refining the adversarial example over several iterations. After each iteration, the perturbed input is projected back onto the $ϵ$-ball around the original input to ensure that the perturbation remains within a specified limit. The update rule for PGD is given by the following:

$$X_{i+1}={\mathrm{clip}}_{X,ϵ}\left(X_i+\alpha \times \mathrm{sign}\left({\nabla }_X\mathcal{L}(X,y)\right)\right)$$

(17)

where $X_i$ represents the modified input at step i, $\alpha$ denotes the step size, and $ϵ$ indicates the maximum allowable perturbation. PGD is considered a more powerful attack compared to the FGSM due to its iterative refinement, increasing the likelihood of deceiving the model.

3.1.3. Carlini–Wagner (CW) Attack

The Carlini–Wagner (CW) attack [58] is another iterative method that minimizes the $L_2$ norm of the perturbation while maximizing the loss function. It solves the following optimization problem:

$$\underset{\delta }{\min }{\parallel \delta \parallel }_2+c·\max (0,f(x+\delta )-f\left(x\right))$$

(18)

where $\delta$ is the perturbation, c is a regularization parameter, and $f\left(x\right)$ represents the model’s logits.

3.1.4. Basic Iterative Method (BIM)

The Basic Iterative Method (BIM) or Iterative-FSGM [59] is an iterative version of the FGSM attack. Instead of applying a single perturbation as in FGSM, the BIM applies multiple small perturbations iteratively. After each iteration, the adversarial input is clipped to ensure that it remains within an $ϵ$-ball around the original input. The update rule for BIM is as follows:

$$X_{i+1}={\mathrm{clip}}_{X,ϵ}\left(X_i+\alpha \times \mathrm{sign}\left({\nabla }_X\mathcal{L}(X,y)\right)\right)$$

(19)

where $X_i$ represents the modified input at iteration i, $\alpha$ is the step size, and $ϵ$ is the maximum allowable perturbation. The BIM is often considered a stronger attack than the FGSM, as it allows for finer-grained control over the perturbation.

3.2. Defense Methods

In response to adversarial attacks, several defense methods have been proposed to increase the robustness of neural networks. This section covers two prominent defense strategies employed in this study: adversarial training and randomized smoothing.

3.2.1. Adversarial Training

Adversarial training is one of the most effective methods for increasing model robustness against adversarial attacks. Adversarial training is a method used to train a model using both clean and adversarial examples. By exposing the model to adversarial perturbations during training, it learns to identify and defend against these manipulations, thus enhancing its robustness. The process of adversarial training can be defined as solving a minimax problem, where the objective is to discover model parameters that minimize the maximum potential loss, as proposed by Madry et al. [57]:

$$\underset{\theta }{\min }\rho \left(\theta \right),\mathrm{where}\rho \left(\theta \right)={\mathbb{E}}_{(x,y)\sim \mathbb{D}}\left[\underset{\delta \in S}{\max }\mathcal{L}(\theta ,x+\delta ,y)\right]$$

(20)

where the $\theta$ represents the model parameters, $\mathcal{L}(\theta ,x+\delta ,y)$ is the loss function evaluated at the perturbed input $x+\delta$ with actual label y, $\delta \in S$ represents the adversarial perturbation within a specific allowable set S, and ${\mathbb{E}}_{(x,y)\sim \mathbb{D}}$ denotes the expectation of the data distribution $\mathbb{D}$. The inner maximization (with respect to $\delta$) represents the adversarial attack, which tries to find the worst-case perturbation that maximizes the loss. The outer minimization (with respect to $\theta$) represents the adversarial training process, which aims to find model parameters that minimize the worst-case loss.

3.2.2. Randomized Smoothing

Randomized smoothing is another defense mechanism employed to increase the robustness of models against adversarial attacks. It transforms a deterministic classifier into a probabilistic one by adding Gaussian noise to the input during evaluation, effectively smoothing out the model’s decision boundaries. The smoothed classifier is less sensitive to small perturbations in the input space, making it more robust for adversarial examples. The process of randomized smoothing can be described as follows:

$$x_{\mathrm{smoothed}}=x+\mathcal{N}(0,{\sigma }^2)$$

(21)

where $\mathcal{N}(0,{\sigma }^2)$ represents Gaussian noise with a mean of 0 and a variance of ${\sigma }^2$. By averaging the predictions across multiple noisy versions of the input, the classifier’s decision boundaries become less susceptible to small perturbations. In this study, randomized smoothing was applied during the evaluation phase, with a noise variance of $\sigma =0.1$. This strategy improved model robustness, especially under strong adversarial attacks like PGD and CW.

4. Methodology

In this study, we systematically evaluate the robustness of several neural network architectures against adversarial attacks across different training conditions and datasets. The focus is on understanding how Kolmogorov–Arnold Networks (KAN) and Multilayer Perceptron (MLP) architectures respond to adversarial perturbations, specifically FGSM, BIM, PGD, and CW attacks, using specific parameter configurations such as epsilon values, step sizes, and iteration counts. The experiments were conducted on system equipped with an Intel(R) Core(TM) i9-14900K 3.20 GHz processor, 64 GB of RAM, and two NVIDIA GeForce RTX 3090 GPUs. We explored three distinct training conditions: standard, adversarial, and randomized smoothing training, and evaluated various architectures, including KAN, MLP, KAN-Mixer, and MLP-Mixer, as well as KAN-convolution and convolution layer models, across three traffic sign classification datasets. The performance and resilience of these models are assessed using key evaluation metrics, including accuracy, SAR, and DoC, to analyze their vulnerabilities and robustness in clean and adversarial conditions. Additionally, t-SNE visualizations were employed to qualitatively assess the latent space representation of clean and adversarial examples. This provided deeper insights into the robustness characteristics of each model. Detailed methodologies and analyses of the different models are discussed in the corresponding subsections.

4.1. Datasets

In our experiments, we utilized three different traffic sign recognition datasets to evaluate the performance and robustness of our KAN models: The Belgium Traffic Sign Dataset (BTSD) [60], the Chinese Traffic Sign Database (CTSD) [61], and the German Traffic Sign Recognition Benchmark (GTSRB) [62]. The BTSD includes a total of 62 classes categorized into three superclasses: mandatory, prohibitive, and danger classes. The dataset is divided into 4591 training images and 2534 test images, providing variety in sign appearances to assess model robustness, as depicted in Figure 3a. China’s CTSD comprises 6164 images in 58 categories, with 4170 training images and 1994 test images, offering a well-annotated dataset frequently used in traffic sign recognition research, illustrated in Figure 3b. Lastly, The GTSRB dataset, widely used to benchmark traffic sign classification models, contains 43 classes with 39,209 training images and 12,630 test images, offering a diverse representation of German traffic signs, as shown in Figure 3c. We observed that certain classes within the BTSD and CTSD had a limited number of images in both the training and test sets, resulting in class imbalance. Our experiments employed various data augmentation techniques to address this issue, such as rotation, flipping, scaling, and adding noise and adversarial examples during adversarial training.

4.2. Kolmogorov–Arnold Networks and Multilayer Perceptron

Our research investigates the resilience of two neural network architectures, namely the KAN and MLP models, against various adversarial attacks and defense strategies. These models were evaluated on multiple classification datasets under different attack scenarios, including the FGSM, PGD, CW, and BIP attacks. Additionally, the models were tested under standard, randomized smoothing, and adversarial training conditions. We also integrated randomized smoothing as a defense mechanism to further assess and enhance robustness.

These KANLinear layers are key to the model’s flexibility. They use grid-based kernels with $gridsize=5$ and $splineorder=3$, which were scaled by 1.0. The grid was dynamically updated within a range of −1 to 1 using a grid eps of 0.02, enhancing its adaptability to the data distribution. The activation function used in the KAN model is SiLU, which allows for smoother gradient flow during backpropagation. Regularization is enforced through adaptive B-splines, which helps control overfitting. Following the configuration outlined in Efficient KAN [27], see Equation (6).

In contrast, the MLP model is a baseline comparison to the KAN architecture. The MLP consists of two fully connected hidden layers, identical in structure to the KAN model, with 256 units in the first layer and 128 units in the second layer. The activation function used in the MLP is ReLU, which is commonly used for its simplicity and computational efficiency. Both models’ output layers consist of units corresponding to the number of classes in the datasets. For training, we used the AdamW optimizer with a learning rate of 0.001 and a weight decay of 1 × ${10}^{-4}$, involving training over 100 epochs with a batch size of 64. The cross-entropy loss function was employed to evaluate classification performance. This consistent training setup ensured a fair comparison between the KAN and MLP models under various adversarial attack scenarios, including FGSM, PGD, CW, and BIM attacks, as well as under standard, randomized smoothing, and adversarial training conditions, with the integration of randomized smoothing as a defense mechanism.

As shown in Figure 4, the left side illustrates how MLP employs ReLU activation functions, and on the right side, the KAN architecture is designed with B-spline-based activation functions.

4.3. KAN-Mixer and MLP-Mixer

This section explores the resilience of two sophisticated neural network architectures, KAN-Mixer [23] and MLP-Mixer [63] as illustrated in Figure 5, against adversarial attacks. The models are trained and evaluated on various datasets to assess their performance under clean and adversarial conditions using FGSM, PGD, CW, and BIM attacks. The KAN-Mixer and MLP-Mixer architectures start by dividing the input image into patches using a convolutional layer during the patch embedding stage.

In the KAN-Mixer architecture, each patch undergoes B-spline-based transformations facilitated by the KAN-Linear layers [23], enabling the model to learn and represent patch-specific features adeptly. Following this, the Mixer Stack—a core component of both architectures—comprises multiple layers that alternate between token-mixing and channel-mixing modules. Specifically, in the KAN-Mixer, the token-mixing modules employ Token-Mixing-KAN layers to amalgamate information across different patches, while the channel-mixing modules utilize channel-mixing-KAN layers to integrate information across the channels within each individual patch. The culmination of these processes involves global average pooling, which is succeeded by a linear KAN Linear layer equipped with SiLU activation functions applied consistently throughout the KAN layers.

As for the small MLP-Mixer architecture, inspired by Tolstikhin et al. [63], it processes each patch by applying Mixer Layers, which consist of token-mixing and channel-mixing MLP blocks. Token mixing involves processing patch information through linear transformations, activation, and dropout, while channel mixing processes information within each channel using fully connected layers. Layer normalization is applied before mixing to stabilize the training. The final output layer converts the token channel representation into class predictions, with GELU activation functions used for the MLP layer.

4.4. KAN-Convolution and Convolution Layer

In this section, we investigate the effectiveness of KANs and CNN for traffic sign classification. We implemented and compared four different architectures, ConvNet_MLP, ConvNet_KAN, KANConv_MLP, and KANConv_KAN, based on the research by [21,22]. All models were trained using a batch size of 512 over 100 epochs, leveraging the Adam optimizer with a learning rate of 0.001 and employing cross-entropy loss as the primary loss function.

The ConvNet_MLP model is the baseline architecture combining standard convolutional layers with a fully connected MLP. It consists of four convolutional layers: two with 32 filters and a kernel size of 5 × 5, followed by two with 64 filters and a kernel size of 3 × 3. These layers are interspersed with ReLU activations and 2 × 2 max-pooling layers, leading to fully connected layers with 256 neurons for class output.

The ConvNet_KAN model builds on this by replacing the fully connected layers with a KAN linear layer, using a 10 × 10 grid and a spline order of 3 to model complex nonlinear relationships. This architecture retains the initial convolutional layers from the ConvNet_MLP model but introduces the KAN layer after flattening the feature maps to capture more sophisticated patterns in the data.

The KANConv_MLP model further explores the potential of a KAN by incorporating it directly into the convolutional layers. It employs two KAN-based convolutional layers, with each consisting of five independent 3 × 3 convolutions, followed by max-pooling. These KAN layers aim to enhance the model’s feature extraction capabilities by leveraging the adaptive spline functions of the KAN during convolution. The output of these KAN-based convolutions is then passed through fully connected layers—similar to those in the ConvNet_MLP model.

Finally, the KANConv_KAN model fully integrates the KAN into the convolutional and fully connected layers using KAN-based layers throughout the network. It applies two KAN convolutional layers, similar to those in KANConv_MLP, and follows them with a KAN linear layer for classification. This model is designed to fully leverage the KAN’s nonlinear, adaptive properties to potentially achieve better performance in recognizing traffic signs, especially under adversarial conditions.

The architecture depicted in Figure 6 showcases the combination of convolutional layers with KAN components to improve robustness in classification tasks.

4.5. Evaluation Metrics

The models were evaluated based on several metrics to assess their performance and robustness against adversarial attacks. Accuracy, Success Attack Rate (SAR), and Degree of Change (DoC) were the primary metrics used. Accuracy measures the percentage of correct predictions in the test dataset, while SAR measures the percentage of adversarial attacks that have successfully attacked and misclassified the model based on the total number of adversarial attacks. In contrast, DoC measures the average magnitude of perturbations introduced by the adversarial attack. The results were recorded for standard, randomized smoothing, and adversarial training conditions. The standard training models were evaluated on clean and adversarially perturbed data (FGSM, PGD, CW, and BIM attacks). In adversarial training, the models trained with adversarial examples were evaluated on clean and adversarial data to assess the effectiveness of adversarial training. Lately, in randomized smoothing training, the defense technique augments training by adding Gaussian noise to the input data. This smooths the model’s decision boundary, making it less sensitive to small perturbations and increasing its robustness against adversarial attacks.

Accuracy metric measures the percentage of correct predictions on the test dataset:

$$\mathrm{Accuracy}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}}$$

(22)

where TP denotes True Positives, TN denotes True Negatives, FP denotes False Positives, and FN denotes False Negatives.

The Success Attack Rate (SAR) of adversarial attacks quantifies the effectiveness of an attack by measuring the proportion of inputs where the model’s prediction changes from correct on clean data to incorrect on adversarial data. It is defined as follows:

$$\mathrm{SAR}={\displaystyle \frac{1}{N}}\sum _{i=1}^N\left[\mathbb{I}\left({\widehat{y}}_i^{\mathrm{clean}}=y_i\right)·\mathbb{I}\left({\widehat{y}}_i^{\mathrm{adv}}\ne y_i\right)\right]\times 100\%$$

(23)

where N is the total number of samples, $y_i$ is the true label of the ith sample, ${\widehat{y}}_i^{\mathrm{clean}}$ is the model’s prediction on the clean input, ${\widehat{y}}_i^{\mathrm{adv}}$ is the model’s prediction on the adversarial input, and $\mathbb{I}(·)$ is the indicator function, which equals 1 if the condition is true and 0 otherwise.

The Degree of Change (DoC) measures the average magnitude of perturbations introduced by the adversarial attack, quantifying how much the adversarial images differ from the original images. It is calculated as the average $L_2$ norm of the difference between the original and adversarial images:

$$\mathrm{DoC}={\displaystyle \frac{1}{N}}\sum _{i=1}^N{∥{\mathbf{x}}_i^{\mathrm{adv}}-{\mathbf{x}}_i∥}_2$$

(24)

where ${\mathbf{x}}_i$ is the original (clean) input image of the ith sample, ${\mathbf{x}}_i^{\mathrm{adv}}$ is the adversarially perturbed image of the ith sample, and ${\parallel ·\parallel }_2$ denotes the $L_2$ norm (Euclidean distance).

5. Results and Discussion

In this section, we present the experimental results evaluating the robustness of various neural network architectures against adversarial attacks. The models under consideration included the KAN, MLP, KAN-Mixer, MLP-Mixer, KANConv_KAN, ConvNet_MLP, ConvNet_KAN, and KANConv_MLP. These models were evaluated across three datasets: BTSD, CTSD, and GTSRB. The experiments encompass a wide range of adversarial attack scenarios, including FGSM, PGD, CW, and BIM attacks, with varying epsilon values $ϵ=0.01,0.1,1$ and $\alpha =0.01$ for BIM, PGD, and CW attacks, and the number of iterations 7 for BIM and PGD, with 1000 iterations for CW. Furthermore, the models were trained with standard training methods and various defense strategies, such as adversarial training and randomized smoothing training, to assess their robustness against adversarial attacks. The evaluation metrics used were the Accuracy, SAR, and DoC. The Accuracy and SAR were evaluated on all test sets of clean and adversarial attacks, and the DoC was evaluated on 200 random samples, capturing the average perturbation magnitude. Finally, to gain further insights into the models’ ability to distinguish between clean and adversarial examples, we utilized t-SNE visualizations. This analysis allowed us to qualitatively assess how well the models can distinguish between the two types of examples based on their latent space representations.

5.1. KAN and MLP Models

Table 1. Performance metrics for KAN and MLP across GTSRB, BTSRB, and CTSRD datasets.

TT	AT	$\mathit{ϵ}$	GTSRB						BTSRB						CTSRD
			KAN			MLP			KAN			MLP			KAN			MLP
			Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC
Standard	N	-	79.11	-	0	82.19	-	0	88.73	-	0	90.08	-	0	97.16	-	0	97.57	-	0
	FGSM	0.01	27.03	53.11	14.82	35.34	48.74	14.82	45.12	44.33	13.49	60.79	30.60	13.49	34.63	62.61	10.73	49.55	48.09	10.73
	FGSM	0.1	11.22	68.04	14.90	16.78	66.03	14.90	25.00	63.77	13.57	36.63	53.61	13.57	9.49	87.75	10.94	11.11	86.54	10.94
	FGSM	1	0.24	78.88	21.83	1.24	80.95	21.82	0.16	88.57	21.62	0.40	89.68	21.38	0.24	96.92	20.02	0.57	97.00	20.13
	BIM	0.01	77.28	2.10	0.28	80.54	1.69	0.28	87.38	1.35	0.28	89.21	0.87	0.28	96.92	0.24	0.28	96.59	1.05	0.28
	BIM	0.1	60.30	20.18	2.55	66.00	16.43	2.55	75.00	13.93	2.53	79.01	11.11	2.53	74.70	22.63	2.40	78.51	19.14	2.40
	BIM	1	5.33	73.79	15.54	8.56	73.74	15.50	18.85	69.88	14.22	23.53	66.63	14.19	5.35	91.81	11.46	6.57	91.00	11.42
	PGD	0.01	26.05	53.99	14.82	34.12	49.91	14.82	44.37	45.08	13.49	61.03	30.40	13.49	33.98	63.26	10.73	48.34	49.31	10.73
	PGD	0.1	11.95	67.27	14.86	17.82	65.08	14.86	30.52	58.33	13.53	44.76	45.60	13.53	15.09	82.16	10.85	19.55	78.10	10.84
	PGD	1	2.16	76.95	17.87	4.89	77.30	17.85	10.32	78.41	16.79	12.06	78.10	16.89	1.70	95.46	15.19	2.51	95.05	15.20
	CW	-	26.87	52.24	11.39	34.66	47.54	9.85	44.92	43.81	9.72	60.52	29.56	8.22	36.90	60.26	7.07	52.39	45.17	5.82
Adversarial Training	None	-	79.02	-	0	82.23	-	0	88.69	-	0	90.04	-	0	97.16	-	0	97.57	-	0
	FGSM	0.01	27.53	52.57	14.82	35.50	48.63	14.82	45.36	44.05	13.49	60.87	30.56	13.49	35.69	61.56	10.73	49.80	47.85	10.73
	FGSM	0.1	11.31	67.84	14.90	16.87	65.95	14.90	25.20	63.53	13.57	36.67	53.53	13.57	9.98	87.27	10.94	11.03	86.62	10.94
	FGSM	1	0.21	78.80	21.83	1.25	80.97	21.82	0.16	88.53	21.62	0.40	89.64	21.38	0.24	96.92	20.01	0.57	97.00	20.13
	BIM	0.01	77.13	2.10	0.28	80.52	1.72	0.28	87.38	1.31	0.28	89.21	0.83	0.28	96.92	0.24	0.28	96.59	1.05	0.28
	BIM	0.1	60.34	20.09	2.55	65.97	16.48	2.55	75.00	13.89	2.53	79.05	11.03	2.53	74.53	22.79	2.40	78.35	19.30	2.40
	BIM	1	5.32	73.70	15.54	8.55	73.78	15.50	19.01	69.68	14.22	23.57	66.55	14.19	5.35	91.81	11.46	6.57	91.00	11.42
	PGD	0.01	26.65	53.36	14.82	34.17	49.89	14.82	44.68	44.72	13.49	61.07	30.36	13.49	34.63	62.61	10.73	48.42	49.23	10.73
	PGD	0.1	12.08	67.03	14.87	17.90	65.03	14.86	30.79	58.06	13.53	44.76	45.60	13.53	15.98	81.27	10.85	19.95	77.70	10.84
	PGD	1	2.23	76.79	17.87	4.71	77.52	17.84	10.60	78.10	16.79	12.06	77.98	16.87	1.46	95.70	15.18	2.27	95.30	15.22
	CW	-	27.40	51.62	11.30	34.70	47.52	9.85	45.24	43.45	9.72	60.48	29.56	8.22	37.47	59.69	7.03	52.80	44.77	5.82
Randomized Smoothing	None	-	77.88	-	0	80.74	-	0	88.73	-	0	89.92	-	0	97.24	-	0	97.32	-	0
	FGSM	0.01	27.97	51.25	14.90	34.76	47.41	14.90	46.11	43.33	13.57	61.15	30.24	13.57	35.69	61.80	10.94	49.72	47.93	10.94
	FGSM	0.1	11.58	66.33	14.99	16.90	64.06	14.99	26.75	62.06	13.65	37.42	52.46	13.65	9.41	87.59	11.12	11.84	85.89	11.12
	FGSM	1	0.25	77.66	21.89	1.14	79.89	21.88	0.08	88.69	21.66	0.32	89.72	21.41	0.16	97.08	20.04	0.57	97.00	20.14
	BIM	0.01	75.53	3.70	2.70	77.43	4.27	2.70	87.46	1.83	2.72	88.57	1.31	2.71	96.68	0.49	2.73	96.76	0.89	2.73
	BIM	0.1	59.56	19.95	3.66	63.33	18.16	3.66	75.44	13.37	3.63	78.81	10.91	3.63	73.80	23.68	3.53	78.26	19.22	3.54
	BIM	1	5.61	72.30	15.58	8.57	72.19	15.55	19.25	69.56	14.27	24.09	65.64	14.25	5.51	91.73	11.62	6.81	90.35	11.58
	PGD	0.01	27.09	51.83	14.90	33.90	48.13	14.90	45.24	44.01	13.57	60.68	30.71	13.57	34.79	62.53	10.94	49.31	48.26	10.94
	PGD	0.1	12.88	65.38	14.95	18.41	62.72	14.95	31.47	57.38	13.61	45.71	44.88	13.61	15.49	81.67	11.04	20.68	77.05	11.03
	PGD	1	2.70	75.24	17.94	4.98	75.75	17.91	11.71	77.02	16.87	12.30	77.58	16.96	1.78	95.46	15.26	3.16	93.99	15.27
	CW	-	29.54	48.88	11.95	38.92	42.41	10.86	46.55	42.38	10.61	65.16	25.08	9.12	39.90	57.58	8.30	58.96	38.44	6.89

TT: Training Type, AT: Attack Type, Acc: Accuracy, SAR: Success Attack Rate, and DoC: Degree of Change on 200 random samples.

illustrates the performance of the KAN and MLP models under three training conditions: standard training, adversarial training, and randomized smoothing. The models were evaluated on three datasets—GTSRB, BTSRB, and CTSRD—using various adversarial attack types, including the FGSM, BIM, PGD, and CW, at different perturbation levels ($ϵ$). The key metrics used to assess performance were the Acc, SAR, and DoC, providing insight into the models’ robustness under different attack scenarios.

In the standard training condition, without any adversarial defense mechanisms in place, both the KAN and MLP models performed well on the BTSRB dataset, achieving notable accuracy rates. KAN reached an accuracy of 88.73%, while MLP outperformed it slightly with an accuracy of 90.08%. However, both models experienced a significant decrease in accuracy when subjected to adversarial attacks, particularly at higher perturbation levels. Under the FGSM attack with an epsilon ($ϵ$) value of 0.01, the KAN’s accuracy on GTSRB dropped to 27.03%, while MLP’s accuracy fell to 35.34%. This highlights the susceptibility of both models to gradient-based attacks, with MLP showing slightly better resilience. As the epsilon value increased to 1, both models suffered a dramatic decline in accuracy, with the KAN achieving only 0.24% accuracy on GTSRB and MLP achieving 1.24%. The SAR values also rose significantly, surpassing 80%, indicating the models’ vulnerability to FGSM attacks. Similar trends were observed with the BIM and PGD attacks, where both models faced substantial reductions in accuracy, especially at higher epsilon values. For example, under the BIM attack with epsilon set at 1, the KAN attained an accuracy of 5.33% on GTSRB, while MLP performed slightly better at 8.56%. Despite this minor advantage, both models struggled to resist adversarial attacks, exhibiting high SAR values and significant changes in their predictions. In contrast, the CW attack had a relatively less severe impact on both models, with KAN maintaining an accuracy of 26.87% on GTSRB and MLP achieving 34.66%.

Adversarial training, incorporating adversarial examples, improved the robustness of both the KAN and MLP. This approach enhanced their resilience, though the improvement varied by dataset and attack type. For the FGSM attack ($ϵ$ = 0.01), the KAN achieved 27.53% accuracy on GTSRB, while MLP reached 35.50%. As $ϵ$ increased to 1, both models’ accuracies fell, with MLP still leading at 1.25% compared to the KAN’s 0.21%. The successful adversarial rate (SAR) remained high for all attack types, indicating that adversarial training helps but does not fully protect against stronger attacks such as BIM and PGD. With the BIM ($ϵ$ = 1), MLP scored 8.55% accuracy, compared to the KAN’s 5.32%. Under PGD ($ϵ$ = 1), MLP had 4.71% accuracy, while the KAN lagged at 2.23%. Overall, adversarial training improved both models’ robustness, but it was insufficient against high-intensity attacks.

Randomized smoothing was assessed as a certification-based defense mechanism that adds noise during inference to enhance model robustness. This approach proved especially effective for the Multilayer Perceptron (MLP). Under the FGSM attack with $ϵ=0.01$, the KAN model achieved 27.97% accuracy on the GTSRB, while MLP reached 34.76%. As $ϵ$ increased to 1, MLP maintained a significant advantage with 1.14% accuracy, compared to the KAN’s 0.25%. SAR values were lower in randomized smoothing, with MLP showing better resistance across most attacks. For the BIM and PGD attacks, MLP consistently outperformed the KAN, achieving 8.57% and 4.98% accuracy, respectively, while the KAN reached 5.61% and 2.70%. In CW attacks, MLP achieved 38.92% accuracy on GTSRB, surpassing the KAN’s 29.54%.

In comparing the three training methods, randomized smoothing consistently outperformed both standard and adversarial training, particularly for MLPs, which achieved higher accuracy and lower SAR values. The KAN method showed some resilience but was less robust than MLPs, especially against high-intensity attacks like BIM and PGD. While adversarial training offered moderate improvements, it was less effective than randomized smoothing against stronger adversarial attacks.

Figure 7 illustrates the loss values during the training of the KAN and MLP models across the GTSRB, BTSRB, and CTSRD datasets using different training strategies: standard, FGSM, and randomized smoothing. The x axis represents the training epochs, while the y axis shows the corresponding loss values.

In the standard training scenario, both models demonstrate a sharp decline in loss during the early epochs, with MLP converging more quickly than the KAN, particularly on the GTSRB and CTSRD datasets. For the BTSRB dataset, the KAN exhibited a more gradual decrease in loss but eventually reaches stability, maintaining a performance level similar to that of MLP after several epochs.

When employing the FGSM as an attack strategy, both the KAN and MLP show a comparable trend in loss reduction; however, the overall loss remains higher than in the standard training scenario. The MLP model displayed slightly better stability in reducing loss across all datasets, though the KAN’s performance on the BTSRB dataset indicates greater resilience, especially in the later epochs.

Under randomized smoothing, both models achieved significant loss reduction. However, the KAN consistently maintained a lower loss than MLP, particularly on the CTSRD and BTSRB datasets. This highlights the KAN’s superior ability to mitigate the impact of adversarial examples, especially when trained with smoothing techniques. These results underscore the KAN’s advantage in robustness when utilizing randomized smoothing, which is a key finding in this study.

5.2. KAN-Mixer vs. MLP-Mixer Models

Table 2. Performance metrics across MLP-Mixer and KAN-Mixer models.

TT	AT	$\mathit{ϵ}$	GTSRB						BTSRB						CTSRD
			KAN-Mixer			MLP-Mixer			KAN-Mixer			MLP-Mixer			KAN-Mixer			MLP-Mixer
			Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC
Standard	none	-	0.715	0	0	0.74	0	0	0.97	0	0	0.985	0	0	0.88	0	0	0.925	0	0
	FGSM	0.01	0.285	0.445	26.97	0.405	0.34	26.97	0.295	0.675	21.47	0.395	0.59	21.47	0.265	0.62	29.63	0.355	0.58	29.63
	FGSM	0.1	0.045	0.67	27.13	0.035	0.705	27.13	0.07	0.9	21.87	0.005	0.98	21.87	0.115	0.765	29.81	0.11	0.815	29.81
	FGSM	1	0.005	0.71	42.56	0	0.74	42.56	0.01	0.96	39.79	0	0.985	39.82	0.02	0.86	43.79	0	0.925	43.81
	BIM	0.01	0.585	0.13	0.55	0.68	0.06	0.55	0.855	0.115	0.55	0.97	0.015	0.55	0.825	0.06	0.55	0.92	0.005	0.55
	BIM	0.1	0.235	0.48	4.88	0.295	0.445	5.02	0.21	0.76	4.43	0.305	0.68	4.67	0.545	0.34	5.02	0.645	0.28	5.07
	BIM	1	0	0.715	27.69	0	0.74	28.01	0	0.97	22.11	0	0.985	22.29	0.015	0.865	30.53	0.03	0.895	30.58
	PGD	0.01	0.095	0.625	26.97	0.315	0.43	26.97	0.135	0.835	21.47	0.255	0.73	21.47	0.2	0.68	29.63	0.285	0.64	29.63
	PGD	0.1	0	0.715	27.04	0.005	0.735	27.05	0.015	0.955	21.63	0.005	0.98	21.66	0.035	0.845	29.72	0.06	0.865	29.72
	PGD	1	0	0.715	33.05	0	0.74	33.32	0	0.97	29.91	0	0.985	30.04	0	0.88	35.31	0	0.925	35.48
	CW	-	0.28	0.435	19.46	0.395	0.345	16.87	0.27	0.7	16.95	0.385	0.6	14.64	0.3	0.58	22.39	0.32	0.605	22.39
Adversarial	none	-	0.31	0	0	0.36	0	0	0.305	0	0	0.205	0	0	0.315	0	0	0.2	0	0
	FGSM	0.01	0.56	0.035	26.97	0.63	0.04	26.97	0.83	0.035	21.47	0.895	0	21.47	0.5	0.03	29.63	0.47	0.02	29.63
	FGSM	0.1	0.53	0.06	27.13	0.555	0.08	27.13	0.945	0.01	21.87	0.905	0	21.87	0.49	0.07	29.81	0.475	0.035	29.81
	FGSM	1	0.02	0.3	42.33	0.05	0.33	42.45	0.15	0.28	39.57	0.11	0.19	39.58	0.09	0.3	43.48	0.045	0.19	43.82
	BIM	0.01	0.28	0.03	0.55	0.29	0.07	0.55	0.3	0.005	0.55	0.175	0.03	0.55	0.29	0.025	0.55	0.15	0.05	0.55
	BIM	0.1	0.025	0.285	5.04	0.025	0.335	4.99	0.155	0.155	4.73	0.065	0.14	4.66	0.14	0.175	5.08	0.055	0.16	5.02
	BIM	1	0	0.31	27.89	0	0.36	27.82	0.05	0.255	22.31	0.055	0.15	22.25	0.015	0.3	30.48	0.005	0.195	30.40
	PGD	0.01	0.455	0.05	26.97	0.495	0.06	26.97	0.645	0.05	21.47	0.795	0	21.47	0.43	0.05	29.63	0.395	0.035	29.63
	PGD	0.1	0.085	0.26	27.05	0.18	0.24	27.05	0.165	0.175	21.67	0.225	0.085	21.67	0.135	0.22	29.72	0.085	0.155	29.72
	PGD	1	0	0.31	33.20	0	0.36	33.18	0	0.305	30.01	0	0.205	29.94	0	0.315	35.30	0.005	0.195	35.19
	CW	-	0.275	0.035	12.49	0.315	0.045	11.04	0.27	0.035	6.01	0.205	0	3.03	0.285	0.03	17.22	0.17	0.03	17.90
Randomized Smoothing	none	-	0.805	0	0	0.72	0	0	0.98	0	0	0.95	0	0	0.87	0	0	0.805	0	0
	FGSM	0.01	0.36	0.445	26.97	0.455	0.28	26.97	0.545	0.44	21.47	0.405	0.545	21.47	0.32	0.57	29.63	0.275	0.555	29.63
	FGSM	0.1	0.075	0.73	27.13	0.255	0.47	27.13	0.16	0.82	21.87	0.225	0.725	21.87	0.175	0.695	29.81	0.17	0.65	29.81
	FGSM	1	0	0.805	42.57	0	0.72	42.57	0	0.98	39.78	0.01	0.94	39.80	0.02	0.85	43.81	0.03	0.775	43.80
	BIM	0.01	0.78	0.025	0.55	0.67	0.05	0.55	0.97	0.01	0.55	0.94	0.01	0.55	0.855	0.015	0.55	0.765	0.04	0.55
	BIM	0.1	0.525	0.285	5.03	0.35	0.37	5.01	0.565	0.415	4.73	0.41	0.54	4.67	0.71	0.18	5.08	0.49	0.33	5.03
	BIM	1	0	0.805	28.12	0	0.72	27.97	0.02	0.96	22.44	0.005	0.945	22.29	0.045	0.825	30.91	0.015	0.79	30.54
	PGD	0.01	0.33	0.475	26.97	0.39	0.34	26.97	0.44	0.545	21.47	0.285	0.665	21.47	0.295	0.59	29.63	0.235	0.595	29.63
	PGD	0.1	0.05	0.755	27.05	0.125	0.595	27.05	0.08	0.9	21.67	0.05	0.9	21.67	0.095	0.775	29.72	0.065	0.755	29.72
	PGD	1	0	0.805	33.41	0	0.72	33.24	0	0.98	30.10	0	0.95	29.97	0.005	0.865	35.65	0	0.805	35.33
	CW	-	0.37	0.435	17.79	0.445	0.275	14.82	0.55	0.43	10.86	0.395	0.555	14.43	0.3	0.57	22.58	0.27	0.535	22.80

TT: Training Type, AT: Attack Type, Acc: Accuracy, SAR: Success Attack Rate, and DoC: Degree of Change on 200 random samples.

shows the performance of KAN-Mixer and MLP-Mixer models under similar training conditions and datasets using various attack types, including FGSM, BIM, PGD, and CW. We evaluated the models using three key metrics, Acc, SAR, and DoC, which reflect their robustness and vulnerability.

Standard training is the baseline, where no adversarial defenses are applied. In this setup, KAN-Mixer achieved 97% accuracy and MLP-Mixer 98.5% on the BTSRB dataset with clean data. However, both models suffered significant performance drops under adversarial attacks, especially at higher $ϵ$ values. For the FGSM attack at $ϵ=0.01$, KAN-Mixer shows better resistance than MLP-Mixer across most datasets but dropped to 0% accuracy on GTSRB when $ϵ=1$. Similarly, for the BIM and PGD attacks, both models experienced sharp accuracy declines, with MLP-Mixer nearing zero on BTSRB. The CW attack was less effective, indicated by lower SAR compared to FGSM, BIM, and PGD. While both models demonstrated some resistance, KAN-Mixer slightly outperformed MLP-Mixer overall.

Adversarial training incorporates adversarial examples during training, enhancing both models’ resilience to attacks compared to standard training, though the improvement varies by model and attack type. For the FGSM attack with $ϵ=0.01$, the MLP-Mixer achieved 83% accuracy and the KAN-Mixer achieved 89.5% on the BTSRB dataset. Although accuracy dropped as $ϵ$ increased, both models still outperformed their standard training counterparts. While adversarial training improved the defenses for the BIM and PGD attacks, the gain was modest; at $ϵ=1$, MLP-Mixer reached 15% accuracy on GTSRB and KAN-Mixer achieved 11%. Both models still showed high SAR values, indicating vulnerability to stronger attacks. For the CW attack, adversarial training was more effective, with both models exhibiting improved accuracy and lower SAR, especially the KAN-Mixer, which shows a slight advantage over the MLP-Mixer.

Randomized smoothing is an effective defense method that adds noise during inference, significantly improving robustness, especially for MLP-Mixer. Under the FGSM attack with $ϵ=0.01$, MLP-Mixer achieved 54.5% accuracy on BTSRB with a low Success Approximation Rate (SAR) of 0.44, outperforming KAN-Mixer. As $ϵ$ increase, MLP-Mixer maintained an advantage, reaching 98% accuracy at $ϵ=1$, compared to KAN-Mixer’s 72%. The SAR values for MLP-Mixer remained consistently lower, indicating its effectiveness against gradient-based attacks. MLP-Mixer also exceled under BIM and PGD attacks. For instance, it achieves 96% accuracy on BTSRB with BIM ($ϵ=1$), while KAN-Mixer obtained 94%. Despite high SAR values indicating some vulnerability to stronger attacks, randomized smoothing provided better accuracy than standard or adversarial training. Under the CW attack, MLP-Mixer showed enhanced performance, achieving 55% accuracy on BTSRB and 39.5% on GTSRB, with KAN-Mixer trailing slightly. Overall, randomized smoothing helped both models defend against this type of attack. When comparing KAN-Mixer and MLP-Mixer across three training methods, it is clear that the choice of model and training approach affected the robustness. Randomized smoothing was the most effective, particularly for MLP-Mixer, which generally achieved higher accuracy and lower SAR values. MLP-Mixer outperformed KAN-Mixer in many scenarios, especially against FGSM and CW attacks. While adversarial training improved both models compared to standard training, MLP-Mixer still demonstrated better resistance to stronger attacks like BIM and PGD. This comparison underscores the importance of both the model choice and training methods in designing robust machine learning systems.

The training performance outcomes of the KAN_Mixer and MLP_Mixer models are depicted in Figure 8, which showed loss during the training phase, respectively, over 100 epochs.

Figure 8 illustrates the loss values during the training of the KAN-Mixer and MLP-Mixer models across the BTSD, GTSRB, and CTSRD datasets using different training strategies: standard, adversarial training, and randomized smoothing. The x axis represents the training epochs, while the y axis shows the loss values. In standard training, both models showed a steady decrease in loss. The MLP-Mixer converged faster, particularly in the early epochs of the GTSRB and CTSRD datasets, while the KAN-Mixer achieved comparable stability after more epochs. Under adversarial training, the KAN-Mixer significantly reduced loss, particularly in the BTSD and CTSRD datasets, indicating better resilience to adversarial perturbations. The MLP-Mixer initially reduce loss rapidly but plateaued at a higher level. However, with randomized smoothing, the KAN-Mixer maintained consistently lower loss values than the MLP-Mixer, especially in the GTSRB dataset, suggesting an advantage from the smoothing technique. This superior performance of the KAN-Mixer with randomized smoothing is a key finding of our study.

5.3. KAN-Convolution and Convolution Layer Models

The

Table 3. Performance metrics for various attack and defense methods on the BTSRD dataset.

Training Type	AT	$\mathit{ϵ}$	KANC_KAN			ConvNet_MLP			ConvNet_KAN			KANC_MLP
			Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC	Acc	SAR	DoC
Standard	BIM	0.01	81.77	2.08	0.27	88.80	2.08	0.28	84.64	1.30	0.28	83.07	1.04	0.27
	BIM	0.1	73.44	11.46	2.47	71.09	19.79	2.51	73.18	12.76	2.51	72.92	11.72	2.48
	BIM	1	5.73	77.86	13.80	6.51	84.38	13.94	5.47	80.47	14.00	4.43	79.69	13.85
	FGSM	0.01	16.93	66.67	13.35	63.54	27.86	13.35	47.66	40.89	13.35	26.82	58.07	13.35
	FGSM	0.1	11.72	71.88	13.43	13.54	77.34	13.44	14.32	71.88	13.44	11.20	73.18	13.43
	FGSM	1	0	83.59	20.36	0	90.89	21.27	0	85.94	21.24	0	84.11	20.46
	PGD	0.01	15.63	67.97	13.35	62.24	29.17	13.35	45.31	42.97	13.35	24.48	60.42	13.35
	PGD	0.1	8.59	75.00	13.39	23.18	67.71	13.40	20.05	66.15	13.40	9.38	74.74	13.39
	PGD	1	1.82	81.77	16.59	1.30	89.58	16.63	1.30	84.64	16.68	2.34	82.29	16.59
	CW	-	17.71	65.89	11.23	67.19	23.70	4.63	46.61	39.32	7.20	27.60	56.51	10.04
	None	-	83.59	-	-	90.89	-	-	85.94	-	-	84.11	-	-
Randomized Smoothing	BIM	0.01	49.48	3.91	0.27	28.13	2.34	0.27	57.03	2.60	0.27	61.98	2.86	0.27
	BIM	0.1	25.26	28.39	2.45	16.93	13.54	2.47	39.84	20.83	2.46	38.54	26.56	2.47
	BIM	1	0	53.39	13.66	8.07	25.00	13.91	22.92	36.98	13.76	0.52	64.32	13.78
	FGSM	0.01	66.93	4.43	13.35	71.61	1.30	13.35	66.15	8.07	13.35	65.10	5.99	13.35
	FGSM	0.1	53.91	10.94	13.43	48.96	5.73	13.42	31.51	30.47	13.44	33.59	31.51	13.43
	FGSM	1	1.04	53.39	20.38	13.54	17.45	19.99	0	59.64	21.08	0.52	64.58	20.54
	PGD	0.01	59.64	6.77	13.35	65.89	2.34	13.35	60.68	9.64	13.35	61.98	7.03	13.35
	PGD	0.1	24.22	30.21	13.39	34.38	10.68	13.40	35.68	26.56	13.39	29.95	34.90	13.39
	PGD	1	0	53.39	16.49	1.30	29.69	16.58	13.80	46.35	16.47	0	64.84	16.52
	CW	-	49.22	4.17	4.72	29.17	1.30	4.23	53.39	6.25	4.37	60.94	3.91	4.68
	None	-	53.39	-	-	30.47	-	-	59.64	-	-	64.84	-	-
Adversarial Training	BIM	0.01	44.01	4.17	0.27	46.88	2.34	0.26	59.64	2.34	0.27	57.55	4.17	0.27
	BIM	0.1	12.76	35.42	2.44	32.81	16.41	2.44	46.88	15.36	2.45	40.63	21.35	2.47
	BIM	1	0	48.18	13.66	21.88	29.17	13.84	32.29	29.95	13.76	1.30	60.42	13.79
	FGSM	0.01	66.41	4.17	13.35	71.61	1.30	13.35	68.75	4.95	13.35	63.80	6.77	13.35
	FGSM	0.1	53.65	10.16	13.43	47.66	11.72	13.41	38.28	26.30	13.44	34.64	27.86	13.43
	FGSM	1	3.65	48.18	20.39	23.44	25.78	18.96	0.26	61.98	21.12	0	61.72	20.51
	PGD	0.01	58.59	7.29	13.35	69.53	1.30	13.35	65.36	6.25	13.35	61.46	8.33	13.35
	PGD	0.1	19.27	32.29	13.39	40.10	15.36	13.39	45.05	18.23	13.39	30.47	31.51	13.39
	PGD	1	0	48.18	16.47	7.29	43.75	16.58	15.10	47.14	16.47	0	61.72	16.53
	CW	-	44.53	3.65	4.83	48.70	0.52	3.96	57.81	4.17	4.40	56.77	4.95	4.36
	None	-	48.18	-	-	49.22	-	-	61.98	-	-	61.72	-	-

AT: Attack Type, Acc: Accuracy, SR: Success Attack Rate and DoC: Degree of Change on 200 random samples.

provides a detailed comparison of the performance metrics for various attack and defense methods on the BTSRD dataset—KANC-KAN, ConvNet-MLP, ConvNet-KAN, and KANC-MLP—under identical conditions.

During standard training, KANC-KAN and ConvNet-MLP achieved accuracies of 83.59% and 90.89%, respectively, but were highly vulnerable to adversarial attacks as $ϵ$ increased. For the BIM attack ($ϵ$ = 0.01), ConvNet-MLP led with 88.80% accuracy, followed by KANC-KAN at 81.77%. As $ϵ$ rose to 0.1, all accuracies dropped significantly, with KANC-KAN at 73.44% and ConvNet-MLP at 71.09%. The SAR values increased with higher $ϵ$, indicating greater susceptibility to attacks. Under the FGSM attack ($ϵ$ = 0.01), ConvNet-MLP score 63.54%, while KANC-KAN only reached 16.93%. At $ϵ$ = 0.1, all models struggled, with KANC-KAN at 11.72% and ConvNet-MLP at 13.54%. For the PGD attack ($ϵ$ = 0.01), ConvNet-MLP (62.24%) and ConvNet-KAN (45.31%) outperformed KANC-KAN (15.63%). As $ϵ$ increased to 0.1, all models’ accuracies plummeted. With the CW attack, KANC-KAN and KANC-MLP performed poorly (17.71% and 27.60%, respectively), while ConvNet-MLP achieved 67.19%. Despite some differences in performance, the SAR values remained high across all models, reflecting their vulnerability to optimization-based attacks.

Randomized smoothing significantly enhanced the adversarial robustness of the models, especially at lower $ϵ$ values. For the BIM with $ϵ$ = 0.01, KANC-MLP led with 61.98% accuracy, followed by ConvNet-KAN at 57.03% and KANC-KAN at 49.48%. The SAR values remained low, indicating the effective mitigation of adversarial perturbations. However, as $ϵ$ increased to 0.1, all models saw a notable drop in accuracy—KANC-KAN and KANC-MLP around 25%, as well as ConvNet-MLP at 16.93%. For the FGSM at $ϵ$ = 0.01, ConvNet-MLP topped with 71.61%, followed closely by KANC-KAN at 66.93%. The SAR values indicate effectiveness against lower-strength attacks, but the performance declined at $ϵ$ = 0.1, with KANC-KAN at 53.91% and ConvNet-MLP at 48.96%. Under PGD with $ϵ$ = 0.01, KANC-MLP and ConvNet-KAN achieved 61.98% and 60.68% accuracy, respectively, while KANC-KAN followed at 59.64%. As $ϵ$ rose to 0.1, all models lost accuracy, though KANC-KAN and KANC-MLP performed slightly better. For the CW attack, KANC-MLP achieve 60.94% accuracy, followed by ConvNet-KAN at 53.39% and KANC-KAN at 49.22%. ConvNet-MLP lagged at 29.17%, but low SAR values across all models suggest effective defense against optimization-based attacks.

Adversarial training enhances model robustness by incorporating adversarial examples into the training process, although its effectiveness varies by model and attack type. For the BIM with $ϵ$ = 0.01, ConvNet-KAN and KANC-MLP achieved the highest accuracies at 59.64% and 57.55%, respectively, while KANC-KAN lagged at 44.01%. With $ϵ$ increased to 0.1, the accuracy dropped across all models, with KANC-KAN at 12.76% and ConvNet-MLP at 32.81%. In the FGSM at $ϵ$ = 0.01, ConvNet-MLP led with 71.61% accuracy, followed by KANC-MLP at 63.80%. At $ϵ$ = 0.1, both models saw declines, with KANC-KAN at 53.65% and ConvNet-MLP at 47.66%. The SAR values remained low at $ϵ$ = 0.01 but increased with stronger attacks, indicating moderate protection. Under PGD at $ϵ$ = 0.01, ConvNet-MLP achieved 69.53%, while KANC-MLP reached 61.46%. With $ϵ$ = 0.1, the accuracies fell, with KANC-KAN at 19.27% and ConvNet-MLP at 40.10%, and the SAR values again increased. For the CW attack, KANC-MLP showed the highest accuracy at 56.77%, closely followed by ConvNet-KAN at 57.81%. KANC-KAN and ConvNet-MLP achieved 44.53% and 48.70%, respectively. Overall, the SAR values remained high, indicating that adversarial training is less effective against optimization-based attacks than randomized smoothing.

Our analysis underscores the effectiveness of randomized smoothing as the most potent defense for enhancing adversarial robustness, particularly for KANC-MLP and ConvNet-KAN models. While adversarial training does offer some benefits, it is less effective than randomized smoothing against high-intensity attacks like BIM and PGD. The KANC-KAN model, while consistent, is less effective under stronger attacks. These findings strongly suggest that by combining randomized smoothing with adversarial training, hybrid strategies may offer the most robust protection against adversarial attacks.

The training performance of the ConvNet_KAN, ConvNet_MLP, KANC_KAN, and KANC_MLP models is depicted in Figure 9, which shows loss during the training phase, respectively, over 100 epochs.

Figure 9 illustrates the loss values over training epochs. Initially, all models showed a sharp decrease in loss, especially in the early epochs, before stabilizing at lower values. ConvNet_KAN with randomized smoothing demonstrated the fastest loss reduction and achieves the lowest final loss, indicating superior optimization and convergence. The other models followed similar loss reduction trends but converged slightly slower, with KANC_KAN and ConvNet_MLP showing comparable final loss values.

5.4. Visualization Results

The visualizations presented in this section provide a comprehensive view of the latent space movements for various models—KAN, MLP, KAN-Mixer, MLP-Mixer, and hybrid models (KANConv_KAN, ConvNet_MLP, ConvNet_KAN, and KANConv_MLP)—when subjected to different adversarial attacks (FGSM, BIM, PGD, and CW) at varying $ϵ$ values. These t-SNE plots enable a direct comparison of model robustness across diverse training approaches, including standard, adversarial training, and randomized smoothing, providing insights into their behavior under attack. Due to space limitations, only a few representative figures are included in this paper to highlight key performance differences between the models. However, the full set of figures, showcasing all latent space movements under different attack scenarios and parameter settings, is available on (https://sie-lab-kr.github.io/Is-KAN-Robustness/, accessed on 6 September 2024). In these visualizations, green bounding boxes represent correct classifications, while red bounding boxes indicate misclassifications. Blue dashed lines illustrate the latent space trajectory of samples misclassified due to adversarial perturbations, while green dashed lines mark samples that remain correctly classified despite the attack.

For instance, under an FGSM attack with $ϵ=0.01$, the MLP-Mixer trained under standard conditions (Figure 10a) exhibited significant latent space shifts, resulting in numerous disrupted correct classifications. Conversely, the KAN-Mixer trained using standard settings (Figure 10b) showed less movement, reflecting better robustness even under the same attack conditions. Furthermore, when trained, the KAN-Mixer (Figure 10d) demonstrated even less movement in the latent space compared to the MLP-Mixer under adversarial training (Figure 10c), emphasizing its greater resilience to adversarial perturbations. Similarly, the KAN-Mixer model trained with randomized smoothing (Figure 10f) showed the most constrained latent space movement, underscoring its robustness, whereas the MLP-Mixer trained with randomized smoothing (Figure 10e) still exhibited more substantial latent space shifts. These visualizations emphasize the superior stability of KAN-Mixer models under adversarial conditions across different training strategies.

Additionally, visualizations comparing the KAN and MLP models under different settings (Figure 10) reveal distinct latent space movements. For example, adversarially trained KAN models exhibits more stability in the latent space compared to standard training (Figure 10h vs. Figure 10g). Similarly, the MLP models show more substantial latent shifts under standard training (Figure 10j vs. Figure 10k), and randomized smoothing results in more constrained movements for both KAN and MLP (Figure 10i,l), suggesting enhanced robustness against adversarial attacks.

6. Conclusions

The comparative analysis of KAN and MLP models across various datasets (BTSD, CTSD, and GTSRB) under different adversarial attack scenarios has yielded significant findings. Both models performed well on clean data, with MLP models, including MLP-Mixer, typically achieving slightly higher accuracy. However, the key discovery was their vulnerabilities under adversarial attacks, particularly at high epsilon values, where their robustness declines sharply. In contrast, the KAN-based model utilizes learnable activation functions on edges, employing nonlinear weights throughout. Each weight parameter was replaced by a univariate function, which was parametrized using spline-based interpolation and dynamic grid adaptation. This design aimed to provide superior flexibility in learning complex, nonlinear relationships, especially KAN-Mixer, demonstrating better resilience and consistently achieving higher accuracy and lower success attack rates (SARs) at moderate epsilon values. Our experiment conducted three training strategies: standard training, adversarial training, and randomized smoothing. Standard training led to substantial accuracy drops during adversarial attacks, while adversarial training offered limited improvements. However, randomized smoothing, which introduced noise during inference, proved most effective. It significantly enhanced the accuracy and robustness of MLP models like MLP-Mixer against gradient-based attacks. Finally, comparing KANConv-based models (KANConv_KAN and KANConv_MLP) with ConvNet-MLP models reveals that KANConv architectures offer a balanced approach. They generally outperformed ConvNet-MLP models in adversarial robustness. While ConvNet-MLP models performed well on clean data, they were more susceptible to adversarial perturbations, particularly at higher epsilon levels. KANConv-based models, on the other hand, maintained better accuracy and lower SAR across different attack scenarios, indicating that KANConv architectures offer a more balanced approach by combining high accuracy on clean data with solid defenses against adversarial attacks. This balance should reassure the audience about the overall performance of KANConv architectures. In conclusion, KAN-based models, particularly KAN-Mixer and KANConv architectures, show greater resilience to adversarial attacks than MLP models. Combining advanced defenses, such as randomized smoothing, with robust architectures like KAN-based models offers promising protection in challenging environments. Further research is needed to explore these models across additional datasets and metrics to understand KAN’s capabilities and limitations better.