Spiking PointCNN: An Efficient Converted Spiking Neural Network under a Flexible Framework

Abstract

Spiking neural networks (SNNs) are generating wide attention due to their brain-like simulation capabilities and low energy consumption. Converting artificial neural networks (ANNs) to SNNs provides great advantages, combining the high accuracy of ANNs with the robustness and energy efficiency of SNNs. Existing point clouds processing SNNs have two issues to be solved: first, they lack a specialized surrogate gradient function; second, they are not robust enough to process a real-world dataset. In this work, we present a high-accuracy converted SNN for 3D point cloud processing. Specifically, we first revise and redesign the Spiking X-Convolution module based on the X-transformation. To address the problem of non-differentiable activation function arising from the binary signal from spiking neurons, we propose an effective adjustable surrogate gradient function, which can fit various models well by tuning the parameters. Additionally, we introduce a versatile ANN-to-SNN conversion framework enabling modular transformations. Based on this framework and the spiking X-Convolution module, we design the Spiking PointCNN, a highly efficient converted SNN for processing 3D point clouds. We conduct experiments on the public 3D point cloud datasets ModelNet40 and ScanObjectNN, on which our proposed model achieves excellent accuracy. Code will be available on GitHub.

1. Introduction

Three-dimensional data, which provide an accurate description of shape and structure, are crucial in fields such as autonomous driving [1] and robotics [2]. Point clouds, as a representative form of these data, are composed of three-dimensional point data, making them highly valuable in modern technology development, like the research in [3,4]. As a fundamental and important branch, the progress in point cloud classification tasks has also made breakthroughs with the development of deep learning technology, such as the proposal of MAT-Net [5]. Despite many advancements, current point cloud processing methods still have some shortcomings. Due to the complexity of data and computation, deploying these methods on resource-limited mobile platforms, such as robots, is quite challenging [6].

Spiking neural networks (SNNs), representing the third generation of neural networks, have advantages over traditional artificial neural networks (ANNs) in some aspects. The working principle of SNNs is based on the transmission of pulses between neurons, and there is a lot of research in this area, such as [7]. When a neuron receives sufficient input signals, it generates a pulse and transmits it to the next layer of neurons. The information transfer in these neurons mimics the information transfer in biological neurons, i.e., via the precise timing of spikes or a sequence of spikes [8]. This event-driven approach allows SNNs to process sparse data more efficiently [9], with lower energy consumption [10].

Considering the advantages of both ANN and SNN, converting the former into the latter is an effective method to fully utilize the dual benefits and apply them to practical tasks, such as achieving high-precision and ultra-low latency SNNs in [11,12], which outperform state-of-the-art and directly trained SNNs in terms of accuracy and time step. Some recent studies have proposed parameter calibration techniques that align the activation patterns of SNNs with those of their ANN counterparts. For instance, Li et al. (2022) demonstrate that adjusting network parameters during conversion significantly reduces the mismatch between ANN and SNN activations, thus preserving accuracy [13]. Similarly, Deng et al. (2021) highlight the importance of optimal parameter tuning to minimize performance loss during conversion, ensuring that the resulting SNN maintains the functionality of the original ANN [14]. Additionally, Rueckauer et al. (2017) provide evidence that efficient event-driven networks can be derived from continuous-valued deep networks through careful conversion processes, enhancing computational efficiency without sacrificing performance [15]. The underlying mechanisms that enable these successful conversions are rooted in the temporal dynamics of spiking neurons, which allow SNNs to process time-dependent information more naturally, as discussed by Pfeiffer and Pfeil (2018) [16]. Moreover, Diehl et al. (2015) show that balancing weights and thresholds during the conversion process can lead to high-accuracy spiking networks that are well-suited for fast classification tasks [17]. These findings collectively underscore the viability of converting ANNs to SNNs, provided that the process is carefully managed to address the inherent differences between the two network types.

SNNs have widespread applications in many fields, especially in two-dimensional data processing, achieving excellent results in image classification tasks such as in [18,19]. In [20], the authors proposed a conversion framework. However, they did not specify the type of surrogate function employed in their approach. This omission suggests that they might have utilized either a Sigmoid or Softplus function, both of which are commonly used as surrogate gradients. But these are not inherently designed for spiking neural networks (SNNs). The implementation of a more specialized surrogate gradient function could potentially enhance the results. Similarly, in [21], the authors demonstrated excellent accuracy on a synthetic dataset. However, they did not extend their evaluation to real-world datasets. This limitation suggests that the model may not be practical enough to process real data that include noise or disturbances. In Section 4.3 of our study, we tested the model on the real-world dataset and observed that the results were less than satisfactory. To solve the problems, in this work, we have two key issues to deal with: (1) We should construct a specialized surrogate gradient function to ensure the spiking neurons trainable under backpropagation-base methods to ensure accuracy performance. (2) We should design a structure making the model robust enough to perform with excellent accuracy on real-world datasets.

We focus on efficiently converting artificial neural networks (ANNs), specifically convolutional neural networks (CNNs), to spiking neural networks (SNNs) for 3D point cloud classification. To address the prior problem, we apply the X-transformation operation that PointCNN [22] utilizes and a current-based LIF spiking model. We define a preprocessing spiking neuron to convert continuous floating-point tensors into discrete pulse signals. Based on this, we design two spiking neurons, one for extracting features into high dimensions and one for generating a feature matrix X. The neurons are fused into the traditional X-Conv operator, reconstructing the spiking X-Convolution module. Compared to the original X-Conv operator, the new X-Convolution unit exhibits better results under the spiking neural network structure. For the second issue, we propose an adjustable surrogate gradient function, enabling the converted network to be correctly trained within the backpropagation training framework. Our proposed surrogate function maximally fits the spiking signals during backpropagation while providing smooth construction to obtain continuous gradients. Hence, the model can be trained effectively under an ANN training framework. Finally, we present a flexible conversion framework that combines our specially designed spiking X-Convolution module with an adjustable surrogate gradient function. We thus raise Spiking PointCNN.

To summarize, in the work, we firstly introduce CNNs on point clouds (Section 2.1) and SNNs (Section 2.2) in the Related Works section. In the Methods part, we introduce the Spiking X-Convolution module (Section 3.1), the proposed adjustable surrogate gradient function (Section 3.2), and the all-purpose ANN-to-SNN converting framework (Section 3.3). In the fourth section, Experiments and Results, experimental settings (Section 4.1), ablation study (Section 4.2), and a comparison with SOTAs (Section 4.3) are demonstrated. Finally, the conclusion is discussed in the fifth part. The main contributions are as follows:

• We define two different spiking MLP neurons and construct the spiking X-Convolution module, which realizes the combination of spiking neurons and X-transformation for the first time. The module enables convolutional neural networks to perform effective local feature extraction and permutation-invariant classification on unordered point cloud data.
• We propose an adjustable surrogate gradient function, which makes discrete signals differentiable during backpropagation, enabling the spiking neural network to be accurately supervised during training.
• We build an efficient spiking neural network converted from a convolutional neuron network under a flexible conversion framework, which exhibits strong robustness and achieves high accuracy on two benchmark datasets, ModelNet40 [23] and ScanObjectNN [24].

2. Related Works

2.1. Convolution Neural Networks on Point Clouds

The convolutional neural network (CNN) is firstly proposed and defined as a model consisting of multiple layers of trainable filters (convolutions) and sub-sampling in [25]. The application of deep CNNs in large-scale image classification in AlexNet [26] highlighted the effectiveness of ReLU activation functions and Dropout regularization, leading to the concept of depth. Proposed in 2015, Residual Networks (ResNets) [27] introduce residual blocks to address the vanishing gradient problem in very deep networks, significantly improving training performance. CNNs now have been extensively applied in a variety of practical applications in grid data processing, demonstrating their versatility and robustness. For instance, in chronological age estimation [28], age and gender classification [29], and more complex tasks like contour detection [30], CNNs have shown exceptional performance. These applications underscore the adaptability of CNNs to a range of challenges in processing structured data.

Despite the success of CNNs in grid-like data structures, applying them to point clouds, which are irregular and unordered, presents unique challenges. Direct convolution between point-to-point features can result in loss of shape information and inconsistent outputs due to the varying order of input points. To address these challenges, pioneering models such as PointNet [31] and PointNet++ [32] were developed to directly process point clouds by learning global features through a shared MLP architecture. Building on this foundation, PointConv [33] redefines the convolution kernel for point clouds. It treats the kernel as a nonlinear function of local coordinates, using MLPs to learn a weight function and employing kernel density estimation to compute a density function. This method enhances the traditional CNN’s capability to handle the irregular format of point clouds, optimizing memory use and supporting deeper network architectures. Such advancements have enabled effective applications of CNNs to point cloud data in tasks like classification, segmentation, and object detection.

2.2. Spiking Neural Network

Spiking neural networks (SNNs) are a class of neural networks that aim to mimic the processing mechanisms of biological neurons more closely than traditional artificial neural networks. The foundation of SNNs lies in the spiking neuron models, which include classical models like Integrate-and-Fire (IF) applied in P2SResLNet [34], Leaky Integrate-and-Fire (LIF) utilized in Spiking PointNet [21], and Hodgkin–Huxley (HH) models. These models capture the dynamic behavior of biological neurons by considering the timing of spikes rather than the rate of firing. Recent innovations and explorations in this field have led to a variety of advanced models and approaches, as discussed in works such as [35].

Recent achievements of SNNs have demonstrated their potential in various applications, such as pattern recognition, sensory processing, and robotics. Novel SNN models have enhanced computational efficiency and performance by leveraging the temporal dimension of spike-based processing. For instance, Carpegna et al. developed an FPGA-optimized SNN accelerator, which significantly enhances both performance and energy efficiency, making it particularly suitable for applications that are energy-sensitive or require low latency [36]. However, this approach demands specialized FPGA expertise and might face limitations in scalability across different hardware configurations. Bybee et al. introduced Spiking Phasor Neural Networks (SPNNs), which leverage complex-valued DNNs and spike timing codes to achieve superior performance on complex datasets such as CIFAR-10 [37]. Despite their promising results, the complexity inherent in SPNNs, particularly in their use of complex-valued operations, could lead to increased computational costs and challenges in practical deployment. Kulkarni et al. provided a comprehensive review of various SNN algorithms and their applications, emphasizing the potential of SNNs in advancing cognitive computing [38]. However, as a review from 2017, it may not encompass the most recent developments in the fast-evolving field of SNN research. These studies collectively enhance our understanding of SNNs’ capabilities in pattern recognition, sensor data processing, and robotic control.

3. Methods

We propose an efficient converted spiking neural network under an all-purpose ANN-to-SNN converting framework with an adjustable surrogate gradient function, which are applied to generate Spiking PointCNN. We primarily combine X-transformation with leaky integrate-and-firing neurons to construct spiking X-Convolution module. Then, an effective surrogate gradient function that can be tuned to fit various models is presented to cope with uncomputable gradient. Next, our innovative, all-purpose ANN-to-SNN framework is demonstrated, which can be applied to various ANN models, especially suitable for CNNs. Finally, we combine the spiking X-Convolution module and the flexible converting framework with the proposed adjustable surrogate gradient function to construct Spiking PointCNN.

3.1. Spiking X-Convolution Module

Inspired by the combination of spiking neurons and typical ANN components, we introduce LIF neuron into X-transformation to construct spiking X-Convolution module functioning in 3D point clouds processing.

3.1.1. Leaky Integrate-and-Firing Neuron

Leaky integrate-and-firing (LIF) model is a simplified spiking neuron widely applied in spiking neural networks. It functions by simulating the accumulation and discharge of voltage in neurons to mimic the firing activity of neurons. The principle of an LIF neuron isgiven by

$${\displaystyle \frac{dV}{dt}}={\displaystyle \frac{-(V\left(t\right)-V_{rest})+R·i\left(t\right)}{{\tau }_m}}$$

(1)

$${\tau }_m=RC$$

(2)

$$V\left(t^{+}\right)=\left\{\begin{array}{cc}{V}_{reset}\hfill & \mathrm{if}V\left(t\right)\ge V_{thresh}\hfill \\ V\left(t\right)\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(3)

where $V\left(t\right)$ represents the membrane potential at time t and ${\tau }_m$ stands for the membrane time constant, which is the product of membrane resistance R and the membrane capacitance C. From the equations, it can be seen that the potential naturally decays towards the resting potential $V_{rest}$, simulating the condition when a neuron is not stimulated. When a stimulus signal external current $i\left(t\right)$ flows into the neuron, the potential accumulates and increases. If the membrane potential exceeds the voltage threshold $V_{thresh}$, it is considered that the neuron has emitted a spike signal and it is then reset to the lower potential $V_{reset}$.

3.1.2. X-Transformation Convolution [22]

Since point cloud data tend to be disorganized, the X transformation module is proposed to retain local correlation features. The key of the transformation is converting traditional convolution transform to suit point cloud data processing. First, applied with KNN algorithm, original point cloud data P are subjected to a local coordinate transformation to P’. Then, coordinated with P’, features F are sent to 2 different MLP operations, one for elevating coordinate dimension and another one for generate the feature transformation matrix, given by

$${\mathbf{F}}_{\delta }=ML{P}_{\delta }\left({\mathbf{P}}^{\prime }\right)$$

(4)

$$\mathbf{X}=ML{P}_{trans}\left({\mathbf{P}}^{\prime }\right)$$

(5)

which were proposed in [22]. $ML{P}_{\delta }$ in Equation (4), as is proposed in PointNet [31], is a multilayer perceptron that maps each local coordinate to a high-dimensional feature space in an attempt to capture the complexity of the local structure. $ML{P}_{trans}$ in Equation (5) is a combination of linear layers and nonlinear activation functions that transforms the input data output into a $K\times K$ matrix, where K is the local neighborhood size. The outcomes of two MLP operations are saved in vector ${\mathbf{F}}_{\delta }$ and P’. Then, ${\mathbf{F}}_{*}$, the concatenating of $\mathbf{F}$ and ${\mathbf{F}}_{\delta }$, is permuted with the learned $K\times K$ size X-transformation matrix $\mathbf{X}$, given by

$${\mathbf{F}}_X=\mathbf{X}\times {\mathbf{F}}_{*}=\mathbf{X}\times (\mathbf{F}+{\mathbf{F}}_{\delta })$$

(6)

which was proposed in [22]. Finally, a typical convolution between $\mathbf{K}$ and ${\mathbf{F}}_X$ is conducted to finish the module. Combined with previously mentioned formulas, the module can be summarized as

$${\mathbf{F}}_p=\mathrm{CONV}\left(\mathbf{K},ML{P}_{trans}(\mathbf{P}-p)\times [ML{P}_{\delta }(\mathbf{P}-p),\mathbf{F}]\right)$$

(7)

where all operations of X-transformation Convolution have been completed. The structure is shown in Figure 1.

3.1.3. Spiking X-Convolution Module

We introduce the LIF neuron into x-transformation convolution to define the spiking X-Convolution module. We design two fused spiking neurons based on the same signal-converting spiking neuron. Then, the spiking neurons are applied in the MLP operations to reconstruct the module. Utilizing specifically designed spiking neurons, the MLP operations are renamed as $ML{P}_{\delta }^{\prime }$ and $ML{P}_{trans}^{\prime }$ with the same function. The converting spiking neuron is defined as

$$\left\{\begin{array}{cc}\hfill I^{\prime }& =\mathrm{FN}\left(\mathbf{F}\right)\hfill \\ \hfill V^{\prime }\left(t\right)& =V_{rest}+R·I^{\prime }\left(t\right)+V^{\prime }\left(0\right)\times exp\left(-{\displaystyle \frac{t}{{\tau }_m}}\right)\hfill \\ \hfill s^{\prime }\left(t\right)& =V^{\prime }\left(t\right)\mid V_{thresh}\hfill \\ \hfill {\mathbf{S}}^{\prime }& =\{s_1^{\prime },s_2^{\prime },\dots ,s_{timestep}^{\prime }\}\hfill \end{array}\right.$$

(8)

where $I^{\prime }$ stands for the simulating current flow after $\mathbf{F}$ is processed by a fully connected layer and $V^{\prime }\left(t\right)$ simulates the accumulated membrane potential by time step t. Membrane potential is separated based on time step performing spike conversion. The resulting spike sequence is stored in ${\mathbf{S}}^{\prime }$. This is a simple spiking neuron that serves to convert the original continuous floating-point signal into preliminary spike signals, acting as a transition for subsequent combinations in other ways. The prior neuron is applied in the spiking neuron functioning in $ML{P}_{trans}^{\prime }$, which is defined as

$$\left\{\begin{array}{cc}\hfill I& =\mathrm{Conv}1\mathrm{d}\left(\mathrm{RS}\left(\mathrm{BN}\left({\mathbf{S}}^{\prime }\right)\right)\right)\hfill \\ \hfill \mathbf{S}& =\{s_1,s_2,\dots ,s_{timestep}\}\hfill \end{array}\right.$$

(9)

where the membrane potential accumulation process is omitted for repeating and I represents the simulated current stimulus obtained from the previous spike signals through batch normalization, reshape, and convolution operations in order. This neuron is the key component in $ML{P}_{trans}^{\prime }$ to reassemble and weighted fusion features. The parallel spiking neuron for $ML{P}_{\delta }$ is defined as

$$\left\{\begin{array}{cc}\hfill I& =\mathrm{FN}\left(\mathrm{BN}\left({\mathbf{S}}^{\prime }\right)\right)\hfill \\ \hfill \mathbf{S}& =\{s_1,s_2,\dots ,s_{timestep}\}\hfill \end{array}\right.$$

(10)

where the membrane potential accumulation process is ignored as well. In the Equation (10), I stands for the current signal after being processed by a batch normalization operation and a fully connected layer. This neuron is used to extract features into high-dimensional space, cooperating with other components in $ML{P}^{\prime }$.

After combining with the spiking neurons, the structure of the proposed spiking X-Convolution module is shown in Figure 2. The usage of spiking neurons enables the processing to filter more noise and improve the temporal efficiency of operations. To assign our proposed module high adaptability in model building, the input and output of it is designed to be continuous by applying other ANN layers.

3.2. Effective Adjustable Surrogate Gradient Function

Supervised methods in ANNs are based on the chain rule and are always efficient. Converted SNNs also apply the method to retain high accuracy. For a neuron $\mathit{j}$ in iteration $\mathit{n}$, the principle part in updating the parameters is computing the correction in weight based on local gradient, which are derived from the backpropagation algorithm described in Simon Haykin’s book [39], given by

$${\delta }_j\left(n\right)={\displaystyle \frac{\partial E\left(n\right)}{\partial v_j\left(n\right)}}={\displaystyle \frac{\partial E\left(n\right)}{\partial e_j\left(n\right)}}{\displaystyle \frac{\partial e_j\left(n\right)}{\partial y_j\left(n\right)}}{\displaystyle \frac{\partial y_j\left(n\right)}{\partial v_j\left(n\right)}}=e_j\left(n\right){\phi }_j^{\prime }\left(n\right)$$

(11)

$$\Delta w_{ji}\left(n\right)=-\eta {\delta }_j\left(n\right)y_i\left(n\right)$$

(12)

where ${\delta }_j\left(n\right)$ is the local gradient, $E\left(n\right)$ is the total error, $v_j\left(n\right)$ is the induced local field produced at the input, $e_j\left(n\right)$ is the difference between actual output and the target, ${\phi }_j^{\prime }\left(n\right)$ is the derivative of the activation function, $\Delta w_{ji}\left(n\right)$ represents the correction in weight, $\eta$ is the learning rate, and $y_i\left(n\right)$ is the output of neuron.

For spiking neurons, they emit a discrete binary pulse and the discontinuous signals are not derivable. As a result, the derivative ${\phi }_j^{\prime }\left(n\right)$ in Equation (11) is zero or infinity, causing the computed change in weight to be zero or infinity as well. In this case, by directly calculating gradient, the supervised method is not working.

To address the problem, utilizing surrogate gradient functions is an effective method. Applying a differentiable surrogate gradient function enables spiking neurons to operate during backpropagation. We propose and apply an adjustable and all-use surrogate gradient function, given by

$$\phi \left(x\right)={\displaystyle \frac{1}{2ln2}}(tanh\left(k(x-V_{thres})\right)+1)·ln(1+exp(-c(x-d)))+B$$

(13)

where $\mathit{k}$ affects the mutation rate of the function near the point and V_thres represents the activation threshold of the pulse. $\mathit{c}$, $\mathit{d}$, and $\mathit{B}$ are other constants regulating the shape and monitoring the usage.

When $\mathit{c}$ equals 0, $\phi \left(x\right)$ is $\mathit{tanh}$-like and possesses the characteristics of $\mathit{sigmoid}$. The gradient can be controlled by different $\mathit{k}$. The $\phi \left(x\right)$ with different $\mathit{k}$ and their gradient functions along with that of $\mathit{sigmoid}$ are shown in Figure 3. It can be seen that when $\mathit{k}$ increases, a more accurate gradient function fitting spiking values is obtained in the backpropagation process. However, if $\mathit{k}$ is set to a large value, the gradient is sharp in a narrow range, leading to the risk of the gradient exploding or vanishing again. The reason is that the final weight gradient is calculated by multiplying gradients through time steps and layers, making it either too big or very small. In contrast, when $\mathit{k}$ is set low, a more inaccurate gradient is calculated, resulting in a severe error final gradient for accumulated discrepancy through time steps and layers. The condition is particularly pronounced in a deep network. Thus, the performance is adversely affected.

Apart from being applied as surrogate functions in spiking neural networks, our proposed function is flexible enough to act as a normal activation function in various models. By tuning the constants, it is able to simulate or even replace activation functions like $\mathit{ReLU}$, $\mathit{ELU}$, and $\mathit{SoftPlus}$, shown in Figure 4.

3.3. The All-Purpose ANN-to-SNN Converting Framework

The spiking neural network structure differs from artificial neuron networks in diverse aspects, resulting in various challenges in converting ANNs to SNNs. Solving the obstacles, we summarize and design an all-use ANN-to-SNN converting framework. The schema is based on a modular design, enhancing generality and flexibility. In this section, we will introduce the framework from different aspects.

3.3.1. Time Steps

Spiking neural networks are composed of spiking neurons that imitate brain activities. In the aforementioned section, information transmission between spiking neurons relies on time step-based spiking signals. However, time steps have not been incorporated into the design of the ANN. To tackle the first barrier in conversion operation, we introduce time steps into original artificial layers to formulate new spiking blocks.

Convolution, batch normalization, and full connection operations are well-designed, robust, and efficient. Thus, our conversion retains the features and revises them by combining time steps without disrupting these efficient transformations. To obtain $\mathit{n}$ time steps discrete signals after continuous numerical transformations, we choose to repeat the tensor $\mathit{n}$ times and add a time dimension. Then, they are packaged as a minimum unit and used for next-step transmission. Before reaching a spiking neuron, the tensor will be unpacked and each sub-tensor is individually processed in arriving artificial layers. When encountering a spiking neuron, the value will be sent into it $\mathit{n}$ times to simulate membrane potential accumulation. In this case, the features and advantages of ANNs are retained while spiking operations are able to be conducted under the framework too.

3.3.2. Spiking Neuron Model

Spiking neurons can be seen as the key transformation in an ANN-to-SNN conversion. In the framework, the leaky integrate-and-firing neuron, as previously mentioned, is applied for its precision and efficiency. The LIF model utilized in the schema works as the activation function in hidden layers, turning a floating-point number into binary spiking pulse. Therefore, during forward propagation, the tensor of a traditional artificial neuron network is converted into spiking signals, altering the ANN to an SNN. Consequently, the proposed framework is able to fuse the spiking neuron into the network, successfully transforming the source model into a spiking neural network. The process is shown in Figure 5.

3.3.3. Training Methodology

Another significant variance between ANNs and SNNs is the training strategy. For SNNs, the most utilized methods are unsupervised learning algorithms like STDP and the Hebbian learning rule. Nevertheless, ANN training is highly reliant on gradient computation like gradient descent and batch gradient descent, which can be regarded as a key factor for their excellent precision.

Under our proposed framework, to sustain the high accuracy, we apply stochastic gradient descent method to train the converted SNN. However, as mentioned earlier, discontinuous spiking signals are not derivable, leading to failure in computing gradient. So, we utilize the surrogate gradient strategy, as previously discussed, using our proposed function in Equation (13). Further, the parameters are supposed to be set at $\mathit{c}=\mathit{d}=\mathit{B}=$ 0 and V_thres represents the minimum activation threshold for spikes. The function works effectively by providing continuous gradient and maximum degree of alignment with pulse signals.

Finally, we combine the spiking X-Convolution module with the effective converting framework containing special gradient function to create our innovative network, Spiking PointCNN. The model is composed of 4 spiking X-Convolution modules, which also utilize the proposed converting schema, and spiking full-connection layers. The structure can be seen in Figure 6.

4. Experiments and Results

4.1. Experimental Settings

To demonstrate the efficiency of the proposed network structure, experiments are conducted on two public 3D point cloud classification datasets: ModelNet40 and ScanObjectNN. ModelNet40 is a synthetic dataset consisting of 40 categories of 3D CAD models for common items. Generated by Princeton University, it provides distinctive features of objects with little distribution occurring. The training part of the dataset comprises 9843 samples, while the testing part comprises 2468 samples. ScanObjectNN is constructed by real-world scanning data, which contain ample noise and interference. The dataset comprises about 15,000 objects, which are divided into 15 different categories with 11,416 instances in the training set and 2882 instances in the testing set. For evaluation indicators, overall accuracy (OA) on entire dataset classification and the mean accuracy (mACC) on different categories are utilized.

Since our proposed structure is converted from an artificial neural network, it contains a large amount of mathematics computing, including the convolution operation. Thus, training on a traditional PC performs better than on neuromorphic hardware. Our experiments are conducted on a PC with two Intel Xeon Silver 4210R 2.40 GHz 10-core processors and an NVIDIA GeForce RTX 3080Ti GPU. Our python 3.8 interpreter is based on PyTorch and PyTorch Geometric. For our replacing spiking layers, overall time latency T is set to 1 by default.

4.2. Ablation Study

Entire ablation experiments are conducted on ModelNet40 first to determine the best structure of Spiking PointCNN and examine the effectiveness of different modules in the model.

4.2.1. Ablation on Components in Spiking X-Convolution Module

Figure 6 presents the structure of our proposed spiking X-Convolution module with a specifically designed spiking $ML{P}_{\delta }$ neuron and spiking $ML{P}_{trans}$ neuron. We designed four different types of X-transformation modules with or without the spiking neurons. Experiments are conducted in the converted SNN framework to demonstrate the effectiveness of the spiking neurons designed module under the proposed framework.

Table 1. Ablation study on spiking MLP components of spiking X-Convolution module.

Components	OA (%)	mACC (%)
$ML{P}_{\delta },ML{P}_{trans}$	84.68	79.98
$ML{P}_{\delta }^{\prime },ML{P}_{trans}$	84.64	80.51
$ML{P}_{\delta },ML{P}_{trans}^{\prime }$	82.46	79.46
$ML{P}_{\delta }^{\prime },ML{P}_{trans}^{\prime }$	85.94	84.48

reports evaluation results on the ModelNet40 testing dataset. It can be clearly made out that, under the framework of converted spiking neural networks, our $MLP$s with appropriate spiking neurons achieve the highest performance. Compared with the structure without any spiking neurons, our proposed module has a 1.26% improvement in OA and 4.50% improvement in mACC. However, when only one spiking neuron is applied to the module, their performance can be even lower than the original one. We supposed that that is caused by the mismatch of two $MLP$ operations. In this case, we choose to apply both the spiking $ML{P}_{\delta }$ neuron and spiking $ML{P}_{trans}$ neuron to construct the spiking X-Convolution module. The spiking X-Convolution module is for building our final classifier in other ablation experiments.

4.2.2. Ablation on Surrogate Gradient Function

Since our proposed surrogate gradient function is adjustable, to fit the usage of simulating spiking signals, the parameters are set to be $c=d=B=0$ and $V_{thres}=0.5$. However, as mentioned earlier, k affects the simulation degree and the smoothing of the gradient. In this case, k cannot be too large or too small. We test different k values in the function on ModelNet40 and record the maximum overall accuracy and mean accuracy in

Table 2. Ablation study on different k in proposed surrogate gradient function on the ModelNet40 dataset.

K	0.5	1	5	10	15
OA (%)	84.24	86.71	84.24	85.94	65.80
mACC (%)	83.31	82.21	79.50	84.48	58.71

. When $k=1$, the network performs best in OA; when $k=10$, the accuracy is worst, which may be caused by large k leading to dramatic changes in the function, resulting in excessively large gradients during backpropagation. In this case, the ablation experiments set $k=1$ as default.

4.2.3. Ablation on Time Steps

Time step T is a key parameter in SNNs, and our evaluation results on the time steps area are presented in

Table 3. Ablation study on different time steps on the ModelNet40 dataset. Spiking PointCNN with $T=3$ presents the highest OA while $T=2$ processes the best mACC.

	T = 1	T = 2	T = 3	T = 4
OA (%)	84.28	85.78	85.94	84.97
mACC (%)	81.85	84.48	83.14	80.71

. We set T to 1, 2, 3, and 4 to conduct the ablation study. We observe that our network achieves the best OA when $T=3$ and the highest mACC when $T=2$. Though OA in $T=3$ performs better than that in $T=2$, we can infer that $T=2$ has higher average OA from the trends in the evolution of best mACC, indicating that the emergence of best OA in $T=3$ may be with some randomness. Generally, the result is consistent with experiments on T in Spiking PointNet. We consider that when $T=1$, the features of input pictures are lost because, in this case, the spiking neurons act as binary machines—that is, they output 1 when the value exceeds the threshold and 0 otherwise. This results in the loss of temporal and spatial sequence information. We also conjecture that when T is too large, the information is redundantly expressed, creating noise beyond the features. For instance, spike sequences may repeat, adversely affecting the learning and fitting of true features. For the experiments, setting a larger T is not practical due to limitations in experimental physics conditions. Naturally, we can infer that increasing T beyond a certain point will not result in better performance. Hence, our other ablation studies are conducted by setting $T=2$.

4.3. Comparison with SOTAs

Table 4. Comparison of various ANN-to-SNN models on ModelNet40 and ScanObjectNN.

Model	ModelNet40	ScanObjectNN
	OA/mAcc (%)	OA/mAcc (%)
KPConv-SNN	70.5/67.6	43.9/38.7
PointNet_Baseline	84.2/-	62.6/-
PointNet++_Baseline	78.3/-	58.0/-
PointNet_Efficient	88.2/-	66.6/-
PointNet++_Efficient	89.5/-	69.2/-
Spiking PointNet	88.6/86.7	66.4/60.4
Spiking PointCNN	86.7/82.2	74.1/73.3

presents the comparison results from various ANN-to-SNN converted models on two benchmark datasets. All the networks convert an ANN to an SNN. KPConv-SNN is an ANN-to-SNN network following [40] to convert an ANN-based method into SNN form in [34]. PointNet_Baseline, PointNet++_Baselin, PointNet_Efficient, and PointNet++_Efficient are converted models compared or proposed in [20], while the efficient ones utilize their proposed converting framework. However, since the code is not open-source, we only refer their overall accuracy on two datasets without mean accuracy. Spiking PointNet [21] is converted by a new framework and achieves good performance. We conduct its official code and simply change the dataset, obtaining the accuracy tested on ScanObjectNN dataset for comparison.

From the table, it can be found that our proposed model performs well on ModelNet40, where the overall accuracy and mean accuracy are 85.94% and 83.14%. Compared with other networks, although it has less accuracy within 3 percent on a synthetic dataset whose data are without any disturbance, the network achieves high accuracy on the real-world dataset ScanObjectNN. It reaches an OA of 76.10% and a mACC of 73.30%, which are higher than those of the other models. The OA on ScanObjectNN is 4.83% higher than PointNet++_Efficient.

The superior performance of Spiking PointNet on the synthetic ModelNet40 dataset is acknowledged; however, our proposed model demonstrates significantly better performance on the ScanObjectNN dataset, which contains real-world disturbances. This suggests that our model is more robust and better suited for practical applications, where background interference is often unavoidable. The ability to maintain high accuracy in such noisy environments underscores the practical applicability and resilience of our approach compared to models optimized on clean datasets.

5. Conclusions

In this paper, we have presented Spiking PointCNN, a converted spiking neural network based on spiking X-Convolution modules. The work was motivated by the enormous development potential of SNN and the demand for and gap in the field of point cloud classification with SNNs. We fused the spiking neurons to construct the spiking X-Convolution module, extracting additional feature dimensions while preserving the sequential relationship between points. Then, we proposed an adjustable surrogate gradient function addressing the non-differentiability issue of pulse activation functions. This function can also be replaced by tuning parameters to substitute for other different activation functions. Subsequently, we proposed a universal framework for converting ANN to SNN, combining our Spiking X-Convolution and surrogate gradient function to obtain our Spiking PointCNN model. The ablation study and experiments on the final model were conducted on the ModelNet40 and ScanObjectNN datasets. Our network achieved excellent performance, especially surpassing other converted SNNs by a large margin on the real dataset ScanObjectNN. This indicates that our model not only underwent efficient conversion but also holds practical utility.