TSPconv-Net: Transformer and Sparse Convolution for 3D Instance Segmentation in Point Clouds

Abstract

Current deep learning approaches for indoor 3D instance segmentation often rely on multilayer perceptrons (MLPs) for feature extraction. However, MLPs struggle to effectively capture the complex spatial relationships inherent in 3D scene data. To address this issue, we propose a novel and efficient framework for 3D instance segmentation called TSPconv-Net. In contrast to existing methods that primarily depend on MLPs for feature extraction, our framework integrates a more robust feature extraction model comprising the offset-attention (OA) mechanism and submanifold sparse convolution (SSC). The proposed framework is an end-to-end network architecture. TSPconv-Net consists of a backbone network followed by a bounding box module. Specifically, the backbone network utilizes the OA mechanism to extract global features and employs SSC for local feature extraction. The bounding box module then conducts instance segmentation based on the extracted features. Experimental results demonstrate that our approach outperforms existing work on the S3DIS dataset while maintaining computational efficiency. TSPconv-Net achieves 68.6% mPrec, 52.5% mRec, and 60.1% mAP on the test set, surpassing 3D-BoNet by 3.0% mPrec, 5.4% mRec, and 2.6% mAP. Furthermore, it demonstrates high efficiency, completing computations in just 326 s.

1. Introduction

Understanding 3D scenes is a fundamental necessity for various applications, including autonomous driving [1], robotics technology [2], medical image processing [3], and 3D reconstruction [4]. By enabling computers to achieve a deeper understanding of 3D scenes, 3D instance segmentation drives the advancement and application of intelligent systems across diverse domains. However, repetitive structures in indoor scenes (e.g., tables, chairs, and beds) present significant challenges for 3D instance segmentation. The similarity in geometric shapes and feature distributions of these objects often leads to instance confusion and crowding in the feature space, increasing the likelihood of missed detection and false positives. Moreover, the close arrangement of objects in indoor scenes, coupled with complex occlusion problems, make it difficult to accurately segment complete instances, particularly when edges are blurred, or objects are overlapped. The geometric and topological finiteness of repetitive structures further exacerbates segmentation difficulties, limiting the model’s ability to segment objects of similar shapes. Additionally, inconsistencies in data annotation introduce noise into the training data, potentially leading to deviations in the learning process and affecting generalization performance. When encountering unseen repetitive structures in the test set or a new environment, the model often exhibits poor adaptability.

In current research on 3D instance segmentation, deep learning methods are predominant, typically involving the training of neural networks for scene segmentation. Among these methods, backbone networks such as MLPs are commonly used for feature extraction. However, MLPs, while simple and easy to implement, often struggle to capture spatial information effectively during feature extraction and incur high computational complexity. As a result, networks relying solely on MLPs may not perform satisfactorily in the segmentation of complex scenes.

Recent researchers have proposed various methods to overcome the limitations of traditional multilayer perceptron (MLP) architectures for 3D instance segmentation. This has led to a growing interest in innovative approaches. Rozenberszki et al. [5] introduced a novel unsupervised method called UnScene3D. UnScene3D extracts color and geometric features from RGB-D data to generate pseudo-masks without relying on labeled data. They employed self-training to refine these masks, resulting in improved segmentation performance. Traditional bottom-up strategies often involve post-processing steps, which can be computationally expensive and require domain-specific assumptions. Kolodiazhnyi et al. proposed a top-down approach called TD3D [6], which is entirely data-driven. TD3D directly predicts instance proposals, filters them using non-maximum suppression, and performs mask segmentation for each proposal. The versatility of deep learning models for segmentation tasks has been a subject of study. While models can excel in specific tasks, they often struggle to generalize to others due to task-specific differences. Kolodiazhnyi et al. [7] proposed OneFormer3D, a unified framework for multitasking 3D segmentation, addressing this challenge through novel query selection and matching strategies. The Transformer architecture, which has shown remarkable success in 2D image processing, has been extended to 3D point clouds. Mask3D [8], proposed as a pioneering Transformer-based approach, directly predicts instance masks from 3D point clouds, eliminating the need for post-processing steps and improving computational efficiency.

A typical method for 3D instance segmentation, known as 3D-BoNet [9], adopts an anchor-free methodology and employs PointNet++ [10] as its backbone network. Leveraging both local and global features, 3D-BoNet directly predicts bounding boxes via the bounding box regression branch and establishing associations between ground truth and predictions using correlation layers. Furthermore, 3D-BoNet improves segmentation performance by predicting instance labels for points using the mask prediction branch. However, 3D-BoNet fails to capture complex features, neglecting the spatial relationships between points.

To address the limitation of 3D-BoNet, we propose a novel 3D instance segmentation framework tailored for indoor scenes. Building upon [9], our framework enhances the backbone network by replacing PointNet++ with the offset-attention mechanism [11] and submanifold sparse convolution [12].

By doing so, the network becomes adept at capturing intricate spatial relationships among points, thus improving its ability to perceive global information. Moreover, TSPconv-Net excels at extracting finer local geometric structures within complex scenes. Notably, our framework eliminates the need for post-processing steps such as clustering, non-maximum suppression [13], or voting. As a result, TSPconv-Net exhibits outstanding efficiency in both training and inference phases.

We conduct experiments on the S3DIS dataset [14], and experimental results indicate that our method can achieve outstanding segmentation performance for repetitive structural objects within complex scenes.

In summary, the contributions of our work are as follows:

• We propose TSPconv-Net, a novel framework for 3D instance segmentation that leverages the OA mechanism and SSC to extract features in point clouds.
• We optimize and improve the backbone network of 3D-BoNet, enhancing the ability of TSPconv-Net to capture spatial relationships within point clouds.
• TSPconv-Net achieves outstanding performance on S3DIS without comprehensive modifications of the model architecture or hyperparameter tuning tailored for each dataset.

2. Related Work

Three-dimensional instance segmentation uses a semantic category label and a unique instance label to annotate each point in a 3D point cloud. Three-dimensional instance segmentation differs from 3D semantic segmentation; it not only requires the consideration of the semantic information of points but also necessitates differentiation between different instances under the same semantic information. Driven by applications in medical image processing, 3D reconstruction and 3D instance segmentation have gained significant interest [15]. Recent approaches can be broadly categorized into proposal-based and proposal-free methods.

proposal-based method. These methods adopt a two-stage approach to address the task: 3D object detection and instance label prediction. This strategy provides benefits such as potentially higher accuracy by initially concentrating on promising regions likely to contain objects. During the first stage, 3D object detection identifies potential objects in the scene. Subsequently, for each identified region of interest (ROI), the method predicts a specific label for each point within that region, effectively segmenting the objects. Hou et al. [16] proposed a method called 3D-SIS for RGB-D-based 3D semantic instance segmentation. This network utilizes a fully convolutional network to extract 2D color features and 3D geometric features, which are subsequently fused for 3D instance segmentation. Yi et al. [17] proposed a Generative Shape Proposal Network (GSPN) based on PointNet [18]. This network utilizes a generative proposal method to generate candidate point sets similar to the shape of the target points and encodes features to construct global and local structural information. Consequently, GSPN can effectively learn the shape features of the target, thereby improving segmentation accuracy. Yang et al. [9] proposed a single-stage, anchor-free and end-to-end trainable network called 3D-BoNet. The network takes a simpler approach, using point-wise MLP to directly predict 3D bounding boxes for each object and point-wise masks to refine the segmentation. Three-dimensional-BoNet also incorporates an optimal assignment method to handle data association effectively, balancing accuracy and speed compared to traditional methods.

In summary, proposal-based methods, exemplified by [19,20,21,22], demonstrate proficiency in handling objects with intricate shapes, achieving precise shapes and poses for each instance. Nonetheless, these methods often adopt multi-stage structures, leading to increased computational overhead and vulnerability to the choice of candidate region generation methods.

proposal-free method. These methods typically regard instance segmentation as a post-processing step of semantic segmentation. These methods operate under the assumption that points belonging to the same instance share similar features, thus emphasizing feature learning. Wang et al. [23] first proposed the deep learning model SGPN, which is founded on similarity measurement and a grouping strategy. SGPN uses a deep similarity matrix to measure the similarity between each pair of points in the point cloud. Additionally, it employs a novel grouping strategy to assign points in the point cloud to their respective instances. Similarly, Liu et al. [24] proposed the instance segmentation network MASC, which is based on SSC and point similarity. This network utilizes a multi-scale point affinity prediction method to measure the similarity between each pair of points in the point cloud. Furthermore, it reduces computational costs using sparse convolution. Given that semantic labels can be utilized for instance label prediction, some methods integrate these two tasks into a unified approach. Wang et al. [25] first proposed the ASIS framework, designed to simultaneously handle both instance segmentation and semantic segmentation tasks. ASIS employs two collaborative approaches: semantic-based instance segmentation and instance-based fusion semantic segmentation, which work in conjunction with each other. Similarly, Zhao et al. proposed JSNet [26] for accomplishing both semantic segmentation and instance segmentation.

Overall, proposal-free methods, such as [27,28,29,30,31,32], directly extract and classify point clouds, predicting instance labels, thereby offering lower computational costs and faster speeds. However, proposal-free methods encounter difficulties in handling objects with complex shapes and may struggle to achieve precise 3D shapes and poses for each instance.

3. Our Method

3.1. Overview

Our method consists of two primary components: a 3D backbone and a bounding box module. The backbone network takes a 3D point cloud $P\in {\mathbb{R}}^{N\times 3}$ as input to extract point cloud features $f\in {\mathbb{R}}^K$. We adopt the OA mechanism and SSC as our backbone. The bounding box module consists of parallel network branches dedicated to bounding box regression and instance mask prediction. The box regression branch takes local features $F_l\in {\mathbb{R}}^{N\times K}$ from the backbone and outputs bounding boxes $B\in {\mathbb{R}}^{H\times 2\times 3}$, and the instance mask prediction branch takes global features $F_g\in {\mathbb{R}}^{1\times K}$ and $F_l$ as inputs and transforms them into binary masks. The bounding box regression branch uses the association index A to associate ground truth bounding boxes with predicted bounding boxes and computes the cost matrix C. Additionally, we use the cost matrix to define the multi-criteria loss function. The overview of our method is illustrated in Figure 1.

3.2. Backbone Network

Compared to the MLP structure, attention mechanisms can capture not only distance information between points but also complex spatial information, enabling the network to better capture global dependencies and enhance focus on important features. However, attention mechanisms lead to increased network complexity and computational demands. Therefore, we introduce sparse submanifold convolution to reduce a large number of redundant computations, improving network training and inference speed. As a result, our backbone network can achieve excellent feature extraction efficiency.

3.2.1. Offset-Attention-Based Global Feature Extraction

Inspired by Guo et al. [11], we embed the input points P into a new feature space ${\mathbb{R}}^E$, obtaining embedded features $F_e\in {\mathbb{R}}^{N\times E}$. The structure of the global feature module is shown in Figure 2. To enable the network to learn richer and more comprehensive feature representations, we stack multiple attention modules in TSPconv-Net. This progressive stacking allows each layer to abstract and extract higher-level features from the output of the previous layer, thereby enhancing the network’s learning capacity and robustness. Additionally, the concatenation operation fuses features from different levels, preserving valuable information. The parameters of each layer are learnable, allowing the model to learn the optimal combination of features at different levels. This enables the model to capture more complex and nuanced feature representations. Moreover, fusing feature representations from different levels enhances the model’s robustness to noise and occlusion. However, directly stacking them can increase the complexity of the network, leading to gradient exploding or gradient vanishing. Therefore, we introduce residual modules to avoid such issues and improve the performance of TSPconv-Net. Then, we input $F_e$ into the stacked attention modules and obtain local features $F_l$ through the LBR layer. The LBR layer, composed of a linear layer, batch normalization layer, and ReLU activation function, enhances the expressive power and accelerates the convergence speed of TSPconv-Net. Finally, the local features $F_l$ are transformed into global features $F_g$ through a pooling layer. The specific calculation formula for attention features is shown in Equation (1).

$$\left\{\begin{array}{cc}\hfill F_1& =A{T}^1\left(F_e\right)\hfill \\ \hfill F_i& =A{T}^i\left(F_{i-1}\right),i=2,3,4\hfill \\ \hfill F_l& =concat(F_1,\dots ,F_i)·W_o\hfill \\ \hfill F_g& =Pool\left(LBR\left(F_l\right)\right),\hfill \end{array}\right.$$

(1)

where $A{T}^i$ represents the i-th attention layer, each layer has the same dimension of output and input, and $W_o$ represents the weights calculated for the final linear layer.

The embedding feature module consists of two cascaded LBRs. Through this combination, the embedding feature module efficiently extracts data features and enhances the fitting ability of our network to nonlinear data. The specific structure is depicted in Figure 3.

After obtaining embedded features $F_e$, further feature extraction is performed using attention mechanisms. However, the original attention mechanism used in the Transformer [33] only aggregates adjacent information near nodes, resulting in corresponding adjacency matrices. Yet, the inherent information of point cloud data is equally crucial. Inspired by GCN [34], we replace the original adjacency matrices in the attention mechanism with Laplacian matrices, resulting in an improved attention mechanism known as the offset-attention mechanism. The specific structure of the offset-attention module is shown in Figure 4. Notably, solid lines indicate the self-attention computation process, while dotted lines indicate the offset-attention computation process. Adding the nonlinear transformation (LBR) of the offsets to the offsets themselves can enhance the ability to learn local features and preserve the original information to form a residual structure. This improves the ability of the model to capture complex geometric structures and represent features.

The specific processing of OA is shown as follows: Firstly, the output vector $F_e$ of the embedding feature module is used as the input eigenvector of the E dimension, and the query (Q), keyword (K), and value (V) matrices are obtained by linear transformation through the linear layer. The corresponding calculation formula is shown in Equation (2).

$$\left\{\begin{array}{c}\hfill (Q,K,V)=F_e·(W_q,W_k,W_v)\\ \hfill Q,K\in {\mathbb{R}}^{N\times d_a},V\in {\mathbb{R}}^{N\times E}\\ \hfill W_q,W_k\in {\mathbb{R}}^{N\times d_a},W_v\in {\mathbb{R}}^{E\times E},\end{array}\right.$$

(2)

where $W_q$, $W_k$, $W_v$ represent the weight matrix of shared learnable linear transformation layers, and $d_a$ represents the dimensions of Q and K.

After obtaining Q, K, and V, the query matrix and keyword matrix are used to calculate the attention weight $\tilde{A}$ according to Equation (3).

$$\tilde{A}={\left(\tilde{a}\right)}_{i,j}=Q·K^T$$

(3)

The attention matrix A is obtained by normalizing the attention weight $\tilde{A}$ from Equation (3) using the SoftMax operator and $L_1$-norm, as shown in Equation (4).

$$\left\{\begin{array}{cc}\hfill {\tilde{a}}_{i,j}& =SoftMax\left({\tilde{a}}_{i,j}\right)={\displaystyle \frac{exp\left({\tilde{a}}_{i,j}\right)}{{\sum }_{k}exp\left({\tilde{a}}_{i,j}\right)}}\hfill \\ \hfill a_{i,j}& ={\displaystyle \frac{{\overline{a}}_{i,j}}{{\sum }_k{\overline{a}}_{i,j}}}\hfill \\ \hfill A& ={\left(a\right)}_{i,j}\hfill \end{array}\right.$$

(4)

The self-attention output features $F_{sa}\in {\mathbb{R}}^{N\times E}$ are obtained by using Equation (5). Then, the offsets of $F_e$ and $F_{sa}$ are calculated. Finally, the offset transformed by the LBR layer is added to $F_e$ to obtain the output feature $F_{out}$, and the offset of $F_e$ and $F_{sa}$ is similar to the calculation of Laplacian matrix.

$$\left\{\begin{array}{cc}\hfill F_{sa}& =A·V\hfill \\ \hfill F_{out}& =OA\left(F_e\right)=LBR(F_e-F_{sa})+F_e\hfill \\ \hfill F_e-F_{sa}& \approx L{F}_e,\hfill \end{array}\right.$$

(5)

where L represents the Laplacian matrix.

Offset attention uses element-wise offsets between input features and self-attention features to approximate the Laplacian operator, thereby enhancing network performance. Compared to self-attention in the original Transformer, offset attention offers stronger adaptability, higher robustness, and improved model performance.

To address potential issues such as gradient vanishing or exploding during training, we integrate a residual network structure into the offset-attention module. Specifically, we introduce ResNet [35], which employs a single residual module structure as illustrated in Figure 5.

In this structure, each residual module consists of a series of convolutional layers followed by a shortcut connection that directly adds the input of the module to its output. This design helps to mitigate the gradient vanishing problem by allowing gradients to flow more easily through the network during backpropagation. Additionally, the use of the shortcut connections facilitates the learning of identity mappings, enabling the network to learn residual functions effectively.

The network performs both max- and average-pooling operations on the extracted features independently. Max-pooling captures the most salient features by selecting the maximum value within each pooling region, while average pooling smoothes the feature map by averaging values. These pooled features are then merged to form the final global feature $F_g$, which encapsulates both the detailed and averaged representations of the input data. This approach enhances the ability of the model to generalize and improve performance across various tasks.

3.2.2. Sparse Submanifold Convolution-Based Local Feature Extraction

To achieve superior computational efficiency and reduce memory consumption compared to traditional voxelization methods, we employ sparse submanifold convolution for voxel aggregation. This method only performs convolution on relevant, non-empty voxels, significantly reducing unnecessary computations and memory footprint while effectively extracting local feature information from the point cloud data.

The processing steps of the local feature extraction module mainly involve voxelization of the original point cloud, local feature extraction based on sparse submanifold convolution, and devoxelization.

Voxelization of the original point cloud. To process point cloud data of different scales using convolution, we first voxelize the point cloud, then normalize the coordinates of the input PCD. All points are translated to a local coordinate system centered at the centroid. Then, all points are normalized to fit within a unit sphere by dividing by the maximum L2 norm max$\left|\right|p_i{\left|\right|}_2$. The coordinates are linearly mapped (scaled and shifted) from [−1, 1] to [0, 1], completing coordinate normalization. The normalized coordinates are represented as $\left\{{\widehat{p}}_i\right\}$, where ${\widehat{p}}_i=({\widehat{x}}_i,{\widehat{y}}_i,{\widehat{z}}_i)$ represents the voxel resolution. After obtaining the normalized coordinate information, the coordinates and voxel resolution r are used to calculate the voxel index for each point. For points falling into a voxel, the features $f_k$ corresponding to the coordinates ${\widehat{p}}_i$ undergo average pooling, with the average feature becoming the current voxel feature. The normalized point cloud $\{{\widehat{p}}_i,f_k\}$ is transformed into a voxel grid $\left\{V_{u,v,w}\right\}$. The specific calculation formula is shown in Equation (6).

$$V_{u,v,w,c}={\displaystyle \frac{1}{N_{u,v,w}}}\sum _{k=1}^n\mathbb{I}[floor({\widehat{x}}_k\times r),floor({\widehat{y}}_k\times r),floor({\widehat{z}}_k\times r)]\times f_{k,c},$$

(6)

where $V_{u,v,w,c}$ denotes the feature value of the c-th channel of the voxel grid at position $(u,v,w)$, r represents the voxel resolution, $\mathbb{I}[·]$ represents a binary indicator of whether the coordinate ${\tilde{p}}_i$ belongs to the voxel grid (u, v, w), $f_{k,c}$ represents the c-th channel feature associated with ${\tilde{p}}_i$, and $N_{u,v,w}$ represents the normalization factor.

Submanifold sparse convolution. To implement the convolution operation, we utilize a hash table and a matrix, each row in the matrix represents an active point, and the hash table contains tuples of all active points along with their positions in the matrix rows. Additionally, we define a rule book as a collection $R=(R_i:i\in F)$, which consists of $f^d$ integer matrices, where f represents the size of the filter, and $F={\{0,1,......,f-1\}}^d$ denotes the spatial size of the convolutional filter. We reuse the input hash table for output and construct an appropriate rule book to implement sparse submanifold convolution. The cost of building the hash table and the rule book is $O\left(l\right)$, and it is independent of the depth of the network, where l is the number of active points in the input layer.

Devoxelization. To map voxel features to their corresponding points in the point cloud, we employ trilinear interpolation [36] for upsampling. This process essentially recovers higher resolution features by considering the eight nearest neighboring points in the local feature space and interpolating their values along all three dimensions. For all points requiring interpolation, the steps outlined above are repeated. The voxelized local features are then mapped back to the original point cloud space, completing the process of voxelization and obtaining $F_l$ of the original point cloud.

3.3. Bounding Box Processing Module

3D-BoNet [9] consists of a backbone network and two parallel network branches, which perform bounding box regression and per-point mask prediction, respectively. This network structure is simple and easy to train and deploy. Compared to anchor-based methods, 3D-BoNet does not require a complex and time-consuming process for generating candidate bounding boxes, resulting in higher efficiency. Compared to anchor-free methods, 3D-BoNet explicitly predicts the bounding boxes of targets, resulting in instances with better precision. Therefore, we adopt the bounding box regression branch and point mask prediction branch from 3D-BoNet as the bounding box processing module in TSPconv-Net.

3.3.1. Bounding Box Regression Branch

The bounding box regression branch uses the global features $F_g$ extracted from the backbone as the input to the branch and directly regresses the bounding box B and the corresponding bounding box score $B_s$. By utilizing the predicted bounding box B and the ground truth bounding box $\overline{B}$ through the bounding box association layer, a loss function is constructed, and the overall network is optimized by minimizing this loss function.

Bounding box encoding. Unlike defining bounding boxes using the center position and three-dimensional lengths, this module represents the bounding box using two sets of parameterized maximum–minimum vertices: $\{[x_{min},y_{min},z_{min}],[x_{max},y_{max},z_{max}]\}$.

Neural layer. This consists of two fully connected layers that use the Leaky ReLU function as the nonlinear activation function and two parallel fully connected layers, forming two branches that output different results. One layer outputs a $6H$ dimensional vector and reconstructs it into an $H\times 2\times 3$ tensor, where H represents the maximum number of bounding boxes the network can predict, and $B\in R^{H\times 2\times 3}$ represents the predicted bounding box. The other layer calculates the corresponding bounding box score $B_s$ using the sigmoid function. The bounding box score $B_s$ is positively correlated with the likelihood of an object being contained within the bounding box.

Bounding box association layer. This part associates the predicted bounding boxes B with the ground truth bounding boxes $\overline{B}\in R^{T\times 2\times 3}$, where T is the number of ground truth bounding boxes. Finally, the overall loss function of the network is defined based on the cost between B and $\overline{B}$. Since the number of predicted bounding boxes usually differs from the number of ground truth bounding boxes (assuming $H\ge T$), the network is required to associate a unique predicted bounding box $B_i$ for each ground truth bounding box ${\overline{B}}_j$ to achieve optimal allocation. Assume C is the cost matrix between predicted and ground truth bounding boxes, where $C_{i,j}$ represents the similarity between the two bounding boxes, with smaller costs indicating higher similarity. Therefore, the bounding box association layer is tasked with determining the allocation matrix A that minimizes the overall cost. The specific calculation is shown as Equation (7).

$$\begin{array}{c}\hfill A=arg\underset{A}{min}\sum _{i=1}^H\sum _{j=1}^T{C}_{i,j}{A}_{i,j},\end{array}$$

(7)

where A is a Boolean association matrix, and $A_{i,j}$ indicates whether the i-th predicted bounding box is assigned to the j-th ground truth bounding box. Each column of $A_{i,j}$ represents the association probabilities between the j-th ground truth bounding box and all predicted bounding boxes, so the sum of the elements in each column is 1. Moreover, since each predicted bounding box is only associated with nearby ground truth boxes, the sum of the elements in each row does not exceed 1.

Due to the sparsity and uneven distribution of 3D point clouds, it is not feasible to directly use the Euclidean distance between the predicted bounding box and the ground truth bounding box as a basis for judgment. As shown in Figure 6, predicted bounding box $PB2$ has more valid points compared to predicted bounding box $PB1$ and a greater overlap with the ground truth bounding box $GB$. Therefore, the ground truth bounding box $GB$ should be associated with predicted bounding box $PB2$. As a result, the bounding box regression branch not only considers the Euclidean distance between vertices and the soft intersection over union but also incorporates the cross-entropy score into the cost matrix calculation. To solve this optimal allocation problem, the bounding box regression branch uses the Hungarian algorithm [37,38] to compute the optimal solution.

The costs of the three components are calculated as follows:

(1) Euclidean Distance. Defined as the Euclidean distance between the i-th predicted bounding box $B_i$ and the j-th ground truth bounding box ${\overline{B}}_j$. It is the average of the squared differences between the vertices. The detailed computation is presented in Equation (8).

$$C_{i,j}^{ed}={\displaystyle \frac{1}{6}}\sum {\left(B_i-\overline{B_j}\right)}^2$$

(8)

(2) Soft Intersection over Union (sIoU). Defined as the sIoU between the predicted bounding box and the ground truth bounding box. The specific calculation is shown as Equation (9).

$$C_{i,j}^{sIoU}={\displaystyle \frac{-{\sum }_{n=1}^N\left(q_i^n\ast {\overline{q}}_j^n\right)}{{\sum }_{n=1}^N{q}_i^n+{\sum }_{n=1}^N{\overline{q}}_j^n-{\sum }_{n=1}^N\left(q_i^n\ast {\overline{q}}_j^n\right)}},$$

(9)

where $q_i^n$ represents the probability of the n-th point being inside the i-th predicted bounding box, and ${\overline{q}}_j^n$ represents the probability of the n-th point being inside the j-th ground truth bounding box. Probabilities are calculated using Algorithm 1.

Algorithm 1 Calculate the probability of points in the input point cloud P being inside the predicted bounding box B. Where H represents the number of predicted bounding boxes, N represents the number of points in the input point cloud, ${\theta }_1$ and ${\theta }_2$ are hyperparameters set to 100 and 20, respectively, for numerical stability.

1: for $i\leftarrow 1$ to H do 2:     Obtain the i-th bounding box minimum vertices $B_{min}^i=[x_{max}^i,y_{max}^i,z_{max}^i]$ 3:     Obtain the i-th bounding box maximum vertices $B_{max}^i=[x_{max}^i,y_{max}^i,z_{max}^i]$ 4:     for $n\leftarrow 1$ to N do 5:         Obtain the n-th point $P^n=(x^n,y^n,z^n)$ 6:         ${\Delta }_{xyz}\leftarrow \left(B_{min}^i-P^n\right)\left(P^n-B_{max}^i\right)$ 7:         ${\Delta }_{xyz}\leftarrow max\left(min\left({\theta }_1{\Delta }_{xyz},{\theta }_2\right),-{\theta }_2\right)$ 8:         Obtain probability $P_{xyz}={\displaystyle \frac{1}{1+e^{(-{\Delta }_{xyz})}}}$ 9:         Obtain point probability $q_i^n=min\left(P_{xyz}\right)$ 10:   end for 11:   Obtain the soft-binary vector ${\mathbf{q}}_i=[q_i^1,\dots ,q_i^N]$ 12: end for

(3) Cross-Entropy Score. Defined as the cross-entropy score between $q_i^n$ and ${\overline{q}}_j^n$, indicating the confidence that the predicted bounding box contains as many effective points as possible. The calculation is specifically shown in Equation (10).

$$C_{i,j}^{ces}=-{\displaystyle \frac{{\sum }_{n=1}^N\left[{\overline{q}}_j^{n}log{q}_i^n+\left(1-{\overline{q}}_j^n\right)log\left(1-q_i^n\right)\right]}{N}}$$

(10)

The bounding box prediction multi-criterion loss is computed by the bounding box association layer, which determines the optimal predicted bounding box for each ground truth bounding box by minimizing the cost. The loss function consists of three components: (1) Euclidean distance, (2) sloU, and (3) cross-entropy, as formally expressed in Equation (11):

$$l_{bbox}={\displaystyle \frac{1}{T}}\sum _{t=1}^T\left(C_{t,t}^{ed}+C_{t,t}^{sIoU}+C_{t,t}^{ces}\right).$$

(11)

The bounding box prediction score loss evaluates the validity of the predicted bounding boxes. After reordering by the association matrix A, the ground truth values for the first T scores are “1” and “0” for the remaining $H-T$ scores. The loss function is defined using cross-entropy as Equation (12).

$$l_{bbs}=-{\displaystyle \frac{1}{H}}\left[\sum _{t=1}^{T}log{B}_s^t+\sum _{t=T+1}^{H}log\left(1-B_s^t\right)\right],$$

(12)

where $B_s^t$ represents the score of the t-th associated predicted bounding box.

3.3.2. Point Mask Prediction Branch

The point mask prediction branch feeds the global features $F_g$ and local features $F_l$ into the neural layer to refine the predicted bounding boxes B, as shown in Figure 7, thereby enhancing the segmentation performance of the predicted bounding boxes. Moreover, focal loss with default hyperparameters is used to optimize the point mask prediction branch.

Neural layers. The global and local features are compressed into 256-dimensional vectors through fully connected layers and further compressed into 128-dimensional fused features ${\tilde{F}}_l$. For the i-th predicted bounding box $B_i$, the predicted vertices are fused with the fused features to obtain the perception features ${\widehat{F}}_l$. Finally, the point mask $M_i$ is obtained through shared fully connected layers.

The loss function of the bounding box processing network consists of the loss functions from both the bounding box regression branch and the point mask prediction branch. TSPconv-Net optimizes the network through this loss function to improve network performance and segmentation effectiveness. The loss function of TSPconv-Net is defined as Equation (13).

$$l_{all}=l_{sem}+l_{bbox}+l_{bbs}+l_{pmask},$$

(13)

where $l_{sem}$ denotes the standard softmax cross-entropy loss function used for learning the semantic information of each point. $l_{pmask}$ represents the focal loss [39] with default hyperparameters, used for optimizing the point mask prediction branch.

4. Experiments and Analysis

We primarily utilize the publicly available dataset S3DIS [14] provided by Stanford University. This dataset comprises 6 educational and office areas, totaling 272 rooms, with over 215 million points in point clouds. These rooms encompass 11 different scenes, such as offices, conference rooms, and auditoriums. We utilize Python and PyTorch 1.10 to construct TSPconv-Net and conducted training on a single GTX 1660Ti. TSPconv-Net was trained using mini-batches of size 32, ensuring a manageable computational load while maintaining gradient stability. The training process was conducted over 250 epochs, allowing the network sufficient time to converge and optimize performance. The initial learning rate was set to 0.01, enabling efficient exploration of the parameter space in the early stages of training, with adjustments made as necessary based on the learning rate scheduler to fine-tune model performance as training progressed.

The dataset comprises 13 semantic categories, with each point in each scene labeled as 1 of these 13 semantic categories, including ceiling, floor, wall, beam, column, window, door, table, chair, sofa, bookcase, board, and other objects. We select Area1, Area3, Area4, Area5, and Area6 as the training set, while Area2 served as the test set.

4.1. Visualization Results

To more clearly demonstrate the optimization of object boundary extraction in our method, we select four representative scenes for visualization. As shown in Figure 8, we can clearly observe that, with the aid of bounding box prediction, our method excels in accurately extracting and distinguishing closely connected objects in the scene.

In these scenarios, traditional segmentation methods often suffer from blurred boundaries or overlapping objects, especially when the objects are densely packed or have complex shapes. However, by introducing bounding box prediction, our method can create clearer boundaries between objects, effectively reducing mis-segmentation.

For instance, in the classroom scene, our method precisely distinguishes and extracts the densely arranged chairs and tables. The bounding box helps to better define the spatial boundaries of the objects, avoiding confusion between chairs or between chairs and tables. Similarly, in conference room scenes, bounding box prediction effectively separates closely positioned chairs and tables, ensuring that each object is accurately recognized and segmented. This approach not only enhances segmentation performance in complex scenes but also significantly improves the model’s ability to handle details, making the boundaries of each object more distinct.

Through this optimization, our method has significantly improved the extraction of individual objects, especially in scenarios where objects are densely arranged or have complex shapes. Compared to traditional segmentation methods, the bounding box prediction technique effectively reduces mis-segmentation, making the final segmentation results more visually clear and accurate. This further proves the superiority of our method in multi-object scenes.

4.2. Comparison Results Analysis

We compare the TSPconv-Net with the SGPN [23], ASIS [25], and 3D-BoNet [12]. The comparison results are presented in

Table 1. Results of TSpconv-Net on the S3DIS Dataset.

Methods	mPrec (%)	mRec (%)	mAP (%)	Total Time (s)
SGPN [23]	38.2	31.2	43.6	8821.3
ASIS [25]	63.6	47.5	55.3	402.3
3D-BoNet [9]	65.6	47.6	57.5	294.1
TSPconv-Net	68.6	52.2	60.1	326.1

. For quantitative evaluation of the data results, we primarily assessed the mean precision (mPrec), mean recall (mRec), average precision (AP), and mean average precision (mAP). From the observations, it can be concluded that our method outperforms the others in terms of mPrec and mRec. Compared to SGPN, our network shows an improvement of 30.4% and 21% in mPrec and mRec, respectively. Compared to the ASIS network, our method exhibits an improvement of 5% and 4.7% in mPrec and mRec, respectively. Compared to the 3D-BoNet [12], our method demonstrates an improvement of 3% and 4.6% in mPrec and mRec, respectively.

Our research demonstrates that proposal-based techniques, exemplified by 3D-BoNet, outperform direct feature-based methods in repetitive object extraction, achieving superior precision. This advantage likely arises from their ability to generate candidate regions likely to contain objects, focusing analysis on these key areas. Furthermore, TSPconv-Net achieves even higher precision than 3D-BoNet by leveraging offset attention (OA) and submanifold convolution (SSC) as the backbone for feature extraction, thus obtaining more effective features. To objectively assess segmentation performance, we employ the mean average precision (mAP), which is a standard metric for evaluating object detection accuracy. As shown in Table 1, our method achieves a mAP of 60.1%, demonstrating a significant improvement of 16.5% compared to SGPN and outperforming ASIS by 4.8%. Additionally, it outperforms the previous version of 3D-BoNet by 2.6%. Finally, we evaluate the time efficiency of our method against existing approaches (Table 1). Due to the introduction of SSC in our network, reducing a significant amount of unnecessary spatial convolution operations, TSPconv-Net demonstrates exceptional efficiency, processing data nearly 26 times faster than SGPN. It also exhibits faster processing compared to ASIS.

In summary, our method demonstrates excellent results in terms of average precision, average recall, mean average precision, and computation time. This indicates that using predicted bounding boxes for single-object extraction can obtain higher precision in individual object extraction.

5. Conclusions

In this paper, we propose an efficient 3D instance segmentation framework called TSPconv-Net. We introduce OA and SSC as the backbone network of TSPconv-Net, greatly enhancing the ability of TSPconv-Net to capture and represent deep features of point clouds. We feed these features into the bounding box processing module based on 3D-BoNet to obtain segmentation results. TSPconv-Net achieves better performance and comparable speed to 3D-BoNet on the S3DIS dataset. This further confirms the effectiveness of using OA and SSC for extracting point cloud features.

However, due to the introduction of the offset-attention mechanism to handle global features of the point cloud, TSPconv-Net undoubtedly increases the computational complexity of the network. Although TSPconv-Net avoids a large amount of meaningless computation through submanifold sparse convolution, its processing speed is still not ideal compared to the original 3D-BoNet. This issue warrants further investigation in future research, aiming to find ways to enhance the network’s efficiency.