PAN: Improved PointNet++ for Pavement Crack Information Extraction

Abstract

Maintenance and repair of expressways are becoming increasingly important due to the growing frequency of their use. Accurate pavement crack information extraction helps with routine maintenance and reduces the risk of traffic accidents. The traditional 2D crack image detection method has limitations and cannot effectively obtain depth information. Three-dimensional crack extraction from 3D point cloud has become a new solution that can capture pavement crack information more comprehensively and accurately. However, the existing algorithms are not effective in the feature extraction of cracks due to the different and irregular shapes and sizes of pavement cracks and interference from the external environment. To solve this, a new method for detecting pavement cracks in point clouds, namely point attention net (PAN), is herein proposed. It uses a two-branch attention fusion module to focus on space and feature information in the cloud and capture features of crack points at different scales. It also uses the Poly Loss function to solve the imbalance of foreground and background points in pavement point cloud data. Experiments on the LNTU-RDD-LiDAR dataset were carried out to verify the effectiveness of the proposed method. Compared with the traditional method and the latest point cloud segmentation technology, the performance indexes of mIoU, Acc, F1, and Rec achieved significant improvement, reaching 75.4%, 91.5%, 75.4%, and 67.1%, respectively.

1. Introduction

Highway pavement crack information extraction is very important in road quality assurance and driving safety. The early-detection methods relying on manual and sensor have high cost, high detection cost, low efficiency, and strong subjectivity, which makes road crack detection complicated and difficult [1]. The modern machine vision detection method improves the detection efficiency and accuracy to a certain extent. It provides a new solution for the identification and maintenance of highway cracks, which helps to reduce cost, improve efficiency, reduce the impact of road cracks on traffic, and improve the safety and service life of highways.

Traditional methods have been extensively studied for automatic pavement crack detection. Image processing-based techniques primarily include threshold segmentation [2,3], edge detection [4,5], and region growth [6,7]. However, these approaches often rely on prior knowledge or optimal thresholds, limiting their applicability on complex and variable urban roads, particularly in reliably detecting cracks with weak connections or uneven geometric topologies. Some studies have proposed an information extraction method for calculating crack length [8] and used a data acquisition system combining radar rangefinder and camera [9]. Image segmentation was carried out using gray adaptive threshold algorithm, crack contour was extracted by GaussLaplace algorithm [10], and video-based software system was developed for calculating bridge crack widths. However, traditional methods often have low accuracy and can only approximate crack locations in complex backgrounds or when there is minimal contrast. These methods are also easily affected by the background.

With the rise of deep learning, crack detection has ushered in a new stage of development, and the quality of crack detection has been significantly improved by learning advanced features [11]. The use of deep learning in pavement crack information extraction allows for automated feature extraction and learning without human intervention. Additionally, Transformer models based on the seq2seq architecture have achieved significant success in crack detection, offering not only feature extraction but also multimodal fusion, addressing some of the limitations of CNNs [12]. While two-dimensional detection has made some progress, these image-based methods still face challenges, as pavement images are frequently obscured by lighting, shadows, dirt, and noise, which hinders the accurate capture of terrain and texture details.

With the advancement of 3D data acquisition technology, mobile laser scanning (MLS) systems have become widely used for generating precise 3D coordinate data, enabling efficient and flexible point cloud acquisition on road surfaces [13]. Pavement point clouds exhibit characteristics that include 3D spatial coordinates and intensity information for each point, with crack points typically differing in elevation and intensity from the surrounding pavement points. These features make the pavement point cloud an effective tool for analyzing road conditions and detecting damage. The point cloud’s irregularity offers rich information about the pavement surface. The normal vector, indicating the direction at each point, is used for crack segmentation. However, road wear and noise complicate distinguishing elevation and intensity changes between cracks and normal pavement, posing challenges for crack detection.

While current methods have shown promising results, point cloud-based crack-detection techniques face three significant challenges that hinder their practical application. (1) Disorder: LiDAR systems generate 3D point cloud data that are inherently unstructured and unordered, creating difficulties for deep learning algorithms. To address this, many existing methods employ dimension reduction techniques to transform 3D point clouds into 2D images, thereby simplifying the processing. However, this conversion often results in information loss [14]. Moreover, these methods often treat point clouds as discrete, uncorrelated sets, overlooking elevation differences between cracked and non-cracked points. (2) Spatial correlation: Existing point cloud-based crack-detection methods tend to ignore the spatial correlation and adjacency of points in complex pavement structures. Since the pavement is relatively flat, neglecting these correlations can lead to insufficient recognition of small crack details, resulting in incomplete detection. (3) Data dependency: Deep learning models based on point clouds typically require large amounts of manually labeled data and involve training a significant number of parameters. This not only increases the input of manpower, time, and cost but also significantly increases the computing cost with the increase of network complexity [15]. These limitations limit the applicability of current point cloud-based crack-detection methods in different scenarios.

Based on the above problems, the following improvements are proposed:

(1)

This paper introduces a PAN network built on a U-Net architecture that processes unordered point sets and directly extracts features, avoiding the information loss often caused by dimensionality reduction in traditional methods. This approach effectively preserves the spatial information and geometric characteristics of point cloud data, enhancing the accuracy and efficiency of point cloud processing;

(2)

In this paper, we present a PC-Parallel module featuring a dual attention module branch parallel structure, which flexibly adapts to pavement point cloud data of varying densities and samples. This design enhances the model’s ability to understand both overall and local features, especially in extracting crack information from complex structures, and addresses segmentation result incompleteness. It not only improves the model’s robustness across multi-scale and multi-density data but also enhances its capability to capture crack edges and details, thereby increasing the accuracy of point cloud pavement crack segmentation;

(3)

The Poly Loss function is introduced. By adjusting the form of the loss function, the imbalance between crack points and background points can be better balanced, and the sensitivity to edge and detail information can be enhanced. The problem of boundary refinement and class imbalance in point cloud pavement crack segmentation is solved effectively, thus improving segmentation accuracy and model performance;

(4)

A set of large-scale 3D point cloud dataset of pavement cracks suitable for semantic segmentation is established.

2. Related Work

2.1. Methods Based on Traditional Image Processing

Image-based crack-detection algorithms have rapidly advanced. Some studies [2,16] introduced an algorithm for extracting crack thresholds using global or local thresholds. However, selecting suitable thresholds for most crack images is challenging, and the method is highly sensitive to illumination and noise, reducing its stability. Other research [17,18] proposed a block-based segmentation approach, but because detection occurs at the block level, it fails to accurately identify cracks. Different researchers [19,20] developed a filter-based algorithm to detect cracks based on the expected response, but it struggles with complex and varied pavement cracks, and selecting appropriate parameters is often difficult and time-consuming. Another study [6] introduced a seed-based region growth algorithm that uses multidirectional non-minimum suppression and symmetry checking for road crack detection. One study [7] proposed CrackTree, which detects cracks by selecting them in a probability graph and uses a recursive tree edge pruning method to form a minimum spanning tree. This method can detect pavement cracks effectively but does not consider the actual crack width. In the freeway environment, the existing methods often need appropriate parameter presets or prior knowledge to achieve the best performance when dealing with the complex and changeable pavement. This dependence can lead to poor model performance in the absence of human intervention. Pavement cracks often have limited connectivity or irregular geometric topologies, and correctly identifying these cracks remains a major challenge.

2.2. Deep Learning Image Processing Method

In recent years, due to its excellent feature learning ability, many researchers have explored various deep neural networks and demonstrated excellent performance in the efficiency and accuracy of road crack assessment. One study [21] for the first time tried, CNN based on LeNet-5 [22], to detect cracks in local areas of images. This method involves converting the model into a fully convolutional network to obtain image segmentation results. Methods like [23,24] use sliding windows to divide pavement images into smaller blocks, utilizing convolutional neural networks (CNNs) to predict cracks within each block, accurately locating crack positions. However, this approach is limited to block-level detection and does not achieve pixel-level precision. Studies [25,26] have demonstrated that using CNNs for pixel-level crack detection in pavement images allows for precise crack localization and improved accuracy. Despite its precision, the block-based process can be time-consuming, and small blocks often lack sufficient context, which limits real-time performance and efficiency. On the other hand, the application of fully convolutional networks (FCNs) in crack detection has delivered impressive results in both accuracy and speed [27,28]. Building on this success, [29] proposed combining a deep convolutional neural network (DCNN) with an edge detector to further enhance detection performance. This integration allows for more efficient processing and better edge definition, improving the overall detection capability of pavement cracks. By integrating the DCNN output with the edge detector, this approach effectively reduces the noise rate in crack identification and significantly enhances recognition accuracy. Compared to other deep learning methods, this new approach excels in noise suppression and accuracy. However, the layer number of convolutional networks is deep, resulting in a large increase in model parameters, which makes the training time of the model quite time-consuming. One study [30] utilized the Faster R-CNN two-stage network for high-precision pavement crack information extraction. Compared to the one-stage network, the two-stage approach is more accurate for multi-scale and small-target issues, offering a more reliable solution for accurately locating pavement cracks. However, this increased accuracy comes at the cost of reduced efficiency, resulting in slower detection speeds. Another study [31] proposed a feature pyramid and hierarchical boost network that processes multiple scales simultaneously and retains more details by transmitting information layer by layer. This allows the network to effectively capture features at different scales, adapting better to pavement cracks of varying sizes and shapes, thereby enhancing robustness and accuracy in pavement crack information extraction. However, with the introduction of multi-scale information, it is also affected by some noise and unnecessary information, which reduces the performance. Despite the impressive results of image-based learning methods, their performance heavily depends on external conditions. Two-dimensional pavement images and videos are frequently obscured by light, shadows, stains, and rust, complicating road crack identification. Additionally, 2D image detection techniques are limited in their ability to describe terrain and crack details, reducing the effectiveness of road maintenance assessments.

2.3. Method Based on Traditional 3D Point Cloud Data Processing

Compared to traditional 2D optical images, 3D point clouds provide more precise spatial coordinates and include intensity information. Unlike 2D images, 3D point clouds are unaffected by ambient brightness, offering a more reliable basis for accurately extracting road cracks. Recently, researchers have extensively studied using point clouds for pavement crack detection. By thoroughly analyzing point cloud data, they can detect and locate pavement cracks more accurately and improve detection robustness and accuracy. Some studies [32,33] proposed a deep crack-detection method using 3D pavement data, employing a straightforward threshold strategy to represent crack depth and length. The ITVCrack algorithm proposed by [34] is an automatic crack extraction framework based on iterative tensor voting (ITV). The algorithm first uses the inverse distance weighting (IDW) algorithm to convert pavement points into geo-referenced feature images and then detects candidate cracks using an ITV-based crack extraction framework. Finally, a morphological refinement algorithm is used to distinguish the crack curves of MLS pavement. However, due to the characteristics of the ITV algorithm, ITVCrack has high computational complexity when processing large-scale data, and its generalization performance under different road and environmental conditions is also limited. In another study [35], The Otsu threshold method was used to extract intensity differences in MLS point clouds to identify crack skeletons. Noise was then removed using a spatial density filter, and the Euclidean distance clustering algorithm divided crack points into distinct crack lines. However, this process is affected by changes in point cloud density and local shape, reducing crack line division accuracy. Another study [36] applied the plane triangulation modeling method to detect crack points on a triangular irregular network dataset constructed with the IDW rasterization method. However, the planar triangulation modeling method may lack sufficient adaptability to crack shape changes and multi-scale problems, resulting in missed or false detections in some cases. One study [37] introduced a Gaussian filter to detect the signal-to-noise ratio distribution gradient, efficiently extracted pavement cracks from terrestrial laser scanning (TLS) point cloud, and effectively optimized the accuracy of pavement damage analysis. However, the anti-interference ability against factors such as noise and occlusion needs to be further improved. In Ref. [38], a random forest classification (RFC) method was applied using LiDAR point clouds captured by drones. Multi-scale and multi-dimensional features are extracted from the intensity and height data of the point clouds and used as input for the RFC method to identify cracks. One study [39] proposed a pavement crack extraction technique by converting MLS point clouds into regular grid structures. This approach introduced a two-dimensional index based on the acquisition time or incident angle of each 3D point. Crack candidate points are detected by combining intensity and height differences. Finally, the MLS data are converted into a “grid map”, demonstrating its feasibility and effectiveness for transverse, longitudinal, and oblique cracks.

Compared to traditional 2D optical images, 3D point clouds offer more precise spatial coordinates and include intensity information. Unlike 2D images, 3D point clouds are unaffected by ambient brightness, providing a more reliable basis for accurately extracting pavement cracks. In recent years, researchers have extensively explored using point clouds for pavement crack detection. By thoroughly analyzing point cloud data, researchers can detect and locate pavement cracks with greater precision, enhancing the robustness and accuracy of the detection process [32,33]. A deep crack-detection method using 3D pavement data was proposed, utilizing a straightforward threshold strategy to represent crack depth and length. However, this approach has limitations in successfully extracting complex and fine cracks. The ITVCrack algorithm [34] offers an automatic crack extraction solution based on iterative tensor voting (ITV). It first uses the inverse distance weighting (IDW) algorithm to convert pavement points into geographic reference feature images. Candidate cracks are then identified using the ITV crack extraction framework, and a morphological refinement algorithm is applied to distinguish crack curves in the point cloud pavement. However, due to the characteristics of ITV algorithm, ITVCrack has high computational complexity when processing large-scale data, and its generalization performance under different road and environmental conditions is also limited. In [35], the Otsu threshold method was employed to extract intensity differences in the point cloud for identifying the fracture skeleton. Noise was then removed using a spatial density filter, and fracture points were divided into distinct fracture lines using the Euclidean distance clustering algorithm. However, changes in point cloud density and local shape during fracture line division reduce accuracy. Researchers [36] used the planar triangulation modeling method to detect crack points in the triangular irregular network dataset constructed by the IDW rasterization method. However, the planar triangulation modeling method may not have enough adaptability to the shape change and multi-scale problems of cracks, resulting in missing or false detection in some cases. In [37], a Gaussian filter was introduced to detect the signal-to-noise ratio distribution gradient, and pavement cracks were efficiently extracted from the ground laser scanning (TLS) point cloud to effectively optimize the pavement damage analysis accuracy. However, the anti-interference ability for noise and shielding factors needs to be further improved. In [38], a random forest classification (RFC) method was implemented using LiDAR point clouds captured by drones. Multi-scale and multi-dimensional features are extracted based on the intensity and height information of the point clouds, serving as input for crack extraction via the RFC method. In [39], by converting the point cloud into a regular grid structure, a two-dimensional index was introduced to account for the acquisition time or angle of incidence of each 3D point. Then, the fracture candidate points were detected by the combination of strength difference and height difference. At last, the MLS data were converted losslessly into a “grid graph”, and their feasibility and effectiveness on transverse, longitudinal, and oblique cracks were proven. Traditional 3D point cloud processing methods often rely on hand-designed features or simple statistical features, which do not perform well in capturing the details of crack points. Effectively identifying and segmenting these features is challenging, often resulting in the loss or misjudgment of crack point features. In pavement point cloud crack segmentation, capturing local features is essential for detail identification, while combining global features helps understand the overall structure. Traditional methods struggle to process both local and global features simultaneously, thus impacting the accuracy of pavement point cloud crack segmentation.

2.4. Point Cloud Data Processing Method Based on Deep Learning

2.4.1. Classic Deep Learning Point Cloud Segmentation Algorithm

The tremendous success of deep learning in image processing has led to its expansion into 3D data processing. In point cloud data processing, classic deep learning segmentation algorithms have achieved impressive results. Their core idea is to efficiently segment unstructured and disordered 3D point cloud data using deep learning networks to accurately identify the boundaries and features of various objects or scenes.

PointNet [40], a pioneering model for point cloud segmentation, uses shared MLPs to directly process unordered point sets as input and extract features from them. Building on PointNet, PointNet++ [41] builds on this by introducing a hierarchical feature learning paradigm that recursively captures local geometric structures. By leveraging local point representation and multi-scale information, PointNet++ achieves excellent performance and serves as the foundation for modern point cloud methods [42,43,44]. An improved PointNet++ [45] was used to classify ejector head defect shapes in seamless steel pipe production, adding a multi-level local feature extraction structure that reduces point count, enhances information acquisition, and provides better stability than PointNet. Advances in natural language processing have led to attention-based methods that excel in exploring point relationships. Models like PCT [46] and Point Transformer [47,48] establish global context in point clouds through self-attention, effectively managing point positions and capturing spatial structures. A Transformer-based point cloud classification network (TransPCNet) [49] was developed to identify sewer defects, using a feature embedding module to map points to a high-dimensional space for feature extraction and closure, learning multi-scale features through a self-attention cascade. To enhance discrimination between similar defect classes, a weighted, smooth cross-entropy loss function was designed to prevent overfitting during classification training. Recently, MLP-type networks have achieved excellent results by simplifying the network structure and strengthening features. PointMLP [50] proposes a geometric affine module to normalize features. RepSurf [51] models the umbrella surface by fitting surface information with triangular planes to provide geometric information. PointNeXt [52] integrates training strategies and model scaling. These methods provide useful exploration and innovation in the development of point cloud processing field.

Currently, research methods are mainly focused on validating the effectiveness of these techniques on relatively more straightforward indoor tasks, while for pavement crack information extraction, there is the challenge that only a few points contain critical information. General point cloud segmentation networks often lack specialized designs for handling sparse feature point clouds, leading to computational redundancy and undue focus on irrelevant points. In extracting pavement crack information, cracks of varying scales require different levels of attention, highlighting the need for further improvement and optimization in current technology. Addressing these issues will enhance the accuracy and robustness of pavement crack information extraction, making it more applicable to real-world complex pavement scenarios.

2.4.2. Road Crack Segmentation Task Based on 3D Point Cloud

In road crack detection, deep learning networks based on point cloud data can generally be categorized into two types: those based on convolutional neural networks (CNN) and those based on graph convolution. These approaches utilize deep learning’s powerful representation learning capabilities to significantly enhance the accuracy and efficiency of road crack detection.

Convolutional Neural Network: In [13], a method utilizing an adaptive wavelet neural network (WNN) was proposed for the automatic detection of concrete cracks and other damage types. However, this approach may encounter errors when identifying cracks with fine textures. In [53], CrackNet, an efficient model based on a convolutional neural network (CNN), was proposed for automatic detection of sidewalk cracks on 3D asphalt surfaces. Although it achieves high pixel accuracy, its efficiency is hindered by a static, non-learnable feature generator, which restricts its ability to learn. One study [54] proposed an enhanced architecture based on CrackNet, namely CrackNet II, which abandoned the feature generator in favor of a more complex framework, which enabled the model to detect smaller or finer features while removing more local noise. cracks and maintain faster computing speeds. One study showed [55] CrackNet-V as an efficient deep network that uses smaller filters and introduces a new shallow crack activation unit. Compared to the original CrackNet, CrackNet-V features a deeper network architecture with fewer parameters, enhancing both computational efficiency and accuracy. However, despite these improvements, its performance gains are modest and highly reliant on the data. In [56], CrackNet-R, based on recurrent neural networks, was shown to incorporate a gated recursive multi-layer perceptron (GRMLP) to iteratively update its internal buffer. The GRMLP performs multiple layers of nonlinear transformations using gated units, allowing for deeper abstraction of input and hidden states. Compared to CrackNet, CrackNet-R improves detection speed by four times and significantly enhances detection accuracy.

Graph Convolution: From [57], it can be seen that CrackGCN is an innovative semi-supervised method for extracting 3D pavement crack information. It uses a novel spatial enhancement strategy and graph-based features to identify crack points from MLS data, boosting the effectiveness of GCN. By relying on a small amount of annotated data, CrackGCN constructs graphs to represent local features, thereby reducing data degradation and minimizing dependence on extensive annotations. RangeSeg [58] is a range-aware instance segmentation framework with a shared encoder backbone and two range-dependent decoders. The heavy decoder focuses on detecting distant and small objects by calculating their distance from the image’s top area, enhancing accuracy for small target detection. Meanwhile, the light decoder processes the entire image to reduce computational costs, effectively balancing efficiency and accuracy. SD-GCN [38], a saliency-based extended GCN network, employs two saliency feature spaces and cylinder-based extended graph convolution to detect cracks in moving laser scanning (MLS) point clouds. Both CrackGCN and SD-GCN improve the geometric structure of road point clouds using spatial enhancement strategies. However, they do not account for long-range neighborhoods and multi-scale features, resulting in incomplete crack detection within complex structures. SCL-GCN [15], a hierarchical contrastive learning graph convolution network, was designed for pavement crack extraction from MLS point clouds. It features a novel dual-branch architecture that utilizes multi-scale graphs to expand the effective receptive field for remote contexts while maintaining low computational costs. A graph feature-contrastive learning module guides the dual-branch GCNs, addressing learning biases from imbalanced data and enhancing convergence and performance.

PAN: Our research focuses on innovating the segmentation of pavement point cloud crack scenes. Unlike existing work, we extract 3D pavement crack information directly from MLS data, avoiding the information loss associated with traditional data conversion methods. To more fully leverage the rich information in point clouds, we herein introduce an advanced extended self-attention mechanism module called PC-Parallel.

The PC-Parallel module combines two types of attention modules, operating in parallel to more effectively capture contextual relationships in point cloud data. The spatial attention module targets the local relationships between different feature points, aiding in accurately locating small cracks. In contrast, the channel attention module focuses on long-range contextual information across the channel dimension to better understand overall road conditions. This organic combination significantly improves the model’s ability to process pavement crack point cloud data. We constructed a large-scale pavement crack point cloud dataset, i.e., LNTU-RDD-LiDAR, by ourselves, providing sufficient training and evaluation data for research. Through comprehensive empirical results, the effectiveness of our proposed method on this dataset was verified. Compared with traditional methods, our model directly detects cracks in MLS data, making the results more accurate and practical.

This innovative research not only made significant progress in methods, but also provided valuable resources for research in related fields by constructing a pavement crack point cloud dataset. Our work injects new ideas into the field of pavement crack information extraction and provides strong support for future research.

3. Materials and Methods

3.1. Point Attention Net Model Overview

As a general point cloud processing framework, PointNet++ excels in tasks like point cloud classification, semantic segmentation, and object detection, demonstrating wide applicability. However, in small target segmentation tasks, such as pavement crack extraction, PointNet++ uses a fixed receptive field. Although this field is gradually expanded through multiple Set Abstraction layers, it relies solely on the feature extraction stage of the PointNet layer to input several points into the fully connected layer, resulting in a relatively simple encoding method with low robustness.

To address these issues, this paper proposes a PAN network based on PointNet++, which directly processes unordered point sets as inputs and uses the Set Abstraction module to extract local features at different levels, capturing local structures in the point cloud. Through hierarchical subsampling and aggregation operations, point cloud information is captured at various scales, allowing the network to consider both local structures and global context. This improves the network’s understanding of the overall and local structure of the point cloud, enhancing its ability to process point cloud data with multi-scale characteristics. Using symmetric functions to handle the arrangement of input point clouds makes the network insensitive to point arrangements, ensuring consistent outputs across different input configurations. This approach enhances the model’s generalization and makes it more adaptable to point clouds with various shapes and structures, boosting its robustness in practical applications. Additionally, the proposed PC-Parallel module enlarges the model’s receptive field and strengthens encoding robustness. This module enhances PointNet++’s performance in small target segmentation tasks such as pavement point cloud crack detection, allowing it to better adapt to varying scales and complexities of point cloud data and improving its effectiveness in real-world applications.

The PC-Parallel module enhances the network’s capability to capture critical features over long distances, increasing adaptability and robustness. As illustrated in Figure 1, the PC-Parallel module is introduced after the SA block, where PointNet layer features are input. This module combines spatial and channel attention in parallel, leveraging the strengths of both mechanisms to improve point cloud segmentation performance. By modeling the attention matrix, the spatial attention component effectively captures the spatial relationship between any two points, enhancing the ability to identify local crack features and expanding the model’s perception of the crack region, which enhances the ability to process crack shapes, filter out irrelevant and noisy points, reduce redundant information in the point cloud, and improve computational efficiency. Simultaneously, the channel attention component focuses on capturing remote context information in the channel dimension, emphasizing feature channels with significant differentiation of crack features while suppressing those with minor contributions. This reduces noise interference and redundant information, improving the model’s overall performance.

Finally, aggregating the outputs of the two attention modules enhances the recognition of crack points across spatial and feature dimensions, achieving more effective multi-scale feature fusion. This process captures fracture point characteristics at different sampling levels, allowing the model to better understand the global correlation among fracture points. The combined use of spatial and channel attention enables the model to fully perceive and interpret the information within crack points, thereby improving its understanding of the overall structure and pattern. By obtaining better feature representations, these features can be used more accurately to predict crack points and provide more powerful modeling capabilities for crack point cloud segmentation tasks. Moreover, Poly Loss is introduced to adjust the form of loss function to better balance the imbalance between crack points and background points and significantly improve the identification accuracy of crack areas.

3.2. PC-Parallel Module

The PC-Parallel module consists of two types of attention branches, as illustrated in Figure 2: the spatial attention branch learns the relationship between different feature points, while the channel attention module captures remote context information along the channel dimension. Finally, the outputs from these two attention modules are aggregated to achieve a more effective point-level feature representation.

Spatial Attention Branch: Spatial attention allows the module to selectively focus on local regions around crack points, enhancing local features and improving the perception of crack areas. This is crucial for crack point segmentation, as crack features are typically small and scattered. By incorporating spatial attention, the model becomes more adaptable to point clouds of varying shapes and structures, enabling a deeper understanding and exploitation of spatial relationships between points to better capture local crack details. This enables the model to focus more on areas with important structure. By guiding the model to focus on the key part of the crack point and filtering irrelevant points and noise points, the computational cost of processing redundant information is effectively reduced, and the overall work efficiency is improved. Furthermore, spatial attention helps the model grasp the global correlations within the point cloud, enhancing its ability to capture overall structures by learning spatial relationships between points. This approach addresses the non-uniformity of point clouds, allowing the model to more accurately process crack points with varying densities and samples, thereby improving its effectiveness in handling pavement crack point cloud data.

We first input local features $A\in {\mathbb{R}}^{B\times C\times N}$, which are initially processed through two convolutional layers to produce new feature maps T and P, respectively, represented as $T\in {\mathbb{R}}^{B\times {\displaystyle \frac{C}{2}}\times N}$, $P\in {\mathbb{R}}^{B\times {\displaystyle \frac{C}{2}}\times N}$. Then, we reshape them into ${\mathbb{R}}^{B\times N}$, where $N=H\times W$. Next, matrix multiplication is performed between the transpose of T and P, followed by a softmax layer to compute the spatial attention map $\mathrm{P}\_\mathrm{att}\in {\mathbb{R}}^{B\times N\times N}$:

$${softmax}_{btp}={\displaystyle \frac{e^{{\mathrm{P}\_\mathrm{att}}_{btp}}}{{\displaystyle \sum _{l=1}^N{e}^{{\mathrm{P}\_\mathrm{att}}_{btl}}}}}$$

(1)

Among them, ${\mathrm{P}\_\mathrm{att}}_{btp}$ is a measure of the positional relationship between t and p. b represents the batch dimension, t represents the first spatial dimension, and p represents the second spatial dimension. We first input the feature $A\in {\mathbb{R}}^{B\times C\times N}$ into another convolutional layer to generate a new feature map $G\in {\mathbb{R}}^{B\times {\displaystyle \frac{C}{2}}\times N}$ and then reshape it to ${\mathbb{R}}^{B\times N}$. Then, we perform matrix multiplication between $\mathrm{P}\_\mathrm{att}$ and $G$ and reshape it to ${\mathbb{R}}^{\mathrm{B}\times N}$. Finally, the results are input into a convolutional layer and are then element-wise summed with feature A after normalization operation to obtain the final output $E\in {\mathbb{R}}^{B\times C\times N}$:

$$E_{att}=\mathrm{GN}\left({\mathrm{Conv}1d}_{\mathrm{out}}\left(\mathrm{softmax}\left({\displaystyle \frac{T^T\cdot P}{\sqrt{c\cdot n}}}\right)\cdot G\right)\right)+\mathrm{residual}$$

(2)

It can be seen from the formula that the final feature of each position ${spatial\_att}_E$ is the weighted sum of the features of all positions, taking into account the original features; c is the number of channels, and n is the number of points. Therefore, it has a global context view and selectively aggregates context according to the spatial attention map, helping the model focus on local areas, enhance local features, and reduce the interference of background point clouds. Similar semantic features improve each other, thereby improving intra-class compactness and semantic consistency.

Channel Attention Branch: Channel attention is crucial for dynamically adjusting feature weights across different channels in the point cloud during learning. It emphasizes the importance of various feature channels, enhancing crack point representation by adjusting each channel’s weight. Important feature channels are highlighted by emphasizing those with significant differentiation for crack points. Meanwhile, redundant feature channels that contribute little to the crack point cloud segmentation task are suppressed, reducing noise interference and improving the model’s overall performance and segmentation accuracy for cracks.

By introducing average pooling and max pooling operations, we integrated the spatial information of the feature map, producing two independent spatial context descriptors named AvgPool and MaxPool, representing average and max pooling features, respectively. These descriptors are then fed into a shared network consisting of a multilayer perceptron (MLP) with hidden layers. To balance parameter count and effectiveness, we set the hidden activation size to ${\mathbb{R}}^{{\displaystyle \frac{C}{r}}\times 1\times 1}$, where r is the decay ratio. After applying the shared network to each descriptor and processing the merged features through the Sigmoid activation function, we obtain the channel attention map $H_{att}$. This map generation relies on learning a shared network with relatively few parameters, reducing computational load. Channel attention is calculated as follows:

$$H_{att}=\mathrm{sigmoid}(\mathrm{MLP}(\mathrm{AvgPool}(A))+\mathrm{MLP}(\mathrm{MaxPool}(A)))$$

(3)

This channel attention design enables the model to adaptively focus on each channel’s information based on task requirements, thereby enhancing the network’s sensitivity to point cloud features. This mechanism effectively captures inter-channel relationships in point clouds, offering a more accurate feature representation for segmentation tasks and improving the model’s performance and generalization capabilities.

Finally, by integrating the channel attention module and the spatial attention module $F\in {\mathbb{R}}^{B\times C\times N}$ through matrix concatenation, the model can learn the relationship between the channel and the location more comprehensively so that the model can understand and represent the details and global information of pavement cracks more comprehensively. The synergistic effect of these two attention modules allows the model to more effectively comprehend the combination of different channels at various positions within the point cloud. This capability enables the model to adapt more efficiently to cracks of diverse shapes, sizes, and positions, ensuring stable performance across different types of cracks. Additionally, this synergy enhances the model’s ability to accurately locate and segment cracks, significantly improving the precision and accuracy of segmentation tasks. By understanding the spatial and channel relationships within the point cloud, the model can deliver more reliable and consistent results in crack detection and analysis. In summary, this fusion strategy provides a more comprehensive and flexible feature learning mechanism for point cloud segmentation tasks.

$$F_{att}=E_{att}+H_{att}$$

(4)

3.3. Poly Loss Function

In the original PointNet++ framework, NLL Loss (Negative Log Likelihood Loss) is used as a standard loss function and is calculated based on probability distribution. By computing the negative log-likelihood between the predicted class probability distribution of pixels or points and the true label, the model is guided to optimize. However, for fracture point cloud segmentation tasks, NLL Loss overlooks the local structure of fractures and the order of points, leading to a lack of global and local structural information and poor robustness against point cloud rotations or translations.

Given the characteristics of pavement crack point cloud data, there is an imbalance in the distribution of crack sample points and background points. To effectively address this issue in pavement crack segmentation, this study introduced Poly Loss [59]. The core idea of Poly Loss is to enhance the deep learning model’s robustness and accuracy by designing specific loss functions tailored to the segmentation task. In the point cloud pavement crack segmentation task, cracks usually occupy a small part of the point cloud, while most of the area is normal pavement, and the data imbalance will cause the original loss function to be unable to effectively identify the crack area. By introducing weighting factors or polynomial terms, Poly Loss effectively addresses the imbalance between crack and background points, significantly enhancing crack region identification accuracy. It makes the model more sensitive to edge and detail information, allowing better preservation and recognition of cracks’ fine structure. By adjusting polynomial coefficients, Poly Loss optimizes the model’s predictive performance for crack point cloud segmentation, in line with the task requirements for pavement crack point cloud segmentation. We must balance the prediction ability of crack point and background point of the model to improve the overall model performance. Poly Loss is defined as follows:

$$L_{\mathrm{Poly}-1}=-\log (P_t)+{ϵ}_1(1-P_t)$$

(5)

Among them, ${ϵ}_1$ is an additional hyperparameter used to adjust the first polynomial coefficient, and $P_t$ is the predicted probability of the crack point category. By introducing an additional term ${ϵ}_1(1-P_t)$ into the original cross-entropy loss function $-\log \left(p_t\right)$ to adjust the first polynomial coefficient, the classification performance is improved. At the same time, the softmax operation is introduced to effectively deal with the problem of category imbalance and strengthen the learning of minority categories.

In this paper, we employed the Poly Loss function to train the entire network architecture. For the semantic segmentation of pavement cracks in point cloud data, Poly Loss effectively addresses category imbalance by adjusting polynomial coefficients to weight crack and background points. This approach enhances the model’s robustness against noise and outliers, significantly boosting performance. It also improves the model’s stability and reliability in complex scenarios.

4. Results

4.1. Implementation

This study used a Ubuntu 20.04 operating system, python v3.8.18, pytorch v1.12.1 deep learning framework, and CUDA v11.5.50. The CPU is Intel(R) Xeon(R) Gold 6133 CPU @ 2.50 GHz, and the GPU is NVIDIA GeForce RTX 3090. All models were trained 300 times, the initial learning rate was 0.001, each training epoch updated the learning rate, the optimizer was Adam, and the batch size was 4. The hyperparameters involved in the Point Attention Net model are shown in

Table 1. Point Attention Net model parameter settings.

Parameter	Value
Learning rate	0.001
Batch size	4
Optimizer	Adam
Momentum parameter	0.9
Training cycle	300

. The proposed Point Attention Net model was trained using LNTU-RDD-LiDAR Road-1 and Road-4, verified using LNTU-RDD-LiDAR Road-3, and tested using LNTU-RDD-LiDAR Road-2.

4.2. Quantitative Assessment Measures

To comprehensively evaluate the effectiveness of construction disaster detection, this study used $IoU$, $mIoU$,${ P}_{re}$,${ R}_{ec}$, and F₁ as the main evaluation metrics. Here, $P_{re}$ calculates the percentage of correctly predicted pavement cracks to assess model effectiveness, as shown in Equation (6). $R_{ec}$ calculates the ratio of correctly identified crack points among all crack points to evaluate detection completeness, as shown in Equation (7). F₁ is the comprehensive evaluation index of $P_{re}$ and $R_{ec}$, as shown in Formula (8). $TP$ represents the number of point clouds in the fracture area correctly identified; $FP$ represents the number of point clouds incorrectly identified as fractures, and $FN$ represents the number of fracture point clouds not identified.

$$P_{re}={\displaystyle \frac{TP}{TP+FP}}$$

(6)

$$R_{ec}={\displaystyle \frac{TP}{TP+FN}}$$

(7)

$$F_1={\displaystyle \frac{2\times P_{re}\times R_{ec}}{P_{re}+R_{ec}}}$$

(8)

$mIoU$ is an indicator used to evaluate the performance of deep learning point cloud segmentation tasks. Its calculation method is shown in Equation (10), where N represents the number of categories. Among them, $IoU$ is only used as an evaluation index for each crack category segmentation, such as Equation (9).

$$Io{U}_i={\displaystyle \frac{T{P}_i}{T{P}_i+F{P}_i+F{N}_i}}$$

(9)

$$mIoU={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N\frac{T{P}_i}{T{P}_i+F{P}_i+F{N}_i}}$$

(10)

4.3. LNTU-RDD-LiDAR Dataset

In similar studies, most pavement crack information extraction methods focus on a single crack type, leading to incomplete crack type coverage. To address this, we developed a pavement crack point cloud dataset named LNTU-RDD-LiDAR for the segmentation task of pavement cracks on point cloud data. Provincial roads served as the collection source, where varying weather conditions and terrain changes resulted in a wider variety of surface cracks. This diversity makes regular, timely, and comprehensive road maintenance more challenging.

Collect: In the data acquisition stage, a laser WPL7-808-10W was used as a point cloud scanner, and combined with an airborne laser measurement system, the road point cloud was recorded to construct the original point cloud dataset. The airborne laser measurement system consists of two WPL7-808-10W lasers, two 3D AT-C2 cameras, and an inertial navigation unit. The clear distance between the bottom of the laser sensor and the road surface was 1900 mm. The point cloud measurement covers a range of 4 m to 2.1 m, the ranging accuracy reaches 0.5 mm, the absolute accuracy is 10 mm, and the elevation error between lanes is only 3 mm. The selection of these scanning parameters and settings was designed to ensure that the obtained point cloud data reached an accuracy level of 0.01 mm in resolution. The entire experimental dataset contains 188 pavement point cloud segments, each of which contains an average of 20 million points. Each segment has a width of 3.8 m and a length of about 16 m.

Table 2. Parameters of 3D laser road acquisition equipment.

Hyperparameter	Range
Point cloud resolution	0.01 mm
Clear distance between instruments and ground	1900 mm
Point cloud measurement coverage	4 m–2.1 m
Point cloud ranging accuracy	0.5 mm
Point cloud absolute accuracy	1 cm
Elevation error between lanes	3 mm
Use environment	Free from ambient light all day long
Detection speed	40 km/h

shows the size and high resolution of the dataset that provides detailed and comprehensive pavement information for this study, which lays a solid foundation for subsequent crack detection and analysis.

Mark: After collecting and processing the pavement point cloud data through the airborne laser measurement system, a pavement crack point cloud labeling task was carried out. The point cloud labeling tool CloudCompare_v2.13 is used in this paper. CloudCompare_v2.13 is a powerful open-source point cloud data processing software with rich point cloud processing functions, including import, export, and editing. Its intuitive 3D visual interface makes visual analysis and interactive operation more convenient. CloudCompare_v2.13 can efficiently process and annotate point cloud data, which provides a good data foundation for subsequent tasks.

Table 3. LNTU_RDD_PCD dataset description table.

Hyperparameter		Range
Dataset size	Point cloud dimension	3D
	Dataset size	713 GB
	Number of datasets	188
Point cloud attribute	Position coordinate	x, y, z
	Color	R, G, B
	Intensity	Intensity
	Category	Label
	Normal vector	Nx, Ny, Nz
Labeling information	Class tag	Pavement point, crack point
	Annotation method	Manual labeling
Dataset partitioning	Number of training sets	127
	Number of verification sets	21
	Number of test sets	20
	Partition method	Random sampling

shows the detailed requirements of the labeling task, including the accurate location, shape, and size of the cracks. The whole labeling process covered 188 crack point cloud pavement segments, among which the ratio of crack point to non-crack point was about 0.6:9.4. Then, the LNTU-RDD-LiDAR experimental data were divided into four subsets: Road-1, Road-2, Road-3, and Road-4, among which Road-1 contains 51 pavement segments. Road-2 contains the latter 17 road segments, Road-3 contains 59 road segments, and Road-4 contains 61 road segments. At the same time, the background points were also marked to consider the overall road surface information. This provides a high-quality labeled dataset for semantic segmentation of pavement crack point cloud-type data, as illustrated in Figure 3. Through multi-person verification and quality control, the accuracy and consistency of the annotation results were ensured, which provides a reliable basis for the training and evaluation of the model.

4.4. Comparative Experiment

To verify the effectiveness of the proposed Point Attention Net model for pavement crack point cloud data segmentation, this study tested the LNTU-RDD-LiDAR point cloud dataset and compared the model with existing methods, including PointNet [40], Pointnet++ [41], Point Transformer [47], and PointMLP [50]. PointNet, a pioneer of point-by-point classification, uses a symmetric function invariant to permutations but struggles to learn local features in complex road scenarios. Point Transformer introduces a self-attention mechanism to capture the spatial structure of point clouds and establish a global context. PointMLP incorporates a lightweight geometric affine module to enhance performance.

From

Table 4. Comparison of semantic segmentation results between PAN and PointNet, PointNet++, PointTransformer, and PointMLP on the LNTU-RDD-LiDAR dataset.

Model	${\mathit{R}}_{\mathit{e}\mathit{c}}$ ↑	$\mathit{F}1$ ↑	$\mathit{A}\mathit{c}\mathit{c}$ ↑	$\mathit{m}\mathit{I}\mathit{o}\mathit{U}$ ↑	Param.
PointNet	48.1	64.9	89.1	68.4	3.5 M
PointNet++	59.4	70.4	90.4	69.2	1.41 M
PointMLP	61.3	73.6	91.4	74.3	12.6 M
Point Transformer	62.2	74.1	91.5	74.5	7.8 M
PAN	67.1	75.4	91.5	75.4	1.76 M

, it is clear that the PAN model improves in mIoU, ${\mathrm{R}}_{\mathrm{e}\mathrm{c}}$, F1, and Acc compared to the PointNet, PointNet++, Point Transformer, and PointMLP models. mIoU improved by 0.9% relative to the Point Transformer model, Acc improved by 0.1% relative to the PointMLP model, and F1 improved by 1.3% relative to the Point Transformer model. As illustrated in Figure 4, experimental results show that, compared with most traditional deep learning algorithms, the Point Attention Net model mainly benefits from two-branch attention fusion, which enhances the ability of description feature coding.

4.5. Ablation Experiments

PC-Parallel:

Table 5. Ablation experiments of PC-Att-Parallel module test on LNTU-RDD-LiDAR dataset.

Model	${\mathit{R}}_{\mathit{e}\mathit{c}}$ ↑	$\mathit{F}1$ ↑	$\mathit{A}\mathit{c}\mathit{c}$ ↑	$\mathit{m}\mathit{I}\mathit{o}\mathit{U}$ ↑
BaseNet	48.1	64.9	89.1	68.4
+Position Attention	56.2	70.7	90.9	72.2
+Channel Attention	57.9	70.8	90.7	72.1
+PC-Parallel	67.1	75.4	91.5	75.4

shows the results of the gradual improvement of pavement crack information extraction performance with the increase of attention blocks. Compared with the original Pointnet ++ baseline network, the introduction of location attention module and channel attention module can significantly improve the segmentation effect of pavement crack point cloud. In this case, mIoU reached 75.4%, $R_{ec}$ reached 67.1%, F1 reached 75.4%, and Acc reached 91.5%.

As shown in Figure 5, incorporating the PC-Parallel module effectively addresses issues of incomplete segmentation and imprecise boundaries. The location attention module learns spatial interdependencies between features, while the channel attention module captures dependencies between channels. Both attention blocks positively impact pavement crack extraction performance. Introducing these modules and fusing their features significantly enhances performance. Consequently, the two-branch attention block demonstrates remarkable effectiveness in the crack segmentation task of pavement point cloud data.

Poly Loss: 

Table 6. Ablation experiments of Poly Loss test on LNTU-RDD-LiDAR dataset.

Method	${\mathit{R}}_{\mathit{e}\mathit{c}}$ ↑	$\mathit{F}1$ ↑	$\mathit{A}\mathit{c}\mathit{c}$ ↑	$\mathit{m}\mathit{I}\mathit{o}\mathit{U}$ ↑
BaseNet	48.1	64.9	89.1	68.4
+Poly Loss	63.3	73.5	91.1	74.0

shows the results of the gradual improvement of pavement crack information extraction performance with the increase of loss function. Compared to the original PointNet++ baseline code, the introduction of Poly Loss to handle class imbalance and focus on difficult-to-classify samples effectively improves the performance and robustness of the model. In this case, mIoU reached 74.0%, ${\mathrm{R}}_{\mathrm{e}\mathrm{c}}$ reached 63.3%, F1 reached 73.5%, and Acc reached 91.1%. Therefore, Poly Loss shows remarkable effectiveness in the pavement point cloud data crack segmentation task. As one can see from Figure 6, replacing the Loss function in the original PointNet++ baseline code with Poly Loss effectively solves the class imbalance problem. The loss function highlights the features of foreground points and background points so that the model can better distinguish the slit points and other points.

From Figure 6, replacing the Loss function in the original PointNet++ baseline code with Poly Loss effectively solves the class imbalance problem. The loss function highlights the features of foreground points and background points so that the model can better distinguish the slit points and other points.

5. Discussion

In this paper, the 3D laser point cloud pavement crack data have better robustness than a 2D image under varying illumination conditions and in a low-intensity contrast environment. It can effectively deal with various kinds of rust and oil covering the road surface. Experiments on the proposed model show that the indexes of mIoU, Acc, F1, and ${\mathrm{R}}_{\mathrm{e}\mathrm{c}}$ are significantly improved, which confirms the effectiveness of the proposed method compared with traditional methods. The introduction of the Poly Loss function helps to better capture the edges of cracks and details.

However, the dataset in this paper includes a large number of 3D point cloud data of pavement cracks collected under clear-weather conditions. However, these data do not fully cover all environmental conditions. On rainy and snowy days, water and snow can mask pavement cracks, making it difficult for laser scans to capture precise crack features. In addition, point cloud data may introduce additional noise, reducing the detection effect and recognition ability of the model.

To overcome these limitations and further explore the application of 3D point cloud datasets in pavement crack segmentation tasks, future research will focus on the following aspects:

1. Expanding datasets and multi-source data integration: We plan to collect more samples under various environmental conditions to enhance the generalization ability of the model. By combining other sensor data such as RGB images and thermal imaging, the deficiency of point cloud data can be supplemented to provide richer environmental information. In addition, we will develop automated data-labeling tools to automatically generate annotated data using deep learning and machine learning algorithms. This will help speed up labeling, reduce reliance on manual labeling, and improve the consistency and accuracy of labeling;

2. Technology integration and practical application: We will study how to seamlessly integrate the technology into the existing monitoring system and apply it to the actual road maintenance scenario. This includes addressing compatibility issues and adding new features without affecting existing system functionality. To ensure the effectiveness of the technology in practical scenarios, we plan to promote the application in cooperation with experts in relevant fields. This will help us identify and address potential implementation issues and optimize the technology based on actual needs. At the same time, we will evaluate implementation and maintenance costs to design solutions that are both efficient and cost-effective to support a wide range of applications.

Through these efforts, we hope to significantly improve the usefulness of the dataset and the applicability of the technology, providing a solid foundation for future road monitoring and maintenance.

6. Conclusions

This paper introduces the Point Attention Net (PAN) network for extracting 3D pavement crack information, aiming to overcome the limitations of previous point cloud-based deep learning approaches. The PAN network incorporates a novel PC-Parallel module that was specifically designed to learn spatial interdependencies and channel dependencies of features separately. This design significantly enhanced the performance of point cloud pavement crack segmentation by allowing the network to more effectively capture and process the intricate details of crack features. Additionally, boundary refinement and class imbalance issues in point cloud pavement crack segmentation were addressed by introducing the Poly Loss function. The test results on the LNTU-RDD-LiDAR dataset show that the proposed method has excellent performance on mIoU, ${\mathrm{R}}_{\mathrm{e}\mathrm{c}}$, F1, and Acc, reaching 75.4%, 75.4%, 67.1%, and 86.8%, respectively. In comparison to existing point cloud segmentation methods, the proposed approach demonstrates superior performance, as the proposed method improved the mIoU and F1 indexes by 1.1% and 1.3%, respectively. The experimental results demonstrate that the proposed method significantly enhances point cloud pavement crack segmentation.

In future studies, we will explore more effective class-balancing strategies to enhance the model’s performance and generalization on unbalanced datasets such as highway pavement crack point cloud data. Additionally, we aim to increase the number of samples and scene categories, expanding the model’s training set to encompass a broader range of practical situations, thereby improving its generalization ability when encountering unknown data.