In the experiment, the mean relative pose error (mRPE) was employed to evaluate the accuracy of the maps. mRPE includes two indexes: the mean relative translation error and the mean relative rotation error. The mean relative translation error reflects the displacement deviation between the constructed map and the ground truth in the horizontal direction. The mean relative rotation error reflects the angular deviation between the constructed map and the ground truth of the map in the vertical direction. To ensure the fairness of the comparison, all the performance and precision results shown in each experiment were averaged over three trials of playback of each dataset.
4.2.1. KITTI Dataset
Two experiments were run on the KITTI dataset to demonstrate the proposed LoNiC method’s efficacy in general scenarios. The first group of experiments was conducted to verify the effectiveness of the odometry method, and the second group verified the performance of the SLAM algorithm.
In order to verify the advantages of introducing constraints on neighborhood information into the odometry, we designed an odometry ablation experiment on the general KITTI dataset. Since the back-end part of LoNiC adopts the same strategy as LIO-SAM, a comparison of the proposed LoNiC method with the LIO-SAM was conducted. The major difference between LoNiC and LIO-SAM lies in their odometry. The LIO-SAM method divides the feature points into the edge and planar points and employs the distance information to constrain the registration, while the LoNiC method adopts the odometry method based on neighborhood information constraints, as discussed in
Section 3.
Table 3 exhibits the relative pose errors of LIO-SAM and LONIC on the KITTI dataset. As can be seen, the LoNiC method proposed outperforms the upper LIO-SAM on the KITTI dataset, especially in terms of mapping accuracy, which can reach the state of the art (SOTA) level. Specifically, the mean relative translation error and the mean relative rotation error of the LoNiC method on nine data sequences are 2.9742% and 0.0095°/m, which are 0.0782% and 0.0095°/m lower than that of the LIO-SAM method, respectively. Therefore, the mapping accuracy of the LoNiC method is better than that of the LIO-SAM method, especially in 01 and 09 sequences. According to the ground truth of sequence 01 in the KITTI dataset, the mapping trajectory of LIO-SAM and LoNiC displayed in
Figure 11, the mapping trajectory of LoNiC is closer to the ground truth without loop optimization, indicating that the odometry of the LoNiC method is better than that of LIO-SAM, and the mapping results are more accurate. The feature points extracted by the odometry of LIO-SAM are more appropriate for the mapping of structured roads, while the description of complicated conditions exhibits unsatisfactory performance. Conversely, the proposed LoNiC obtains feature points with surface information with the angle distribution of neighborhood normal vectors, effectively illustrating the rugged terrain and raising the mapping accuracy. Similarly, according to the ground truth of sequence 09 in the KITTI dataset, the mapping trajectory of LIO-SAM, and the mapping trajectory of LoNiC displayed in
Figure 12, the experimental results of the LoNiC method are better than those of LIO-SAM with loop closure detection (at the starting position). We analyzed the ground truth of these nine sections of data and found that the elevation fluctuation of the 01 and 09 sequences was more obvious than in other sequences. The result further demonstrates the effectiveness of the feature extraction scheme and constraint method proposed in this paper, which can lessen the mis-registration of feature points and improve the precision of the odometry.
- 2.
Comparative Experiment
This paper compared the proposed LoNiC with another SOTA method called LeGO-LOAM on the KITTI dataset to verify the mapping ability and effect of the LoNiC algorithm and evaluated the mapping results by using the mean relative pose error (mRPE).
According to
Table 4, the mean relative rotation error and mean relative translation error of LoNiC are significantly better than those of LeGO-LOAM in the nine sections of data except for the 06 and 08 sequences. The possible reason why the error of sequences 06 and 08 of LoNiC is slightly worse than that of LeGO-LOAM is that the clustering strategy in LeGO-LOAM provides extra label information in the process of point cloud registration. Additionally, a combination of features helps to obtain better results. The mean translation error of LeGO-LOAM on these nine sections of data is 7.4884%, which is 4.5142% lower than that of LoNiC, being 2.9741%; the mean rotation error of LeGO-LOAM is 0.0099°/m, while that of LoNiC is 0.0094°/m, which reduces by 4.04% compared with LeGO-LOAM. This result indicates that LoNiC can reduce the cumulative errors of mapping. The actual mapping effect of sequences 05 and 07 are shown in
Figure 13. According to the point cloud images, our map has no obvious ghosting and drift, indicating that the LoNiC algorithm has excellent mapping ability in general scenes. According to the mapping trajectories of sequences 05 and 07 on the KITTI dataset shown in
Figure 14, the trajectories estimated by the LoNiC method proposed in this paper are closer to the ground truth compared with LeGO-LOAM. The reason is that LeGO-LOAM just adopts a single distance constraint to identify adjacent frame point cloud data and ignores reliable feature correspondence in complicated environments. Differently, our method employs a multi-scale constraint description to enhance the discrimination of the difference of point cloud data between frames and improve the reliability of feature correspondence. Experimental results suggest that the precision of translation and rotation estimation can be effectively improved by introducing neighborhood information constraints in the SLAM framework.
4.2.2. Experiments on the JLU Dataset
In order to verify the performance of the method proposed in this paper, we collected data from four sections of rugged roads that contained undulating roads on the campus of Jilin University. The mean relative pose error (mRPE), which was used to evaluate the experiment results, was employed on the JLU dataset in the same way that it was on the KITTI dataset.
First, we contrasted the odometry of LoNiC and LIO-SAM to demonstrate the superior performance of odometry in rugged terrains.
Table 5 demonstrates that LoNiC performs better than LIO-SAM in most sequences on the JLU campus dataset, with a slightly worse relative rotation error than LIO-SAM on JLU-062801. The mean relative translation error of LoNiC is 3.3061%, which is 0.4317% lower than that of LIO-SAM; the mean relative rotation error of LoNiC is 0.0173°/m, which is 19.53% lower than that of LIO-SAM. This is because the road surface is bumpy in the region of data acquisition, while LIO-SAM is based on the premise of structural roads without consideration of the changes caused by point cloud pitching. Therefore, the odometry based on neighborhood information constraints has stronger discrimination against the difference between feature points of adjacent frames than the traditional odometry based on distance information constraints, thus improving the precision of point cloud registration. As a result, the LoNiC method has more advantages on the JLU campus dataset than the KITTI dataset. According to the trajectory of sequence JLU_070101 on the JLU campus dataset shown in
Figure 15, there is no obvious drift of the odometry in both LoNiC and LIO-SAM when the vehicle starts to move. However, the cumulative errors of the LIO-SAM method gradually rise over time and deviate from the ground truth, while the LoNiC method still shows no obvious drift. Based on this, it can be concluded that the LoNiC method proposed in this paper can effectively reduce the cumulative errors of mapping compared with the odometry method based on LIO-SAM, and the figure obtained by the estimated trajectory is the closest to the ground truth.
To better prove the performance of LoNiC, three sections of rugged roads in the sequence JLU_070101 were selected for display, and their positions are shown in
Figure 15 according to the a, b, and c of the trajectory diagram. Each section is shown in three small figures. The first figure displays the comparison of elevation fluctuation trend of the current road section; the second figure displays the comparison of horizontal mapping trajectories of the current road section; the third figure displays the mapping performance of the current road section. According to
Figure 15a, the elevation change trend estimated by the LoNiC algorithm and the LIO-SAM method in the first half section of the data is far from the ground truth. The possible reason is that the first half section of the data is characterized by many trees and no obvious buildings, so it is difficult to extract valuable feature points, particularly in the case of vehicle undulation. However, in the second half of the data, the surface features increase as the vehicle passes through the teaching buildings, and the LoNiC algorithm gradually fits the ground truth, while the elevation estimation of the LIO-SAM method still deviates from the ground truth, indicating an advantage in the feature discrimination of the LoNiC method proposed in this paper. Additionally, as shown in the mapping trajectory, considering that point a is at the end of data acquisition, which is the most significant position of drift for mapping algorithms, the results of the LoNiC method are closer to the ground truth compared with the LIO-SAM method although the LoNiC algorithm has a certain degree of cumulative errors. This suggests that the LoNiC method can ensure the precision of the odometry for a longer time. The road section corresponding to
Figure 15b is relatively narrow and long, with buildings on both sides. Theoretically, it is more suitable for the surface feature extraction in the LIO-SAM method. However, the undulation of the data due to the rugged terrains interferes with the corresponding relationship between the feature points, resulting in mis-registration and ultimately leading to large errors in the elevation estimation. The multiple constraints adopted in the LoNiC method can lessen the possibility of mis-registration to a certain extent, thus leading to smaller errors as well as a fluctuation trend that is more similar to the actual situation from the perspective of results. Compared with the first two sections, the road section corresponding to
Figure 15c has a wider surrounding environment and fewer reference landmark buildings or objects. Therefore, the performance of the two algorithms is similar to that of the first half of
Figure 15a. Although the result of the LoNiC method is close to the ground truth, there is still a gap. In conclusion, the elevation trend and horizontal mapping trajectory of the LIO-SAM method present greater drift in these three sections on the whole, while the results of the LoNiC method are closer to the ground truth. Hence, it can be concluded that feature points with surface information can enhance the scene description, and multi-scale constraint description can effectively reduce the mis-registration of feature points, better correct the errors in elevation and horizontal direction in the case of road fluctuation, and further improve the overall mapping precision.
- 2.
Comparative Experiment
The LoNiC and LeGO-LOAM methods are compared on the JLU campus dataset in this section to verify the excellent registration precision and mapping effect of the LoNiC method in rugged scenes.
Table 6 demonstrates that the LoNiC method significantly improved over the LeGO-LOAM method in the mRPE results on the JLU campus dataset, including the sequences of JLU_062802, JLU_070701, and JLU_070702. Specifically, the mean relative translation error of the LoNiC method is 2.8744%, while that of the LeGO-LOAM method is 4.0143%, realizing an improvement of 1.1399%. Similarly, the mean relative rotation error of the LeGO-LOAM method is 0.0230°/m, while that of the LoNiC method is 0.0173°/m, reducing the error by 24.78% compared with LeGO-LOAM. Additionally, the LoNiC method performs much better in the JLU_070701 and JLU_070702 sequences than the LeGO-LOAM method, indicating that the LoNiC method has a greater registration precision for long sequences. The feature points with surface information extracted by LoNiC are more discriminative in complex environments, which effectively improve the accuracy of point cloud registration compared with the corner points and surface points extracted by LeGO-LOAM. With the gradual growth of the motion trajectory, the cumulative error generated by LeGO-LOAM grows quickly compared with that of LoNiC, which indicates that LoNiC outperforms LeGO-LOAM on long sequences. The actual effect of the LoNiC method on the JLU_062801 and JLU_070101 sequences in the JLU campus dataset are displayed in
Figure 16 and
Figure 17. The following three images are the bird’s-eye view (BEV) image, the point cloud image generated by LoNiC, and the superimposed renderings of the BEV image and the point cloud image. The two superimposed renderings suggest that the point cloud map generated by the proposed LoNiC algorithm can be highly coincident with the actual scenes, whether it is a short-range mapping (≤1 km) or a long-range mapping (≥2 km), which further proves the excellent mapping effect of the LoNiC algorithm.