Study of the Effect of Exploiting 3D Semantic Segmentation in LiDAR Odometry

Round 1

Reviewer 1 Report

The paper studies the benefits of semantic segmentation of LiDAR point clouds in various environments using KITTI dataset

1- Abbreviations should be defined before the first time they are used such as LiDAR, SLAM, …

2- I suggest modifying the caption of Figure 1 by removing “and subsequently fed to the LiDAR odometry algorithm”

3- If possible, change the background of the point cloud in Figure 1 to white so that the point cloud would be clear.

4- The English language should be extensively improved.

5- What do you mean by the number of occlusions on line 100?

6- What is your interpretation of the very large errors in Table 1 that are in the order of hundreds of meters such as in urban-F? The same applies to Table 2.

7- What is the criteria for removing outliers to create figure 3? 

Author Response

The paper studies the benefits of semantic segmentation of LiDAR point clouds in various environments using KITTI dataset

We would like to thank the reviewer for the comments and suggestions that have allowed us to improve the quality of the article as well as to correct the errors of the first version. We hope that this new version will meet the expectations of the reviewer.

1- Abbreviations should be defined before the first time they are used such as LiDAR, SLAM, …

We appreciate the help of the reviewer, by pointing out this drawback in the previous version. Following the advice of the reviewer, we have addressed this problem and added the meaning of the missing abbreviations.

2- I suggest modifying the caption of Figure 1 by removing “and subsequently fed to the LiDAR odometry algorithm”

Following the suggestion of the reviewer we have modified the caption of Figure 1, removing the text indicated.

3- If possible, change the background of the point cloud in Figure 1 to white so that the point cloud would be clear.

We understand and share the concern of the reviewer regarding to this point. However, after trying some different background options, we have found that setting a white background produces an even worse visualization, due to the lack of contrast between the background and the light points. On the other hand, the color of the points comes “predefined” by the labels of the Semantic-KITTI dataset and are not easy to modify. Subsequently we had no option but to leave the original color. We hope that the reviewer will understand this decision...

4- The English language should be extensively improved.

Following the advice of the reviewer, the text has been deeply reviewed.

5- What do you mean by the number of occlusions on line 100?

We appreciate the help of the reviewer by pointing out this misinformation in the text. In the original version, we included occlusions as a factor to take into account , as the pedestrians can be occluded in subsequent frames, thus they may not be valid as reference points. However we agree that this assumption is rather misleading, and we decided to remove it from the text.

6- What is your interpretation of the very large errors in Table 1 that are in the order of hundreds of meters such as in urban-F? The same applies to Table 2.

We agree with the reviewer that the explanation for these two tables may be scarce. In order to correct this error, the following explanations were added. 

“Because of the nature of the APE metric ,table 1 ( the obtained values depend on the length the sequence, this means that a larger sequence will have larger error due to the drift error associated to these algorithms...” and for table 2“... As it is shown in table2, RPE error results are quite smaller as it measures the mean values along meters of sequence”. These explanations were added in the text and are highlighted in yellow in the new manuscript.

7- What is the criteria for removing outliers to create figure 3?

The reason of removing the outliers is the aforementioned large errors in some of the sequences. These large numbers cannot be represented in Figure 3, because that will fill the Figure with outliers and the actual box plots would be impossible to visualize. Subsequently, the big outliers were removed until the box plots were properly visible. In order to clarify this point, the sentence mentioning the outliers have been rephrased. We hope that this responds to the doubts of the reviewer.

Reviewer 2 Report

My big concern with this paper is that I am not sure it argues for it's own value well enough. Though the conclusion states there are important differences between several filtering methods, and that they demonstrably do better than the reference/raw point cloud utilization, I do not see that in the actual results. Part of this might be due to Figure 3, which, due to the "static/F" method, has poor resolution for the other methods--thus I cannot see that there is an actual significant difference between methods A,B and C. A "zoomed in" look at methods A-E in a Figure 3.2 would be most helpful here, as would some way of refiguring Tables 1 and 2 to highlight the better methods in a bit more organized fashion (eg a table 3 grouped by best results for APE and RPE). 

In general, my main concern is with significance. If you can do a better job indicating why this study is important, and why the results are significant, it will help a lot. 

Author Response

My big concern with this paper is that I am not sure it argues for it's own value well enough. Though the conclusion states there are important differences between several filtering methods, and that they demonstrably do better than the reference/raw point cloud utilization, I do not see that in the actual results. Part of this might be due to Figure 3, which, due to the "static/F" method, has poor resolution for the other methods--thus I cannot see that there is an actual significant difference between methods A,B and C. A "zoomed in" look at methods A-E in a Figure 3.2 would be most helpful here, as would some way of refiguring Tables 1 and 2 to highlight the better methods in a bit more organized fashion (eg a table 3 grouped by best results for APE and RPE). 

In general, my main concern is with significance. If you can do a better job indicating why this study is important, and why the results are significant, it will help a lot.

Comments to reviewer:

We would like to thank the reviewer for the comments and suggestions that have allowed us to improve the quality of the article as well as to correct the errors of the first version. We hope that this new version will meet his expectations.

We appreciate your comments about the figures and tables which help us to improve the quality of the paper. It has been difficult to find an efficient way to show the results due to the large amount of data collected from the experiments.

Regarding Figure 3, we appreciate the suggestion of the reviewer, but even if we “zoom in”, methods A, B and C will are still very similar. This is probably due to the amount of filtered points in configurations B and C being small, as we mentioned in the “Discussion” section (4.2). This is also shown in Figure 2. In order to deeply compare these configurations, it is better to analyze the data from tables 1 and 2. In order to clarify this point we included the following sentence after the figure, in section 4.2:

“In the figure it can be observed that while configurations A , B and C provide similar results (as it shown in tables 1 and 3) the difference among the rest is visible.”

Furthermore, we are thankful for the idea of a third table that groups the best results for APE and RPE and we have included it in the new version of the manuscript.

Finally, following the concerns of the reviewer, we have modified the manuscript to better detail the purpose of this study and why it is important for the research community by adding further explanations in the introduction section.

Round 2

Reviewer 1 Report

The authors have replied to all the comments and the revised manuscript is good.